8

McGrath: Red Hat’s commitment to open source

 1 year ago
source link: https://lwn.net/Articles/936405/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

McGrath: Red Hat’s commitment to open source

[Posted June 26, 2023 by corbet]
Red Hat's Mike McGrath responds to the many criticisms aimed at the company since it changed its policy regarding RHEL source code.
Ultimately, we do not find value in a RHEL rebuild and we are not under any obligation to make things easier for rebuilders; this is our call to make. That brings me to CentOS Stream, of which there is immense confusion. I acknowledge that this is a change in a longstanding tradition where we went above and beyond, and change like this can cause some confusion. That confusion manifested as accusations about us going closed-source and about alleged GPL violations. There is CentOS Stream the binary deliverable, and CentOS Stream the source repository. The CentOS Stream gitlab source is where we build RHEL releases, in the open for all to see. To call RHEL “closed source” is categorically untrue and inaccurate. CentOS Stream moves faster than RHEL, so it might not be on HEAD, but the code is there. If you can’t find it, it’s a bug – please let us know.

(Log in to post comments)

A note to commenters

Posted Jun 26, 2023 19:38 UTC (Mon) by corbet (editor, #1) [Link]

This topic has generated a lot of discussion recently; I suspect that will be the case for this posting as well. To make life easier for everybody, can we please try to avoid rehashing other recent debates and only post here if we truly have something new to add?

Thank you.

McGrath: Red Hat’s commitment to open source

Posted Jun 26, 2023 20:12 UTC (Mon) by mb (subscriber, #50428) [Link]

> I feel that much of the anger from our recent decision around the downstream
> sources comes from either those who do not want to pay for the time, effort and
> resources going into RHEL

I guess that means RH is now going to pay all upstream developers for the time, effort and ressources going into it, right?

> Simply rebuilding code, without adding value or changing it in any way, represents
> a real threat to open source companies everywhere.

Maybe. But it is fully compliant to the licenses that upstream decided to apply to their software.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 6:13 UTC (Tue) by oldtomas (guest, #72579) [Link]

> I guess that means RH is now going to pay all upstream developers for the time, effort and ressources going into it, right?

They have been doing it all along. By employing contributors to key projects. By sponsoring important initatives (e.g sourceware.org). If all commercial companies involved in free software behaved like RedHat, we'd be in a better place.

No, I don't work for RedHat. I don't even use their product (I'm more of a Debian type). But I do acknowledge that they have made my life better.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 8:12 UTC (Tue) by TRauMa (guest, #16483) [Link]

I wrote software that's included in Red Hat and I'm still waiting for my cut. Or any contribution. So?

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 8:28 UTC (Tue) by paulj (subscriber, #341) [Link]

Ditto.

I even applied for a job, and the manager who interviewed me it became quickly apparent the only reason they'd arranged the interview was to ask me about the background / politics of a fork of code I was working on. Pretty shit experience.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 14:12 UTC (Tue) by clump (subscriber, #27801) [Link]

Red Hat's hiring process has always been a bit "challenging". It's generally a disorganized process, with a pretty big disconnect between hiring managers and recruiters. When I interviewed people, I learned to first ask "Was the position explained to you?" and "Do you know the role you're applying for?"

What ends up happening is that referrals move to the front of the line because everyone knows each other and it bypasses the typical funnel. This seems pretty consistent with other tech companies but is still disappointing if it significantly disadvantages other applicants.

I don't know about your particular case however I heard frequent complaints about the hiring process from applicants.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 14:22 UTC (Tue) by paulj (subscriber, #341) [Link]

Mine was kind of weird. I was applying for a role that had nothing to do with the free software project I'd worked on - indeed, I was happy to get away from that project. The role was in the same general area of computing as the project, but not related otherwise.

It was clear to me afterwards the manager went into the interview with 0 intention of considering me for the role, but he just did it cause he wanted to ask me about issues with that (unrelated to the role) project. Something I didn't really want to discuss much - certainly not in that setting. Perfunctory questions to query my knowledge, no discussion about the role, and he concluded the interview telling me already had someone for it and sorry - but then the job req was left open for something like 12 months after.

He got my hopes up by scheduling an interview for a role I would have really liked, tbut wasted my time, and - as the months went by and I could see the ad still open - made me feel like he'd treated me like a fool. Just to satisfy his curiosity.

Still a bit annoyed by it.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 14:36 UTC (Tue) by paulj (subscriber, #341) [Link]

Oh, and in telling me he had filled the role, he said my experience and CV were great and it was a shame. So... whatever the actual reason - whether it was just his curiousity, or cause RedHat have some loyalties related to that project (which I wanted to get away from anyway), I felt treated like a fool.

To get back to the higher level point:

If your code is shipped by RedHat in RHEL, to paying customers, as some of mine was for years and years and is to this day I think, but that doesn't mean RedHat will ever support you.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 15:24 UTC (Tue) by paulj (subscriber, #341) [Link]

And to add a higher-level point again:

"Just buy a $WHATEVER_ENTERPRISE_LINUX licence - that's a good way to support open-source developers"

is not per se true.

These "enterprise Linux" corporations do not generally go and find some way to support all the developers of all the software they ship and support, where those developers do not already have some other support (e.g. another employer, or their own consulting). They employ leading developers on high-profile projects that they care about; and they employ maintenance engineers to support all the other stuff, often engineers from outside any of the communities of said software.

And a developer on some project, who really could use some kind of security in terms of employment, that was at least friendly to some OOH or even 10% time on non-core-job free software project maintenance, may be excluded from consideration by enterprise Linux corporations for any number of reasons - from where they live (e.g., not somewhere the corp hires), to industry politics (dev has issues with some other industry corporates, and said Linux corporate has some level or relations with those other industry corporates).

So that idea above, that $WHATEVER_ENTERPRISE_LINUX corps are good ways to help fund FOSS, is not one that I could agree with at all.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 16:37 UTC (Tue) by mb (subscriber, #50428) [Link]

>> I guess that means RH is now going to pay all upstream developers for the time, effort and ressources going into it, right?

>They have been doing it all along.

No. They haven't been paying all developers. By far. And that is perfectly acceptable and fine. It adheres to the licenses that the developers chose for their code.

>By employing contributors to key projects. By sponsoring important initatives (e.g sourceware.org).
>If all commercial companies involved in free software behaved like RedHat, we'd be in a better place.

Yes, I completely agree.
Red Hat is one of the biggest single contributors to Free Software.
But it's also far from paying all of the work that goes into their product.
How much of the work do they pay? 1%, 2%? 10%? Probably not more.

And that is perfectly acceptable and fine. It adheres to the licenses that the developers chose for their code.

What is not Ok is saying things like this:

>> I feel that much of the anger from our recent decision around the downstream
>> sources comes from either those who do not want to pay for the time, effort and
>> resources going into RHEL

because these people are just doing what the licenses permit.
These people are exploiting the *same* rights that Red Hat exploits to run their business. That is the basic concept and foundation of Free Software. These rights are at the base of Red Hat's business. It's what makes their business possible. If GNU/Linux was proprietary software, then they would have to pay for every single change set going into it.

But Free Software allows them to not pay all these bills. And that is fine.

Companies are using my Open Source and Free Software to make millions of Dollars. From where do I know? Because these companies frequently contact me for free support.

And that is perfectly acceptable and fine. It adheres to the licenses that I chose for my code. I knew that from the beginning and chose to go that route.

What would not be Ok is if these companies would put further restrictions on *my* code. E.g. by applying a threat model to stop business with their customers upon redistribution.

McGrath: Red Hat’s commitment to open source

Posted Jun 26, 2023 20:28 UTC (Mon) by davisr (subscriber, #154260) [Link]

> Simply rebuilding code, without adding value or changing it in any way, represents
> a real threat to open source companies everywhere.

My opinion, as an author who sells FLOSS, is that fewer people today are willing to pay for any software at all. Corporations are using mega-scale techniques to (a) sell user data to anyone willing to pay, and (b) entice users with gratis features, but lock them via proprietary/SaaS and slowly make them pay.

Users are coming to expect that software should be gratis. I make a frank appeal to my customers: either they pay for the software, or there will be no software. Pay for RedHat, or there will be no Rocky or Alma Linux. Out of all the companies I've worked for, they all paid next to nothing for their software, and they all eventually got what they paid for.

McGrath: Red Hat’s commitment to open source

Posted Jun 26, 2023 20:53 UTC (Mon) by Depereo (subscriber, #104565) [Link]

Extremely large technology companies are built on the back of unpaid and unappreciated open source software. Linux runs everything; on it sits open source databases, providing query responses to open source web application frameworks frontended by haproxy. The network equipment that connects this all runs on linux or BSD. Communication between all of this relies on OpenSSL, a thing that like, two people were maintaining for a long time without much if any investment.

Exploitation of open source / the commons from large organizations is so publicly and obviously profitable that I'm not surprised no one wants to pay for code. Not paying for labour is how people consolidate wealth, and lots of people want to be wealthy.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 0:41 UTC (Tue) by Paf (subscriber, #91811) [Link]

I’m not going to engage with the second part, but for the first part:

A huge part of the point and benefit of open source software is that it doesn’t have to be paid for. This is the source of enormous benefit to the world at large, and has dramatically reduced duplication of effort and certain kinds of concentration of power. Microsoft is a far less powerful company because of open source software.

This also creates a tragedy of the commons style difficulty getting development paid for. Well, so be it. It is a necessary corollary of that freedom/openness.

McGrath: Red Hat’s commitment to open source

Posted Jun 26, 2023 21:06 UTC (Mon) by kleptog (subscriber, #1183) [Link]

To be honest, this bugs me too. At my work we use a lot of Linux, mostly Debian but we also use all sorts for projects like Rust, Python, Django, VueJS, etc... You can download it all for free and build products on it, but it feels a bit like freeloading. I've tried to suggest that we have budget that we annually distribute as donations to open source projects we use a lot, but it doesn't work with management. Invoices they understand, donation are complex. All they hear "we would need to make lots of little payments to random people/organisations without PO numbers, that sounds like work and nothing goes wrong if we don't do it".

I've managed to arrange that if we find bugs, that we get time to figure the root cause, write a patch and submit it upstream, and that's a sort of payment. But sometimes it just doesn't feel like enough.

If someone has suggestions how to get a big organisation to contribute financially to some of the smaller open source projects they depend on, I'm all ears. I'm sure the mechanical process of payment can probably be solved, but the mindset issue is much harder.

McGrath: Red Hat’s commitment to open source

Posted Jun 26, 2023 21:30 UTC (Mon) by NYKevin (subscriber, #129325) [Link]

1. Don't characterize it as a donation. Characterize it as a "sponsorship."
2. Don't try to give money to every FOSS project you use. Sponsor things that management is willing to sponsor, or could be persuaded to sponsor (because the alternative is giving no money at all). For example, if you give substantial money or other assistance to Debian, they will (subject to various formalities) put your logo on https://www.debian.org/partners/ and write mildly nice things about you - that might be an easier sell than another project that doesn't have such a page.
3. Don't (just) sponsor projects directly. Sponsor conferences, events, organizations, and anything else that could reasonably help a project thrive.
4. If you can't get them to sponsor an event, at least try to get them to expense entry fees for any interested employees.
5. Talk to marketing and find out whether developers are a cohort they want to target. If so, they may be supportive of sponsorships.
6. If a project looks seriously underfunded, show management https://xkcd.com/2347/ and ask them what they plan to do when that one developer in Nebraska gets bored or retires. Write down their answer, and characterize it as an official contingency plan. If they refuse to answer, then document the non-existence of a plan, as overtly as possible (without looking like a smart ass).
7. If management wants to purchase a support contract, try to steer them towards whoever is actually doing the work, not just whoever has the lowest rate. Remember, the people who wrote the code are often best equipped to do something about bugs and other issues. A third-party support contract is much less valuable, because they may be unable or unwilling to do much more than file exactly the same bug report you would've filed yourselves.
8. Accept that a corporation is a profit-maximizing machine, and they won't always do what is right, or even what is in their own long-term interest. Do the best that you can, and don't worry too much about things you can't change.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 10:44 UTC (Tue) by tialaramex (subscriber, #21167) [Link]

Right. "Our name on a hat" is something companies can understand. A mid-size firm probably has made such arrangements already, just not for Free Software. Does a local kids sports team play in kit with your company name? Maybe that local theatre used by the town amateur dramatics society has seats with your company's name etched onto a little label?

Red Bull pays Grandpoobear to play Mario https://www.redbull.com/gb-en/athlete/david-grandpoobear-... while wearing their hat (with a Red Bull logo) and (obviously) not saying this is a terrible beverage, nobody should drink it. Is Mario an extreme sport? Sure, if you're as good as he is.

Sponsorship is good because companies understand what they're not getting for their money. Your logo on the Debian web site does not entitle you to bug fixes, or to decide what is or is not packaged.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 18:16 UTC (Tue) by HenrikH (subscriber, #31152) [Link]

It's also much easier to get money out of the PR department than having to force the software developer side create a budget for the expenses.

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 9:44 UTC (Wed) by davidgerard (guest, #100304) [Link]

> 6. If a project looks seriously underfunded, show management https://xkcd.com/2347/ and ask them what they plan to do when that one developer in Nebraska gets bored or retires. Write down their answer, and characterize it as an official contingency plan. If they refuse to answer, then document the non-existence of a plan, as overtly as possible (without looking like a smart ass).

this is brilliant

McGrath: Red Hat’s commitment to open source

Posted Jun 30, 2023 19:19 UTC (Fri) by kpfleming (subscriber, #23250) [Link]

In the large-company world this is called "Business Continuity Planning" and is 100% appropriate. Many more companies should be doing this, for each and every piece of software they rely on to operate their business.

McGrath: Red Hat’s commitment to open source

Posted Jun 26, 2023 21:52 UTC (Mon) by Wol (subscriber, #4433) [Link]

This is why organisations like Red Hat, SUSE, Canonical, are important. Companies can take out a support contrtact, and then these can dole out money to the appropriate foundations looking after the "software of importance". Although, of course, what is seen as important, and what is important, are often not the same.

Cheers,
Wol

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 3:43 UTC (Tue) by Matt_G (subscriber, #112824) [Link]

I'll second the paid support is extremely important to large organizations. I work in an Industrial Plant (a large smelter) we have a lot of older system running custom apps (C/C++ code) for things like HMI's which are extremely critical. These mostly run HPUX or AIX, there are newer systems running Linux (Red Hat - I think) - we have had incidents in the past where companies have flown engineers to our plant to fix issues. Sometimes they will have written custom patches etc. for specific issues. The deployed life of these assets tend to be a long time - so long term support is crucial i.e. more than 2 to 3 years.

So I think this is one area where there is potentially a lot to be made in open source if you can offer this type of support there is potentially big money in the contracts and companies can and do pay for these services.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 17:05 UTC (Tue) by lyubomyrk (guest, #165831) [Link]

I work at a IMU design and manufacturing company and recently was asked to automate wafer inspections on 20 year old microscopes that are complete black boxes running MSDOS. No one remembered where the source code was and we got no support, running it for decades hoping nothing breaks otherwise we would have to spend $100,000 on new equipment. I still have nightmares when I opened up that Visual C++ project.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 9:03 UTC (Tue) by pabs (subscriber, #43278) [Link]

File bugs asking for them to send you an invoice you can send to your boss?

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 21:10 UTC (Wed) by zonker (subscriber, #7867) [Link]

The mechanical part of payment is *really hard* -- especially since many projects simply are not set up to receive money the way that corporations are set up to distribute it.

In a perfect or even better world all companies would follow your lead on bug reporting/patching. That would go a long way.

What would be even more amazing is if companies of certain sizes hired open source "janitors" or utility players to do that full time. Or if you can identify a project that is really important to you, hire a person to work on that project full time.

That could make vendor neutral projects much more sustainable and provide more work for developers who want to contribute to open source outside the vendor model.

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 22:55 UTC (Wed) by pizza (subscriber, #46) [Link]

> The mechanical part of payment is *really hard* -- especially since many projects simply are not set up to receive money the way that corporations are set up to distribute it.

s/many/nearly all/

And even if you do have a way to receive money, the odds of you receiving enough money to do anything other than buy a pizza once in a while is effectively zero.

One project I'm heavily involved with receives (on average) enough to cover about 1/3 of its monthly hosting fees, to say nothing of compensating anyone for their actual time/work. And that's by far the most successful of any of 'em.

> That could make vendor neutral projects much more sustainable and provide more work for developers who want to contribute to open source outside the vendor model.

Even this "Vendor neutral" thing requires a project to be at least of a certain (large!) size to be viable. For every one of those there's probably 1000 (or more, looking at "modern" language ecosystems) that are entirely dependent on one or two volunteers.

</grump>

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 9:30 UTC (Thu) by kleptog (subscriber, #1183) [Link]

It remains one of the great missed opportunities of the internet that no functional widely available micropayments mechanism took hold. I know everything financial suddenly becomes complicated, but sometimes it's just too complicated.

But it can be done. If some YouTuber does a live-stream you can during the live-stream, with a few taps, donate some money. That works because one company, in this case Google, made it happen. I think maybe PayPal comes closest, but it's just a bit too clunky for micropayments. There's GitHub sponsors, but then I have to pick projects specifically.

What I'm kind of thinking of is that websites, projects, etc could include a header or something giving their donation details, a PayPal link or something. Then once a month a plugin could examine your browser history, summarise all this and calculate relative usages and present me with a list. I can then type in how much I want to donate this month and it will in the background donate to all the sites on the list in one go in proportion to usage (or some other metric).

This would take out much of the pain, since now I only need to do the payment steps once a month, rather than than for every site individually. I'd like to be able to donate €100 per month split across dozens of projects without the pain. Looking around it seems like all the individual parts are there, but it's all splintered.

One can dream.

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 10:23 UTC (Thu) by excors (subscriber, #95769) [Link]

On Twitch (and I assume YouTube live-streaming is similar), I think the main motivation for donations is the social (or parasocial) aspect. You donate $5 and your username pops up on the screen for every viewer to see, and the streamer says "thanks [your username]", and you get some minor perks (like no Twitch ads on that channel for a month). Or you donate a lot of money and you get a big flashy animation popping up the screen, and the streamer is like "wow, that's so amazing, you're the best, let's have some poggers in the chat for [your username]" and lots of the other viewers spam emojis in response to your generosity, and you become a well-known member of their community, and get permanently displayed on a donation leaderboard, and maybe the streamer does a little dance for you or something.

I don't know if there's any data to back this up, but it often seems that most of the donated amount comes from a very small number of users (the whales) who spend hundreds or thousands of dollars a month on a single streamer - sometimes because they're very rich and like showing off, but sometimes they can't really afford it and they're trapped in this exploitative relationship with the streamer.

I'm not sure you could successfully replicate that donation model in other contexts (like for open source projects), without the associated community and real-time feedback to make people feel highly rewarded for their donations. The mechanism for frictionless payments is important, but the motivation for payment is even more important and seems like the harder problem.

McGrath: Red Hat’s commitment to open source

Posted Jun 30, 2023 10:33 UTC (Fri) by milesrout (subscriber, #126894) [Link]

Another question is: do you really want to replicate that model? As you rightly point out, the so-called "donations" (they're not donations, they're tips) are largely coming from a small number of users. Most of them are not rich. They're people stuck in parasocial relationships with streamers (and other members of those streamers' communities) because frankly they don't have any other social relationships. It's pretty exploitative. I think a big part of it is that it's a bit like the old "one-armed bandit" slot machine: there's a somewhat random reward mechanism involved. If you tip an amount, it might get a big reaction or a small reaction depending on what's happening on the stream, what time of day it is, how many people are there, random chance, etc. As far as I understand, it's been established that randomised rewards are more effective at creating Pavlovian conditioning than consistent rewards. It's essentially gambling money for social "reward".

There are programmers that livestream programming "content" on Twitch and YouTube, such as Jonathan Blow, Casey Muratori and others. I can only presume that they make a reasonable amount of money doing it, or they likely wouldn't. They definitely have parasocial communities. That aside, even if it were possible outside of livestreaming, is it really a good thing? "Free software funding" is an end, and perhaps an admirable one. But that doesn't necessarily justify the means. Isn't this just exploiting people? What are people actually getting from these tips? Transient applause? Whales are basically gambling addicts. To use an analogy, slot machines make pubs a lot of money, but - to put it mildly - they're a bit ethically questionable.

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 10:51 UTC (Thu) by paulj (subscriber, #341) [Link]

There are websites that exist to make crowd-funding FOSS easier, e.g. Snowdrift.coop (https://lwn.net/Articles/625051/), LiberaPay, BountySource and such. Snowdrift wiki has a good page with a comparison table of various such sites:

https://wiki.snowdrift.coop/market-research/other-crowdfu...

As for micropayments, well, some might say the Internet has seen some distributed, open-source, payments systems become wildly successful since the above LWN article was written. E.g., Bitcoin and - more suitable for true micro-payments - Monero (added advantage of providing very good privacy, anonymous tipping).

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 11:59 UTC (Thu) by anselm (subscriber, #2796) [Link]

I would be very reluctant to mention the terms “Bitcoin”, “payment system”, and “very successful” in the same sentence. If the last decade has shown us anything, it is that Bitcoin is completely – and unfixably – unsuitable as a payment system at scale. (With transaction fees currently at US$2.70 or so, that goes especially for micropayments!) It is barely scraping by as a type of investment, and even that is likely to go down the drain for good eventually, now that the SEC has the big crypto exchanges in its sights.

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 12:32 UTC (Thu) by paulj (subscriber, #341) [Link]

Have you looked at fees for international bank transfers? E.g., SWIFT? Have you looked at the fees that card payment processors charge?

Bitcoin is indeed more expensive than it should be, but it's comparable to the above. Anyway Monero fixes most of Bitcoin's issues. Fees on Monero transfers are sub-cent as a result. (Monero isn't perfect - not good for instant - but pretty good).

Bitcoin is _so_ much easier for transferring value to arbitrary others than the banking system - even just within Europe. I had to pay something fairly large recently, on behalf of a company, and it took _days_ to get funds consolidated into 1 account. And then it took 2 days just to transfer money _within the same bank_ - along with most of a morning spent on the phone with the bank, to figure out limits and bugs in their online banking (one limit being "sorry, can't do anything about that - you have to physically go to a branch").

Separately, I have been waiting for _2 weeks_ and counting for a bank transfer to me via SWIFT. There are large chunks of the world that are stuck with either slow-slow SWIFT or else exploitative money-transfer companies.

The whole pump-and-dump investment bros around crypto are a good reason to dislike and be sceptical (esp as an investment), but as a global clearing system this stuff works /way/ better than the banking system.

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 21:13 UTC (Thu) by kleptog (subscriber, #1183) [Link]

> Have you looked at fees for international bank transfers? E.g., SWIFT? Have you looked at the fees that card payment processors charge?

That's the depressing part: there no technical reason for the charges to be so high or it to take so long. It's just laziness and greed. Once you're used to SEPA Instant with free (or nominal cost) payments in under 10 seconds to anywhere in the Euro area you suddenly see how weird it is that most of the world lives with shitty payment systems. And it's not that banks suddenly figured they'd do this to improve customer service. No, the EU simply regulated that it should happen, and while the banks fought tooth and nail all the way, it did happen in the end. Same with the credit cards fees being capped at 0.3%.

(I get not everywhere in Europe has SEPA Instant, but it demonstrates that there's no technical limitations here.)

So while Bitcoin could be competitive in places with truly awful banking systems, that says more about how bad most banking systems are than how good Bitcoin is.

Of course, currency exchange is a special problem that still affects some SEPA transactions but again, there's no technical reason for the costs, only greed and laziness.

McGrath: Red Hat’s commitment to open source

Posted Jun 30, 2023 9:50 UTC (Fri) by paulj (subscriber, #341) [Link]

The domestic banks in my country do not support SEPA Instant. It's very frustrating.

Also, SEPA is only free for retail users. Business customers pay fees for SEPA transfers. Those fees are generally significantly higher - 3+ orders of decimal magnitude - than, say, Monero transaction fees. And card acquiring fees of 0.2% are still damn high.

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 23:41 UTC (Thu) by anselm (subscriber, #2796) [Link]

Bitcoin is indeed more expensive than it should be, but it's comparable to the above.

Jet liners sometimes crash, but that doesn't imply that flying carpets are a better alternative.

In fact, the current US$ 2.70 average transaction fees are near the low end for Bitcoin, and only because hardly anyone uses it for payments these days (most of the transaction volume is wash trading inside exchanges). Back during the Bitcoin hype of 2020, average transaction fees of US$ 25 were not unheard of, with even higher spikes – which is OK if you're paying for a Tesla; less so for buying a cup of coffee or a newspaper, let alone making a “micropayment”. Of course with Bitcoin, the transaction fee is basically a bribe to entice Bitcoin miners to consider your transaction at all, so if you post a transaction fee that is too low, your transaction will never be picked up for the blockchain. At least with SWIFT, for all its hassle and obvious faults, a transaction will eventually go through; with Bitcoin, there is no such guarantee.

I haven't paid a lot of attention to Monero, but since it, like Bitcoin, is based on a proof-of-work blockchain it is likely to share most of Bitcoin's main disadvantages, i.e., it is potentially a grotesque waste of energy and won't scale to usable size. The additional issue with Monero is that it is a common ingredient of malware and viruses and therefore virus scanners will pick up on Monero miners. Its enhanced privacy means that it is becoming the cryptocurrency of choice for ransomware, and it is also popular on darknet markets, which means that many crypto exchagnes won't touch it. Some alternative.

McGrath: Red Hat’s commitment to open source

Posted Jun 30, 2023 10:09 UTC (Fri) by paulj (subscriber, #341) [Link]

That most transactions are on exchanges is... utterly irrelevant. This is true for Euro and Dollars too. While this might be part of a point about the investment/speculation side of currency markets - points I might agree with you on largely - it's not really relevant to a discussion about utility as a clearing system. It's like comparing a currency exchange to SWIFT. Different thing.

Bitcoin fees are generally no worse than, and usually much better, than SWIFT. And... I wouldn't even prefer Bitcoin, I'd go with Monero for clearing.

Energy use: The cheapest energy for mining is energy that can not otherwise be sold. E.g., hydro-power in remote areas (dams may be built for water management, with a turbine... just cause - e.g. to power the dam; there may be little local demand), hydro-power and other renewable power in off-peak times, gas flaring in hydro-carbon extraction (yes, we should greatly reduce HC dependence, but till we do, this will be happening - before it was just burned, now some oil companies get some work out of that heat through mining). A large chunk of BTC mining is from such energy sources - it is dishonest to estimate the energy of BTC mining and then just consider /all/ that energy to be a) from hydro-carbons b) energy that would not have been used if not for BTC mining.

I myself run a miner in the winter. I am going to use electricity to heat water, to heat the air. Instead I use some electricity to do computation and the waste heat heats the air. Essentially, I get a (very small) discount on my electricity bill - energy I was going to use anyway. Rather than just put the energy into a dumb heating element and radiator to heat a portion of my house, my computer gets to help play a small part in a global financial clearing system to heat a portion of my house.

Finally, the idea that humanity could somehow use less energy is ridiculous. Energy is quality of life. Energy is modern, western, life. A very large chunk of the world still aspires to achieving that. And they are not going to stop till they get it. And the privileged minority enjoying that life do NOT get to tell those other people to stay in their shacks and "think of the environment!".

We _are_ going to use _more_ energy in the future. And we _must_ figure out how to generate _much more_ energy, without HCs. Sustainable, CO2-neutral and *abundant* energy is required for the future. The answer is pretty obvious, but - sadly - fellow greens insist on denying it (which just ensures we continue with HCs, sigh).

As for privacy in financial transactions, I think it's a good thing. And I think we should have it. That some people use privacy for bad things doesn't mean we should ban privacy. The flip-side, of having governments able to view and control /all/ financial transactions between people (a future governments are actively building towards), has very very bad potential too.

Our editor isn't going to be happy if we continue though, probably. You can figure out my email address I think if you want to continue.

McGrath: Red Hat’s commitment to open source

Posted Jun 30, 2023 10:14 UTC (Fri) by paulj (subscriber, #341) [Link]

Also, I have never had a transaction on Bitcoin take anywhere as long as SWIFT transactions do. I have had BTC txes take days, in rare cases, which is still quicker than SWIFT.

I have had more than 1 case where bank transfers just got lost, and it took weeks to prod the banks into doing something of substance to go find it. The sender would say "We definitely sent it" - receiver "We definitely didn't", and that was it for /weeks/. A SEPA transaction no less.

SWIFT transfers typically take at least 3 days, usually 5, and sometimes 2 weeks.

The banking system is _terrible_. It's horrendously inefficient compared to Monero.

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 12:40 UTC (Thu) by paulj (subscriber, #341) [Link]

tl;dr: Monero would be quite suitable (technically) for projects to accept micro-payments.
- cheap, suitable for sending cents.
- anonymous by default (sender of a tx can always reveal themselves, of course, and prove it)
- open-source
- good community around it

McGrath: Red Hat’s commitment to open source

Posted Jun 26, 2023 20:54 UTC (Mon) by mgulick (subscriber, #63735) [Link]

The gitlab.com repository seems pretty hard to navigate. Most of the repositories in the CentOS Stream RPMs group don't have tags, just 'c8s' and 'c9s' branches. How do I find the corresponding source for a given package release? It looks like I have to browse the history and looks at each of their diffs to find when the package version and release changed in the spec file.

McGrath: Red Hat’s commitment to open source

Posted Jun 26, 2023 23:56 UTC (Mon) by jgg (subscriber, #55211) [Link]

I'm very interested in this as well, can someone post a git link to the commit of a RHEL kernel in Centos stream? eg one with the security backports after RHEL x.y is released?

If we think about a security commit to the kernel there will be at least three versions of it
- One in Linus's tree
- One backported to Centos Stream HEAD (eg for the next RHEL release)
- One backported to a released RHEL kernel x.y.z update

The way I read the blog post makes it sound like the first two are public, but the last one is not? Is that right?

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 5:00 UTC (Tue) by pbonzini (subscriber, #60935) [Link]

Yes, exactly. For example https://gitlab.com/redhat/centos-stream/src/kernel/centos... is a commit that might plausibly end up in 9.2 (I haven't checked, I am on the phone).

For the kernel and 20-30 other packages you have the entire "exploded" source tree on GitLab, with git history including merge commits; the rpm doesn't include individual kernel patches because they're just too many.

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 23:19 UTC (Wed) by mmcgrath (guest, #44906) [Link]

> The gitlab.com repository seems pretty hard to navigate

I'm not trying to be facetious, but that's life in the big city. To get from that code to a RHEL distribution is a *lot* of work. Providing debranded source rpms in git.centos.org saved rebuilders a TON of effort. They're welcome to do the work we're doing from CentOS Stream (or straight from upstream if they're brave enough).

I really do think it's a double standard for anyone to think Red Hat should have to do that work, but asking the rebuilders to do it is too much.

McGrath: Red Hat’s commitment to open source

Posted Jun 26, 2023 21:31 UTC (Mon) by thebluesgnr (guest, #37963) [Link]

> Simply rebuilding code, without adding value or changing it in any way, represents
> a real threat to open source companies everywhere.

Ehh, this is exactly what a Linux distribution is.

McGrath: Red Hat’s commitment to open source

Posted Jun 26, 2023 22:07 UTC (Mon) by quintesse (guest, #14569) [Link]

Creating a Linux distro is a HUGE amount of work! Without them you'd be forced to go to the sources of each and every component you want to install on your system (check the installed packages on your system, there are thousands), figure out what dependencies they need and what versions of those dependencies work together, then compile them (have you thought about how to compile those sources when you don't have a running system yet?), install them, configure them in the correct way and then hope your system will boot.

So distros definitely don't just "rebuild code without adding value", nobody would be running Linux if it wasn't for them. Heck even Linus would probably still be running Minix if it wasn't for distros.

McGrath: Red Hat’s commitment to open source

Posted Jun 26, 2023 22:13 UTC (Mon) by NightMonkey (subscriber, #23051) [Link]

Does this mean that every time I use Gentoo's Portage to install a new package I am a "real threat to open source companies everywhere"?

McGrath: Red Hat’s commitment to open source

Posted Jun 26, 2023 22:43 UTC (Mon) by dullfire (subscriber, #111432) [Link]

Shh. I don't think you are supposed to point that out. Or the fact reproducible builds are a thing. Or any other fallacies that may or may not have been in that blurb.

McGrath: Red Hat’s commitment to open source

Posted Jun 26, 2023 23:55 UTC (Mon) by flussence (subscriber, #85566) [Link]

Oof. That that line ever got published is a huge failure on the part of IBM's PR and legal teams.

Are they saying RHEL *is* this threat, or that it *isn't* this - are they confessing that their product differentiator is in modifying other people's codebases and then being obstinate and hostile to upstream in the same vein as Apple or grsecurity? Are they reversing their endorsement of things like Flatpak?

It's hard to see this reaction as anything but dismissive contempt for all of the people who actually built most of the OS they repackage into RPMs and slap a price tag on. It reminds me a lot of elementaryOS at this point.

There was never a question as to whether behaviour like this is legal (it is - just like what Oracle does to them is), but they're not coming back from below the trust thermocline with this weasel talk.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 0:55 UTC (Tue) by tuna (guest, #44480) [Link]

"Are they saying RHEL *is* this threat, or that it *isn't* this - are they confessing that their product differentiator is in modifying other people's codebases and then being obstinate and hostile to upstream in the same vein as Apple or grsecurity? Are they reversing their endorsement of things like Flatpak?"

RH is probably the best company in the world in pushing things upstream. According to the blog post that won't change.

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 8:10 UTC (Thu) by milesrout (subscriber, #126894) [Link]

> Are they saying RHEL *is* this threat, or that it *isn't* this - are they confessing that their product differentiator is in modifying other people's codebases and then being obstinate and hostile to upstream in the same vein as Apple or grsecurity? Are they reversing their endorsement of things like Flatpak?

Why are you commenting if you actually haven't got any idea about the issue at all? They contribute a lot upstream. But you can't contribute "packaging" upstream, because application and library developers don't *want* to be distributions, and quite reasonably. RHEL does not upstream its integration work, and it would make no sense to. No distribution does.

What RH is no longer interested in doing is *downstreaming* its integration work to freeloading rebuilders.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 0:20 UTC (Tue) by jepler (subscriber, #105975) [Link]

I'm curious if it's possible to estimate what proportion of the work of having software in my favorite distro can be attributed to the project authors vs the packagers.

The most blunt instrument would be to look at the overall size of a source package vs the packaging. I tried comparing raw disk usage (du -s), non-empty lines (find -exec grep . {} + | wc -l) and sloccount for debian ffmpeg 4.3.5. The first two gave a similar figure of around 0.9% of ffmpeg being in the packaging; the latter gave 0.03%! This is because sloccount treats most files in debian/ as not "source code".

A second package, ntp 4.2.8p12+dfsg, is 1.76% packaging by du, 0.45% by wc, and 0.08% by sloccount.

As a gut check, "packaging software is 1% of overall software effort" doesn't seem outrageous.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 1:17 UTC (Tue) by pizza (subscriber, #46) [Link]

> As a gut check, "packaging software is 1% of overall software effort" doesn't seem outrageous.

This doesn't account for integration & subsequent testing.

One can quibble over the portion, but it's more than zero.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 3:11 UTC (Tue) by wtarreau (subscriber, #51152) [Link]

Indeed, building and testing costs a huge amount of developers time. Just figuring correct build options that end up as a single line in a spec file or any distro-specific build recipe can sometimes take a full day. Anyone having played with ./configure and build the same stuff 50 times to try to figure the correct paths etc so that everything assembles nicely knows it. Other distros know it pretty well, including the large free ones (debian, gentoo etc).

Here one difference is that a lot of developers' time is made available by a single company that collects the money to pay their salaries via sales. That's an important part of the ecosystem. Also one must not forget their significant representation among various critically important projects such as the kernel itself.

It continues to be important to me that this company (and a few other ones like Canonical) continue to make profit and pay as many developers as they can because ultimately they are the ones doing the free work we all want to rely on. Many people love to see Linux as the week-end hobbyist project it used to be, that's no longer the case. Just look at the amount of messages on LKML on week-ends, it's super low. The reality is that the projects we love are entirely run by people being paid to do that work, and that's what makes it scale that large.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 9:14 UTC (Tue) by pabs (subscriber, #43278) [Link]

Personally I think it is a huge problem that such a large percentage of paid upstream FOSS development depends on and is reliant on and directed by RedHat and other large companies. The funding for FOSS needs to be way more diversely sourced and distributed. This is especially concerning because of the types of projects that get funded and the audiences and activities that those projects enable.

Apart from that, it is definitely important that the existing funding RedHat represents doesn't dry up.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 13:32 UTC (Tue) by wtarreau (subscriber, #51152) [Link]

It can be seen as a problem or as an opportunity. I'm used to say that many opensource projects last only 3 years, i.e. while the creator is a student with lots of spare time. Once the creator gets a job and a family, the project dies by lack of time. The only sustainable model is when the author(s) get paid for their work so that they can do that *instead* of doing something else. Small companies are having a hard time achieving this except in a few situations, because they're always struggling to get money to pay their existing employees. As a result it's essentially well established large companies that can afford to pay a (sometimes large) team to spend time on opensource, and the community benefits from the regularity of their commitment. This obviously has an impact in their operating costs and products/services prices, but it's important to think about them when one wants to participate financially to the whole ecosystem. With that said, if you only have a little bit of money, think about the small companies first, because for them it's even more important given that their efforts put them in a much more risky situation. And self-funded volunteers definitely are heroes if they can do that for a living.

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 11:23 UTC (Wed) by timofonic (guest, #165836) [Link]

> The funding for FOSS needs to be way more diversely sourced and distributed. This is especially concerning because of the types of projects that get funded and the audiences and activities that those projects enable.

I tought the same many years ago, even more after knew about IBM buying RedHat and now this.

My following opinion might be controversial: I consider GPLv4 must be made and designed in a clear way to avoid toxic behaviors like RedHat/IBM one.

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 8:11 UTC (Thu) by milesrout (subscriber, #126894) [Link]

RedHat is doing precisely what the GPL is intended to allow! That's the entire POINT of the GPL.

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 11:56 UTC (Wed) by paulj (subscriber, #341) [Link]

In some utopian world we'd have some kind of "FOSS clearing house" - akin to the way there are royalty clearing houses for music in some places.

There'd be some kind of representative sampling of computers running FOSS, to see what FOSS got used - voluntary/opt-in. With some kind of relatively rigorous correction for sampling biases. Then companies and individuals could donate to the clearing org, who could distribute proceeds onward to FOSS developers in some more-or-less fairish way.

Obviously, a lot of difficult and subjective things there.

We do have FOSS steward non-profits I guess, but they can be selective. Your project may not get taken in, and/or trying to do so may be an extremely long and opaque process. Some such stewards also take donations from large corporates - large corporates who have different interests to small FOSS developers - which may cause conflicts of interest in the stewarding org, particularly if the stewarding organisation (or officers thereof) have other activities around FOSS (licence enforcement, licence development, FOSS corporate outreach, etc.).

That's not meant to disparage such orgs, just it's a difficult world for FOSS.

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 13:11 UTC (Thu) by nim-nim (subscriber, #34454) [Link]

> In some utopian world we'd have some kind of "FOSS clearing house" - akin to the way there are royalty clearing houses for music in some places.

That ’s a bad analogy, it implies the basic problem is insuring every dev is paid, and a distribution is just the sum of the work of those devs. Integrating a pile of source code into a reliable system that keeps working year after year is a *huge* amount of work. When you take into account deployment in the field, non-dev work involved into deploying the code the dev wrote, can easily get into several orders of magnitude more than the code writing itself.

As long as devs refuse to see they are just one part of the ecosystem they will be unable to imagine a payment structure that works in the real world.

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 13:15 UTC (Thu) by paulj (subscriber, #341) [Link]

Oh, quite agree. Some kind of "good" sustainable model for FOSS development and *maintenance* is something we still don't have great answers for.

I'm sure LWN has had at least some articles on the challenges wrt maintaining FOSS, e.g.: https://lwn.net/Articles/712215/ - I thought there was another more recent than that, but can't find.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 9:16 UTC (Tue) by dottedmag (subscriber, #18590) [Link]

Having to wrestle with build systems is such a waste of everyone's time, and largely an self-inflicted wound. Moreover, the idea of hundreds of optional dependencies and configurations is also quite troublesome.

Having spent last several years profesionally in an ecosystem where there is no ./configure wrangling and no guess-the-dependency-version game, I'd suspect that streamlining it for C and C++ would release immense amount of effort to be applicable to something more useful than just building stuff (testing it, for example).

Alas, I couldn't see any way to coordinate improvements: in a Red Queen race one cannot stop and think how to change the race completely. New build systems for C and C++ improve ergonomics, but they do not question the way software is built.

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 18:16 UTC (Wed) by jepler (subscriber, #105975) [Link]

it's sure possible that "integration & subsequent testing" cost a lot of effort without yielding a lot of lines of code, and are done more by packagers than by upstream maintainers.

another major item I didn't account for is packager work that is (whether immediately or later) accepted upstream, which wouldn't be shown when counting lines in the distribution packaging.

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 19:29 UTC (Wed) by mb (subscriber, #50428) [Link]

>it's sure possible that "integration & subsequent testing" cost a lot of effort without yielding a lot of lines of code, and are done more by packagers than by upstream maintainers.

Fortunately the GPL does not talk about development effort, packaging effort, maintenance effort, support cost and lines of code at all.

If somebody decides to distributes somebody else's GPL software, they can't further restrict it.
If somebody decides to distribute their own software under GPL license, why would they complain? Just distribute it under non-GPL (just like SuSE did with YaST), if you don't want other people to exploit GPL freedoms. But distributing GPL software and than complaining that customers exploit their right is just ridiculous.

It is as simple as that. If a company does not want their distribution to have GPL freedoms as a whole, just put a core component under a more restrictive license. If the company doesn't want to do that, because they want to be all-free software, then stop complaining, please, if people exploit their freedoms.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 5:48 UTC (Tue) by Cyberax (✭ supporter ✭, #52523) [Link]

> As a gut check, "packaging software is 1% of overall software effort" doesn't seem outrageous.

Then what's the big deal? Just get a one-time copy of RHEL source code and maintain it yourself.

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 8:08 UTC (Thu) by milesrout (subscriber, #126894) [Link]

And what are the percentages for the vast majority of packages that are smaller than large packages like ffmpeg?

What about the time and effort that goes into it? Everyone knows "lines of code" is a terrible way to measure difficulty. Some lines of code are a lot harder to write and maintain than others.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 11:42 UTC (Tue) by nim-nim (subscriber, #34454) [Link]

>> Simply rebuilding code, without adding value or changing it in any way, represents
>> a real threat to open source companies everywhere.

>Ehh, this is exactly what a Linux distribution is.

Nope, otherwise competitors would have no problem restarting from upstream sources, instead of complaining bitterly Red Hat makes it harder to copy all the stuff it added over upstream sources to make them build and run reliably.

Why bother cloning something if it adds no value over public upstream ?

Distributions add a hell of a lot of value over raw upstream sources in the form of finding the build settings that actually work, finding the versions combinations that actually work, testing the result at scale on multiple hardware architectures and wrapping it all in an uniform deployment format so end users are not exposed to differences of dev opinion on how stuff should build.

(I intentionally omit all the bug fixing and other dev work distributions also perform, because even if upstream code was perfect and without a bug there would still be a huge value on the other non dev stuff performed distro-side).

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 14:12 UTC (Tue) by farnz (subscriber, #17727) [Link]

As an aside, note that much of Red Hat's stuff added over upstream sources is available in https://gitlab.com/redhat/centos-stream/ - while this isn't everything in RHEL, the difference only matters if you're either not keeping on top of security-relevant changes upstream yourself, or if it really really matters that you're on the same package version as RHEL N.M, and not on the version that's going to be part of RHEL N.M+1 or RHEL N+1.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 19:39 UTC (Tue) by mfuzzey (subscriber, #57966) [Link]

Not really.

A Linux distribution does far more than "rebuilding code", even if 100% of the source they use is available in the various upstreams.

They select components to use, figure out the dependencies, package it all as a coheseive whole, handle security updates and bug fixes and often add quite a bit of their own code. This is true for all distributions both commercial and community. And all of that is a *lot* of work.

I agree with Red Hat that *pure rebuilding* of an already created distribution without changing or adding anything doesn't add any value. On the other hand I don't agree that it is necesarilly a "threat to open source companies everywhere". A company that just does support wouldn't be threatened by rebuilders - their money comes from support contracts that are paid by those that want them. The problem for Red Hat is that they don't want to be (just) a support company but effectively sell a distribution. Rebuilders do threaten that model by reducing the number of customers.

McGrath: Red Hat’s commitment to open source

Posted Jun 26, 2023 23:44 UTC (Mon) by sadoon (subscriber, #155365) [Link]

Man's got a point. Developers need to live after all.
I don't care for the attitude of the cloners. They're not forks, they're not offering anything different, what's the point of their existence other than getting RHEL for free?

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 0:49 UTC (Tue) by Paf (subscriber, #91811) [Link]

Nothing. And, yeah - that’s open source for you. It’s open, especially the free software sort. That makes it hard to sustainably charge money for, because if it’s important and you charge enough, people can do this sort of thing. It’s an unavoidable corollary of the openness. (The thought behind my words is here is that impugning their motivations, etc, won’t work - it will just keep happening as long as RHEL is open source and what they charge for access is high enough.)

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 1:40 UTC (Tue) by pabs (subscriber, #43278) [Link]

In open source, you don't get a monopoly over commercial exploitation of your work, including the trademark-replacing clones:

https://drewdevault.com/2021/01/20/FOSS-is-to-surrender-y...

RedHat itself started out as just a clone of other people's source, the RHEL clones too will do extra QA, report bugs upstream, send patches etc. This is inevitable because it is how the culture of open source works; share everything, contribute back. Also because not contributing back means more work in the long run rebasing patches etc.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 4:52 UTC (Tue) by geofft (subscriber, #59789) [Link]

I think the point being made is that Red Hat is absolutely going to contribute things upstream and absolutely will make any fixes/enhancements they develop upstream - they just aren't going to make it particularly easy to be a bug-for-bug clone of Red Hat. And I'm a little bit sympathetic to that because the only people who really need a bug-for-bug clone of Red Hat are people who are running software (FOSS or proprietary) from some other commercial vendor who certifies it as running on literally Red Hat and nothing else, or people who want to run 10 Red Hat machines and 90 not-Red-Hat machines and report bugs to Red Hat regardless of which machine they came from.

The broader FOSS ecosystem doesn't really benefit from this. I don't think I've ever seen an upstream developer say "Oh you ran into this on Fedora, but can you reproduce it on Red Hat or a Red Hat rebuild?" It wouldn't be useful to the broader ecosystem if they said that, because how could you expect to use that software on Debian or Arch or anything? How could that software participate in the FOSS community as a whole instead of just Red Hat and its clones?

The broader FOSS ecosystem absolutely does benefit from Red Hat sending patches and bug reports upstream, and as a backstop for when Red Hat doesn't engage upstream, the broader ecosystem absolutely does benefit from Red Hat's patches and customizations being published to the world. Which are the things that Red Hat is committing to doing. If they find a bug in OpenLDAP and they patch it for their customers, that patch - and its commit message, and the discussion on the merge request, and a whole lot of other things that aren't in the SRPM! - is on GitLab.com for anyone else to pick up. The only thing they're no longer committing to is saying whether that patch was applied in openldap-2.6-20.el9.x86_64.rpm or openldap-2.6-21.el9.x86_64.rpm. That piece of information is wholly irrelevant to the culture of open source; you don't need it unless you're specifically trying not to have any patches of your own. It's only interesting if your interest in Red Hat publishing sources, and being open source at all, stops once you have an equivalent binary.

(Or, admittedly, unless you're Software Freedom Conservancy trying to verify that in fact they did push all the patches to GitLab that they claimed they were going to. That's a valid use case and I hope that gets resolved. I'm concerned by comments on the question I asked yesterday https://lwn.net/Articles/936138/ about how there are private branches, because that means it's easy to forget to push things publicly. Even a commitment to automation would be meaningful.)

All of Drew's examples there are about people adding value to your work in some way - incorporating it into their own work, becoming an expert and consulting/teaching/writing about the software, packaging it nicely, etc. Every single one of these use cases could be accomplished just as well with CentOS Stream as with Red Hat, if not better. If I wanted to, say, repackage 389-ds and run a little LDAP company, I would actively not want to be bound to Red Hat's release cycle and their discretion of what bugs are important! I would want to see their decisions about what patches go into stable releases and potentially make my own depending on what issues my customers are having. I wouldn't object if the base I'm working from is a little bit newer than Red Hat's commercial product; in fact I would find that useful.

The only use case where unthinking replication of Red Hat's decisions and renunciation of the ability to make your own decisions are valuable is when your product's value is entirely in it being the same as Red Hat's product but cheaper. It's true that Red Hat is also surrendering their ability to prevent this by redistributing FOSS, but it is telling that this isn't a use case where you can write a convincing story about how it's a good thing for the world.

(And, in turn, the money that Red Hat charges goes to fund their ability to continue participating in the ecosystem. If you look at e.g. the development stats article right below this one https://lwn.net/Articles/929582/ , a lot of the major contributors are hardware vendors who want to sell you their hardware, or companies like Google or Facebook that have their own internal use cases and their own agendas. The argument being made in the article is that a world where Red Hat's business model is not commercially viable is actually worse for FOSS than a world where Red Hat survives by doing things that just barely comply with the GPL.)

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 7:28 UTC (Tue) by taladar (subscriber, #68407) [Link]

RHEL clones are mainly useful to build and test software meant for RHEL. As such they make supporting RHEL easier for people who do not have a big investment in RHEL, adding value to the RedHat ecosystem. And no, many people do not want to deal with some sort of license, even a free one, to support that use case.

It is bad enough, frankly, that 10 year support distros like RHEL often force us to make software work with current and 10+ year old software at the same time at all.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 9:51 UTC (Tue) by tuna (guest, #44480) [Link]

"It is bad enough, frankly, that 10 year support distros like RHEL often force us to make software work with current and 10+ year old software at the same time at all."

You are not forced to write software for certain platforms. You might have a marketing incentive to write software for certain platforms, but no one is forcing you to write software.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 12:37 UTC (Tue) by pizza (subscriber, #46) [Link]

> RHEL clones are mainly useful to build and test software meant for RHEL.

No; developer use of the clones is a very tiny minority of their user base.

I'd put money down that over 95% of the deployments of the clones are from folks that want a "LTS" linux distribution they don't have to pay for, deploying anywhere from snowflake production systems to hundreds of servers (and/or workstations running Very Expen$ive Software) to tens of thousands (or more) of cattle compute nodes.

(and for the record, I've done or worked at companies that do all of these. The latter even had a handful of RHEL licenses they used for "official" support if their compute clusters ran into problems and it was reproducible on genuine RHEL).

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 18:52 UTC (Tue) by wtarreau (subscriber, #51152) [Link]

I totally agree, I've seen the same. Consultants proposing CentOS because "the customer cannot afford a single RHEL", despite paying $5K worth of consulting time. Many times I said them "that's not correct, you're hurting the ecosystem, this is a wrong excuse".

I really think Red Hat should propose a free version themselves that can be upgraded to a supported one without having to reinstall anything. Just download the ISO for free like you do with Ubuntu, and if one day you decide that your server has become sensitive enough to warrant paying, just do it. At least there would be *their* distro in field for all users, free and paid. The problem they're facing now is that for absurd reasons of pricing (or even the perceived difficulty of reselling when you only expected to deliver service to a customer), they end up forcing their potential future customers to deploy a competing solution instead of theirs. And *this* is hard to upgrade later, as the customer certainly doesn't want to touch their running system when it becomes critical.

They'd basically just need an motd reminding the registration link so that those logging in via SSH to inspect something are greeted with "Experiencing some trouble? Need some help? May be it's time to subscribe. ->link". It would completely fix the consultant problem above: nothing is sold, it's only installed and the customer may decide later to pay.

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 14:28 UTC (Wed) by ewan (subscriber, #5533) [Link]

> I really think Red Hat should propose a free version themselves that can be upgraded to a supported one without having to reinstall anything.

That's *exactly* what Red Hat Linux was.

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 15:25 UTC (Wed) by zdzichu (subscriber, #17118) [Link]

And RHEL still is? You can download and use it on 16 servers for free. It's not "grab an iso and install" easy, but creating a red hat account is simple enough.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 12:43 UTC (Tue) by anselm (subscriber, #2796) [Link]

RHEL clones are mainly useful to build and test software meant for RHEL. As such they make supporting RHEL easier for people who do not have a big investment in RHEL, adding value to the RedHat ecosystem.

Red Hat is already going out of its way by providing free RHEL developer licenses to support this.

And no, many people do not want to deal with some sort of license, even a free one, to support that use case.

These people get actual RHEL for free already. How much farther do they want Red Hat to bend over backwards on their behalf? Would they like an engineer from Red Hat to come to their site, for free, to install RHEL for them? Perhaps write their code for them while they're drinking beer by the pool? This is entitlement, pure and simple.

Anyway, why would you even futz around with some RHEL clone in the first place? The main reason to deal with RHEL at all is that you have paying customers who run RHEL, and who would prefer you to test your software on genuine RHEL and not just some clone, even if the clone purports to be just the same as RHEL – but who is prepared to guarantee that? The clone maker? You?

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 20:55 UTC (Tue) by amacater (subscriber, #790) [Link]

TL;DR - If you want to move from Red Hat, do it now but make it a reasoned decision in your best interests. Support from any one particular distro is always a cost/benefit decision: they are all equally bad when your bug can't be fixed.

Full disclosure: I currently have a free RHEL developer subscription. I use it to try small things for people, set up machines and VMs to gain expertise for myself. I *can't* force anyone to pay for Red Hat so I sometimes prototype on RHEL and use Rocky. I've a distro mirror next door - I built it on RHEL because I need to show it to RHEL-native sysadmins. Because I can't force the RHEL natives to take out another RHEL subscription, I took the scripts and remade it on Rocky.

Self support on the free tier makes it *really hard* to file a bug for RHEL. Two show stopper bugs took forever to be raised. I could demonstrate that they only occurred with RHEL signing certificates / Red Hat subscription manager. The response made me less enamoured of RHEL. I'd quite like to raise a bug that RHEL 9 doesn't run on my HP Microserver but no-one will listen so the Microserver now runs on Rocky 8.

But RHEL developers |= RHEL system architects and customer supporters |= RHEL senior management. Former RHEL |= current RHEL and current RHEL == IBM.
Red Hat (was) a large company with differences of opinion - hating one RHEL person for what they may or may not say in a press release is perhaps not useful.

If the problem really is Oracle Linux (and not Rocky/Alma) can we just get some popcorn
and watch as IBM take on Oracle? Both have licence compliance mavens but IBM has over a century's worth of attack dog lawyers and a significantly longer corporate memory.

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 8:50 UTC (Wed) by zonker (subscriber, #7867) [Link]

“RedHat itself started out as just a clone of other people's source”

Did it? And for how long?

As I understand it, the first Red Hat Linux release was based on SLS. Not a clone, a fork with additional work added. And, I think, SLS had actually ceased distribution by then. It’s a far cry from a fair comparison.

If the discussion was about Red Hat trying to prevent a project or company from building something novel off RHEL sources instead of just xeroxing whole RHEL releases, it’d be very different.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 1:36 UTC (Tue) by pabs (subscriber, #43278) [Link]

I have already seen some projects dropping support for RHEL, I wonder if there will be any projects rejecting upstream contributions from RHEL folks because of this.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 3:26 UTC (Tue) by adam820 (subscriber, #101353) [Link]

> some projects

Which ones?

McGrath: Red Hat’s commitment to open source

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 12:27 UTC (Tue) by pizza (subscriber, #46) [Link]

He's not dropping "support" for RHEL, he's cropping "official support", while continuing to "officially" support Alma, Rocky, and CentOS Stream.

Which... is a problem how? RH should be the front line of support for everything RHEL. It is, after all, what its users are paying for.

That said, his definition of "no official support" is pre-emptively removing all mention of RHEL in what he distributes, and actively closing any bug tickets from RHEL users as "not reproducible" (when it's 99.9% certain that any RHEL bug would of course manifest in Alma or Rocky and thus be relevant to what he does "officially" support) That's his right of course, but it's one thing to not do any effort, it's another thing entirely to put active effort into ... not doing effort -- it all comes off as pretty petulant, and that's the sort of thing that leads to more (*ahem*) inclusive forks happening.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 13:02 UTC (Tue) by jafd (subscriber, #129642) [Link]

> it all comes off as pretty petulant

I'm reading it more like an "abusive ex situation" rather than "petulant" and choose to interpret it like that. It's his right to cut ties with an entity he thinks did him/others wrong. Which means that if I'm using his stuff with RHEL and encounter a bug, it stands to reason that it's on me to reproduce it elsewhere and then report it.

Here's another angle: there exist fixed bugs in macOS and Macs which have been reported indirectly by the hackintosh community. But they had to go an extra mile to reproduce those bugs on real Macs, or research them enough to convince Apple this was a problem with fully legit usage before they would report them. The difference was that hackintoshes would run into those problems every time you looked at them funny, and real Macs could have them once in a blue moon. But you could not just approach Apple with a "Your system exhibits problems on my non-Apple hardware" and expect them to listen to you (or not take a legal action depending on who you were).

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 13:50 UTC (Tue) by pizza (subscriber, #46) [Link]

> I'm reading it more like an "abusive ex situation" rather than "petulant" and choose to interpret it like that. It's his right to cut ties with an entity he thinks did him/others wrong. Which means that if I'm using his stuff with RHEL and encounter a bug, it stands to reason that it's on me to reproduce it elsewhere and then report it.

Again, it's not that he's ceasing to expend effort to support RHEL users, it's that he's actively *removing* already completed RHEL integration, thus making his software objectively *worse* along the way. Which means that downstream users (possibly including RH themselves) will now have to start maintaining their own forks, or utilize something different.

History has repeatedly shown that anti-inclusive attitudes like this are detrimental to the long-term health of a project.

> Here's another angle: there exist fixed bugs in macOS and Macs

Ugh, MacOS represents the overwhelming majority of the user-support requests for one of the F/OSS projects I'm involved with. Coupled with Apple's unofficial policy of breaking something we need with each new MacOS release and their tools removing support for older OS versions, we're _very_ close to officially throwing in the towel entirely, instead of responding to most prolems with "none of the active developers have _access_ to a Mac, much less the technical knowhow to fix Mac-specific problems and generate new builds."

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 15:28 UTC (Tue) by jafd (subscriber, #129642) [Link]

> it's that he's actively *removing* already completed RHEL integration

Only wording about it (if he's supporting Rocky/Alma, it's in essence THE SAME THING, it's just that you shouldn't be counting on explicit support if you are on bona fide RHEL — this is not like some active harmful activity subject to being run on RHEL). And then, if you look at his repos as of today, the words like “RHEL” or “Red Hat” are still there — search/replace is quite a hassle, and Jeff does look like a person with more interesting things to do.

Now if you're wondering who is being petulant here, you should probably take a look in the mirror.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 15:32 UTC (Tue) by pizza (subscriber, #46) [Link]

And I quote:

"Today: Removing official support for RHEL in Ansible role metadata, so users searching for roles to work with RHEL will not find my roles (I would rather they find roles that are actually tested against RHEL, because I cannot guarantee my roles will work for these users).

"Ongoing: As issues crop up on various roles against RHEL, I will decide on a case-by-case basis whether to strip all RHEL-like support (effectively making the project only run on Fedora, Arch, Ubuntu, Debian, or other distros), or attempting to fix the problem so existing users of Rocky Linux, AlmaLinux, and CentOS Stream may still benefit from the fix.

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 8:18 UTC (Thu) by milesrout (subscriber, #126894) [Link]

Jeff Geerling is just being a drama queen. His immediate reaction to the post we are commenting on was to post lies on Mastodon. He's just acting out to get attention from his followers.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 2:08 UTC (Tue) by ayay (guest, #144420) [Link]

I work for a small non-profit.

We started out small, and used CentOS. As we didn't have resources, we would report bugs, submit patches, documentation etc. as a way of giving back. That also came to include our workstations, running Fedora.

I already knew Red Hat wasn't eager to deal with small potatoes when I first wanted to get my employet to pay for some certifications. The price was too steep for me to even mention.

And then, when the first CentOS debacle came to be, we decided it was time to look into paying for RHEL.

Their pricing structure makes it crystal clear they do not want to deal with small or medium organizations... it's bonkers. We would be better off moving all of our infrastructure to Ubuntu and paying for THAT instead, and it would be no small job to go forward with the migration, but goes to show how much cheaper - and a lot more straightforward - their pricing and licensing is.

Rocky and Alma came to the rescue, so we carried forward on the RH ecosystem. That has been an issue with our new hires - young. talented developers who grew up on a cloud-first environment. They are all in on Ubuntu. They become familiar with the RHEL ecosystem through us.

So we are stuck now: it's clear Red Hat thinks "the juice is not worth the squeeze" for small and medium sized companies, and yet, we can't reason - nor afford - paying them what they're charging. The only thing we can conclude is that they really don't want us using their product.

I am still dumbfounded, though: even if you consider we're not worth that much to Red Hat as customers, if nothing else, we are training grounds for some amazing developers that come to learn and like RHEL's ecosystem through us, and take that with them when they move to higher-paying jobs in bigger companies that can pay what they deserve, after they learn and take off their training wheels.

Then I recall that, in the 90s, Microsoft seemed very eager to cut some sweetheart deals with schools, a move that Google and even Apple also followed eventually.

It strikes me as an extremely short sighted decision. Now, seeing that I am not rich nor a shareholder, that may be why I don't understand it...

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 5:04 UTC (Tue) by pbonzini (subscriber, #60935) [Link]

You can use CentOS Stream too.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 8:18 UTC (Tue) by TRauMa (guest, #16483) [Link]

I couldn't recommend a distro where security fixes are deliberately held back to anyone. And I'm not talking about embargoes, all the fixes. It's one of the ways Red Hat "adds value" to RHEL.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 12:41 UTC (Tue) by pizza (subscriber, #46) [Link]

> I couldn't recommend a distro where security fixes are deliberately held back to anyone. And I'm not talking about embargoes, all the fixes.

Then you'd be happy with CentOS stream, which lands non-embargoed fixes *before* they go out into RHEL, meaning at worst it's no slower than the pre-Stream rebuild flow.

> It's one of the ways Red Hat "adds value" to RHEL.

Um, you act like "adding value for your paying customers" is a bad thing.

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 2:21 UTC (Wed) by ayay (guest, #144420) [Link]

Heh, the machines I converted to Stream almost got me fired. Then I decided to do a "from-scratch" install to prove a point, did not help, quickly walked it back in shame.

n = 1 and all that, but I've had less issues with Fedora Server. The only unfortunate thing is that it requires frequent rebooting, as it updates often - of course, that's exactly what it says on the tin, so although it's not surprising, it's not ideal for our use case.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 5:08 UTC (Tue) by geofft (subscriber, #59789) [Link]

Serious question - why not run CentOS Stream? You don't need 100% drop-in compatibility with Red Hat, do you? It feels like you are exactly their target audience - you want to run something that's essentially Red Hat, but you were never going to be in a position to pay for actual Red Hat.

(Second serious and practical question, but less relevant to the discussion: are you considering Debian? It seems like one of the better choices for an "enterprise" distro, especially since they seem to have quietly switched to time-based releases, and your new hires familiar with Ubuntu will be very comfortable with it. We've been happy with it at my day job for about a decade.)

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 6:55 UTC (Tue) by maniax (subscriber, #4509) [Link]

I'd love to understand that, too.

I think that people feel like CentOS stream is like ... "Debian unstable", all new stuff goes in there and when it's decided to be stable, a release is made that's actually the stable one. I feel like a lot of this is perception, not the truth, but nobody seems to want to use it.

And as for stability, in the last month I had two separate kernel bugs, one in RHEL7 and one in RHEL8, that if ran on KVM hosts led to VM hangs/stalls, and at some point having something move fast enough so you get your bugs fixed quickly might be the better choice than depending on something that releases every 3 months and is supposed to be stable.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 10:14 UTC (Tue) by jafd (subscriber, #129642) [Link]

> Serious question - why not run CentOS Stream?

Right now, as I'm writing this, I'm trying to build ~200 packages with mock using CentOS Stream 9. The process is failing as AppStream's repodata checksums are mismatched. Now they match, but other packages are not yet mirrored and thus I'm left to dry until they all sync or something. There's a bug report in Pagure about this about twice a year, it seems, from internal people too.

My project is a perfect use-case for CentOS Stream: I want some LTS on par with Ubuntu so I don't do the time-consuming "upgrade to the next major version" busywork dance every year, thank you very much. I don't need more guarantees, certifications, and most other fluff RHEL target audience may find useful. But I also want to plug package and container building into a CI/CD pipeline, and if their mirrors keep barfing once in a while without a reliable indication when it happens and how to plan around it, it's quite a sad state of affairs.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 22:27 UTC (Tue) by geofft (subscriber, #59789) [Link]

This seems like a decent use case for someone to be a CentOS Stream rebuilder - start with what's in Git alone, build your own RPMs from scratch and host your own repo, and be willing to patch for major security issues.

(Honestly this would get you eligibility to be on the closed distros list, so you could patch for embargoed vulnerabilities at the same time as RHEL! You still wouldn't be guaranteed to be binary-identical but it would address the concern of unpatched vulnerabilities in Stream, which seems like it would be plenty for people who need a Red-Hat-ish enterprise OS but don't need to 100% mimic RHEL.)

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 16:02 UTC (Tue) by NYKevin (subscriber, #129325) [Link]

+1 to Debian. In my informal hobbyist experience, it is an excellent distribution for making systems that "just work" for years on end with very little fuss or effort.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 20:47 UTC (Tue) by zonker (subscriber, #7867) [Link]

"The only thing we can conclude is that they really don't want us using their product."

I understand why you think that, but I think it's more accurate to say "we are not their target audience."

Rightly or wrongly, Red Hat (and other businesses) make choices about personas and product fit for specific use cases, pricing, etc. You're right that Red Hat isn't eager to deal with small potatoes. It is, after all, called "Enterprise" Linux for a reason.

You cite Microsoft - but that's not apples to apples. Microsoft's annual *marketing* budget is several times RHEL's revenue overall. Probably Red Hat's revenue overall. Microsoft can afford to push into schools and such because 1) its revenue and budgets absolutely dwarf Red Hat's and 2) they're trying to own client computing, backend computing, etc.

It may be a shortsighted decision, I haven't made up my mind just yet and I also don't have the information in front of me that McGrath and the other execs at Red Hat have to inform their decision. They know how many people have renewed RHEL subs, how many are still on RHEL 7 and haven't moved to RHEL 8 or 9, and so forth. This may be a reaction to any number of things that the community doesn't have knowledge of -- and whether it's the right one or not, who knows?

I know in 2003 people threw the same arguments at Red Hat about ending Red Hat Linux and closing off access to Red Hat Advanced Server, etc. With 20 years of perspective it seems that Red Hat didn't implode.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 20:55 UTC (Tue) by pizza (subscriber, #46) [Link]

> I also don't have the information in front of me that McGrath and the other execs at Red Hat have to inform their decision.

To quote McGrath:

"The generally accepted position that these free rebuilds are just funnels churning out RHEL experts and turning into sales just isn’t reality. I wish we lived in that world, but it’s not how it actually plays out. Instead, we’ve found a group of users, many of whom belong to large or very large IT organizations, that want the stability, lifecycle and hardware ecosystem of RHEL without having to actually support the maintainers, engineers, writers, and many more roles that create it. These users also have decided not to use one of the many other Linux distributions."

That lines up with my experience, at both small (<20 employee) companies, at large-ish companies (3000-5000 people) and very large highly-regulated companies (>60,000 employees).

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 13:35 UTC (Wed) by pbonzini (subscriber, #60935) [Link]

Also quoting him from Reddit:

"The problem of rebuilders has been around forever. Things heated up a couple of months ago when we detected what we think was a continued bad-faith action from one of the rebuilders, not on the code/engineering side but on the commercial/money making side of their house. That's as far as I'll go publicly. After that it was just a matter of discussion on what to do about it, so we landed on the announcements I made last week."

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 14:35 UTC (Wed) by pizza (subscriber, #46) [Link]

In case anyone else is interested, here's all of McGrath's posts:

https://old.reddit.com/user/mmcgrath

And another pertinent comment:

"People keep saying that but that doesn't make it true. You don't want free as in freedom. You have that. The code is all out there today and will be in the future, all of it. You want the promise red hat applies to that code, that our engineers will continue to support it for 10 more years. And even though we will push all that code upstream, and put it in CentOS Stream first, that's not good enough. No, you want that service free as in beer.

"RHEL is product, not a project. We aren't preventing rebuilders from building a rebuild, but we aren't going out of our way to make it easy for them any more.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 2:12 UTC (Tue) by cjcox (guest, #60378) [Link]

Insane. Red Hat distributes binaries, they cannot without the source, and even if you think they can hold the source for ransom (again, they cannot), they are actually forcing you to get a support subscription. And that's very much a violation of the GPL.

GPL in summary: You distribute binaries, you supply source. It's that simple. Mic drop.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 3:29 UTC (Tue) by adam820 (subscriber, #101353) [Link]

You can't get access to the binaries without the subscription, ergo you have access to the source.

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 22:39 UTC (Wed) by mmcgrath (guest, #44906) [Link]

> GPL in summary: You distribute binaries, you supply source. It's that simple. Mic drop.

We're still committed to provide the source for any binaries we have distributed in compliance with the GPL. But the GPL does not force us to continue a business relationship with anyone - so it's possible that access to future binaries will go away.

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 22:33 UTC (Thu) by ben0x539 (subscriber, #119600) [Link]

Surely this is not what anyone who ever published code under the GPL was hoping to achieve.

McGrath: Red Hat’s commitment to open source

McGrath: Red Hat’s commitment to open source

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 23:08 UTC (Wed) by mmcgrath (guest, #44906) [Link]

A more recent tweet: https://twitter.com/geerlingguy/status/1673860031501017091

I've had some interactions with Jeff during this process. Seems like a smart guy and while I don't want to speak for him. I think he started last week with an anger level dialed up to 11, and now he's had a bit more time to process, has decided he's still angry, but probably at a level 4.

The confusion around this event has been really frustrating, if people lose trust over Red Hat, or are angry at Red Hat, it should really be because of something we did or a choice we made - not what the headline grabbers are saying we did.

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 23:21 UTC (Wed) by Wol (subscriber, #4433) [Link]

> it should really be because of something we did or a choice we made - not what the headline grabbers are saying we did.

I've noticed a lot of the commenter subscriber numbers are about 50,000. I guess it's rent-a-mob. They obviously came here some while ago, went away, and this was an excuse to come back :-(

Cheers,
Wol

Subscriber numbers

Posted Jun 29, 2023 7:56 UTC (Thu) by pbonzini (subscriber, #60935) [Link]

I'm #60000 and I subscribed like in 2009. #50000 are fairly old mavens!

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 11:21 UTC (Tue) by rgilton (subscriber, #31330) [Link]

What I'm not getting from McGrath's post is anything about why it has suddenly become a problem for them... Redhat has been operating as a profitable company for years.

My *guess* is that the shareholders are demanding "growth", and sadly they only mean in financial terms. They'll cash-in on the RHEL name for a few years, before no-one is interested in it any more.

He appears to be trying to reframe the idea of FOSS (note: only referred to as "open source" in his post) as just submitting patches upstream. Nothing about avoiding lock-in, etc.

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 12:09 UTC (Tue) by nim-nim (subscriber, #34454) [Link]

> What I'm not getting from McGrath's post is anything about why it has suddenly become a problem for them...

Because changes in the business market take a *loong* time, and we’ve just reached the point where big companies, having dumped proprietary unixes and switched to Linux, were feeling certain enough about Centos’ continued existence, to stop paying RHEL for a *huge* part of their IT systems, deploying no-cost Centos instead.

(And some other companies were starting to sell Centos “support”, as in pay me and I’ll start harassing Red Hat and upstreams in your stead when you have a problem. Mass support ticket opening as a service, no commitment to actually fix anything included)

McGrath: Red Hat’s commitment to open source

Posted Jun 27, 2023 18:50 UTC (Tue) by HenrikH (subscriber, #31152) [Link]

Who says that it was sudden? Most likely they have seen this as a problem for years but "let's not deal with it right now since it's a mine field".

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 18:07 UTC (Wed) by zonker (subscriber, #7867) [Link]

I wouldn't generally look to any company to say "here's exactly the competitive landscape as we see it and the adverse conditions we are trying to navigate around" except as required in public statements. And even then they're going to be as opaque as possible while giving investors enough information that they can't be accused of withholding material information.

Since RHEL is not enough revenue to be broken out separately in IBM's results I don't know if they'd call RHEL subscription numbers as a risk or specifically the amount of annual renewal, etc. IBM's Q2 2023 results should land around July 19, so that might shed some light on this?

Some of the things that you might consider as reasons, note that these are examples and may or may not be accurate.

- Channel sales historically were important for RHEL. As cloud usage goes up there's more difficulty in making RHEL the default.
- I'm not sure what percentage of users/organizations are on EL7, EL8, etc. Red Hat may be looking at a cliff where RHEL 7 is approaching EOL and customers have to migrate to something, but what? Red Hat may want to do as much as possible to cut off other options.
- They're seeing CIQ bragging about winning business from NASA
- Related to the above - one too many "Rocky Linux / AlmaLinux" in the "loss" field for win/loss data from sales.
- Alternately, any uncertainty about the future of Rocky or Alma before this may have fed into higher renewals/subscriptions and they decided to step on the gas.

It's not sufficient for Red Hat to be "profitable" -- they have to be maintaining renewals at a certain percentage, projecting continued renewals and new sales at certain percentages.

That's a long way of saying that "growth" is a fair guess as to "why this and why now?" but I wouldn't expect the company to actually spell that out, especially approaching the end of a quarter.

"He appears to be trying to reframe the idea of FOSS (note: only referred to as "open source" in his post) as just submitting patches upstream. Nothing about avoiding lock-in, etc."

Defining FOSS is sort of like defining punk rock. There's a few bands we'd all agree on as punk, but disagree on others. (Insert your favorite genre of music if punk isn't your thing...)

Everybody argues for their pet definition of open source or FOSS. The only (mostly) agreed-on definition is the open source definition and that only speaks to licenses and not the surrounding practices. So it's not a re-frame so much as Red Hat's definition.

Let's also acknowledge that Red Hat is doing more, even today, than just submitting patches. They're driving new features and doing a big chunk of project maintenance that flows into other distributions. Even if you have never used RHEL, if you've used Linux you benefit from Red Hat's work in a significant way.

retiring but retaining expertise

Posted Jun 28, 2023 13:28 UTC (Wed) by mtaht (guest, #11087) [Link]

I would personally like to find a way to find another mere 2k/mo in income so I could semi-retire, and mostly work on writing down and passing along stuff that I know that not enough folk seem to know, and help make a few improvements there or there to slow start, cake, wifi, etc.

I am not up to major projects anymore, and do wish fervently that somehow more companies realize that expertise is vanishing as the initial wave of linux developers begin to hang it up. But they don't know what they don´t know, and have paved paradise to put up a parking lot.

LibreQos.io, which is my primary project now, I hope succeeds wildly one day for at least some cloudy applications, but donations remain pretty low, the ongoing cost of the initial development, high, and while I like to think longterm a redhat would benefit, taking the time to package it for them seems less worthwhile by the day. That said, without redhat´s investment into eBPF, libreqos would not exist today, so there´s that.

The FLOSS future really seems uncertain so long as the real costs of software development are not better cared for. One of the ironies I keep watching is the whole "made in america" set of arguments made by the current US administration, without a hint that software, at least compiled in america, would help a lot to control a bunch of national security issues introduced by the long tail of ancient software and lagging support.

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 17:39 UTC (Wed) by abufrejoval (subscriber, #100159) [Link]

McGrath writes:
>There was a time, not too long ago, that Red Hat found value in the work done by rebuilders like CentOS. We pushed our >SRPMs out to git.centos.org in a neat package that made them easy to rebuild; we even de-branded it for them. More >recently, we have determined that there isn’t value in having a downstream rebuilder.
and
>This demand for RHEL code is disingenuous.

I believe this decision to be a) stupid, b) disingenouous, c) unargued

and believe Redhat should have held an open discussion right at the beginning of this path: that could have kept them from shooting themselves into the foot.

c) unargued and b) disingenouous
He admits, their opinion around CentOS has changed, but doesn't really argue the "value", which ones he means, how they were assessed, and how they developed.

If lack of direct remuneration is the issue (which seems his major complaint), that hasn't changed as CentOS was never a money-maker.

If the value assessment was much more complex, there must be data, trends and arguments. Leaving them out completely is being disingenouous, in their original stance (what was the value originally?), their current stance (why is it gone?) and in the fact that the change isn't explained.

"This is our call to make" puts the Enterprise Linux ecosystem at their whim. That has always been legally so, and whoever did not see this coming, evidently misplaced their trust.

But now that it's crystal clear Redhat is as decided about community enterprise Linux as Putin about controlling the Ukraine, the community will react.

And that brings me to

a) stupid

He (and all those, who made that call) seems to believe, that if someone benefits from Redhat's work, that should also benefit Redhat, because otherwise they are just parasites.

They also seem to believe that denying any such benefit to parasites will benefit Redhat.

Well, we might have to wait a bit until this plays out, but I believe the reason why Microsoft Windows and Office continue to be available for relatively easy pirating or at under-the-counter prices is because Microsoft learned long ago, that enforcing full licensing compliance would have killed their babies: they understand that it's only because few consumers pay full license cost for M$ that they can continue to charge the bigger prices to corporate and government users. It's because consumers walk into offices and then vote with their feet to support habits and convenience.

Likewise Redhat achieved its current domination only, because there were so many users runing CentOS. And I am pretty sure that a lot of feet will now drag themselves to other Enterprise capable Linux variants which will deprive Redhat of the community user base, which enabled their domination.

Redhat's position in the Enterprise space suffers from evaporation into the clouds, not from the CentOS fanbase. The only thing they can achieve by annoying those who decided not to pay for RHEL after being offered the choice, is faster evaporation to the side as well.

The damage this decision will inflict on people who are not direct Redhat customers will be giant, not because there are huge untapped budgets, but because the cumulative effect on millions of users will add up. But while is no Redhat revenue potential, the pain it inflicts on the community is very real and its appreciation of greed very low.

The eventual damage to Redhat could kill RHEL and I don't know how much money Redhat hopes to make on OpenShift without all those feet in the door.

If I had any shares, I'd sell or sue. And let's please hold onto those bonuses until the story played out.

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 19:20 UTC (Wed) by pizza (subscriber, #46) [Link]

> and believe Redhat should have held an open discussion right at the beginning of this path: that could have kept them from shooting themselves into the foot.

What would RH have got out of this, other than folks saying "Hey, keep providing everything we want from you (ie RHEL), for free, forever?" Seriously asking here.

> He admits, their opinion around CentOS has changed, but doesn't really argue the "value", which ones he means, how they were assessed, and how they developed.

This "value" is from RH's perspective; ie the benefit they get back from investing their resources. While they're under no obligation to share the details of their analysis or their raw data, I don't think it's unreasonable to take RH at their word that their "investment" in making things nearly trivially easy to rebuild RHEL was actively harming them -- due to certain rebuilders selling their own commercial support offerings.

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 20:16 UTC (Wed) by abufrejoval (subscriber, #100159) [Link]

The rebuilders were reselling commercial offerings, long before they took on CentOS. Money evidently wasn't a concern then, because they never talked or complained about it.

When they did take CentOS on, they said it would not change things. And people who had reasons not to use RHEL under contract, therefore trusted that to remain the case. Still there were no complaints about rebuilders.

Then they redefined CentOS as upstream, but waxed about everything except the impact on rebuilders. People were sceptical about the benefits of upstream, but Redhat felt compelled to emphasize that it wasn't about the rebuilders but agility benefits all around.

Now they cut off the repo access, wax on again about this and that, but not about rebuilders and only after being pushed hard, they admit that it was all about the rebuilders after all and Redhat not having any direct benefits from them.

They weren't honest about intention and motives for years. And now they pulled the plug without warning, because "it's our call" and they do not care about collateral damage.

Yes, they were under no legal obligation to share, but they were hiding their intent, did not give warning and as a consequence a lot of people are getting harmed.

And when people are being burned, they aren't very understanding (Maslov), they just won't forgive whoever is responsible.

It doesn't matter if McGrath then explains that you shouldn't have touched the pan without a RHEL support glove, or that it's unfair to expect the hot pan to be cool on the handle without paying for the protection.

Pain destroys trust and followings very radically and there is plenty of data out there on how human networks amplify churn e.g. in TelCos when pain and anger are involved.

And they show just how incredibly expensive it would be to turn such a tide.

IMHO the bean counters didn't have the right numbers for that risk on their balance sheet.

Let's not argue, but revisit in a year or two: by then we might know if they shot themselves a bullet in the foot or vitamins in the arm.

P.S. If anyone had actually invested into making it easier to clone RHEL, then that person should now be sued. I can't believe that would have been an 'honest mistake'.

McGrath: Red Hat’s commitment to open source

Posted Jun 28, 2023 21:09 UTC (Wed) by pizza (subscriber, #46) [Link]

> The rebuilders were reselling commercial offerings, long before they took on CentOS. Money evidently wasn't a concern then, because they never talked or complained about it.

No, the rebuilders (other than perhaps Oracle) were not previously "reselling" anything.

Is it really so dificult to comprehend that something might have grown from a non-problem in the past to something much more serious in the present? (And given the reaction to when they are talking about it now, are you seriously saying the response would have been any different if they'd done this sooner?)

> When they did take CentOS on, they said it would not change things.

No, they announced their plans for the CentOS "brand" almost from the very outset, and the status quo of today is pretty close to what they laid out. (Note that CentOS itself was barely alive by the time they acqu-hired the folks doing all the work!)

> And people who had reasons not to use RHEL under contract, therefore trusted that to remain the case.

Without a contract, you didn't, and still don't, have the right to use "RHEL."

> Then they redefined CentOS as upstream, but waxed about everything except the impact on rebuilders. People were sceptical about the benefits of upstream, but Redhat felt compelled to emphasize that it wasn't about the rebuilders but agility benefits all around.

Yes... and by every measure, CentOS Stream is a massive improvement for folks that don't actually require RHEL, especially when compared to the prior CentOS model that saw updates considerably delayed from when RHEL customers received them.

> Now they cut off the repo access, wax on again about this and that, but not about rebuilders and only after being pushed hard, they admit that it was all about the rebuilders after all and Redhat not having any direct benefits from them.

First, the only point of the repos was to make things easier for rebuilders. Why should RH continue to do work they not only don't benefit from, but actively harms them instead?

> They weren't honest about intention and motives for years. And now they pulled the plug without warning, because "it's our call" and they do not care about collateral damage.

Be careful, you're claiming to know and speak to their intentions and motives.

> Yes, they were under no legal obligation to share, but they were hiding their intent, did not give warning and as a consequence a lot of people are getting harmed.

Who, exactly, is "getting harmed" by this? Everything Red Hat has ever released remains available. It's all Free Software, after all. Or are you seriously claiming that non-RedHat-customers are somehow entitled full access to Red Hat's future work, for free, in perpetuity?

> Let's not argue, but revisit in a year or two: by then we might know if they shot themselves a bullet in the foot or vitamins in the arm.

Yep, time will tell. But I should point out that this is the third or fourth time I've seen this argument spelling doom and gloom over something that Red Hat has done.

> P.S. If anyone had actually invested into making it easier to clone RHEL, then that person should now be sued. I can't believe that would have been an 'honest mistake'.

Huh? I'm not able to follow your logic here. This is about RH no longer handing rebuilders everything they need on a silver platter; the software is _all_ F/OSS, so other parties are completely free to make up the (considerable) difference if they think the effort is worth it.

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 10:03 UTC (Thu) by farnz (subscriber, #17727) [Link]

> Let's not argue, but revisit in a year or two: by then we might know if they shot themselves a bullet in the foot or vitamins in the arm.

Yep, time will tell. But I should point out that this is the third or fourth time I've seen this argument spelling doom and gloom over something that Red Hat has done.

One thing I have noted each time this has come round (and I've been watching this since Red Hat Linux 5.2, so saw the creation of Fedora and RHEL) is that the "doom and gloom" for Red Hat (RH) is based on the argument that the change RH is making will cost RH goodwill, and that loss of goodwill is going to lead to a long term decline for RH.

That's always been a very weak argument about any company - goodwill is notoriously hard to value - and RH in any case also generates goodwill by working upstream, such that any change like this is a blip downwards in a constantly regenerating stream of goodwill.

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 8:01 UTC (Thu) by zeroheure (guest, #165859) [Link]

> Ultimately, we do not find value in a RHEL rebuild

Greg Kurtzer provided an interesting answer to that in 2021 in an interview for thenewstack.io:

"Linux is a community endeavor, and it is freely available. There are thousands of separate projects that are all built together in a Linux distribution, but the distribution is simply the culmination of these packages. While there is value in assembling it, it is still comprised exclusively of freely available software. For that reason, there always needs to be freely available options that are not controlled by corporate needs to ensure proper mitigation of inherent conflicts of interest."

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 8:24 UTC (Thu) by milesrout (subscriber, #126894) [Link]

Good thing there's Arch, Debian, and every other Linux distribution out there, then, eh? What significant Linux distributions are "controlled by corporate needs" other than RHEL and Ubuntu?

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 10:07 UTC (Thu) by aragilar (subscriber, #122569) [Link]

SUSE? I don't think alpine is, but I'm not sure who employs the maintainers (which may or may not change opinions). There are also the cloud provider distros, which may also change the balance between community and "corporate".

I think if you're looking at "LTS" distros though, Debian is the only independent community one as far as I know, and there are no independent RPM-based ones?

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 12:33 UTC (Thu) by kleptog (subscriber, #1183) [Link]

I think that seriously underestimates the effort involved in integration. The effort it takes to go from "works on my machine" and "works on everybodies machine" is about a factor of three, at least. And distributions do a large part of that work. The idea that the author of a software project is going to, without help, make it work on a dozen different computer architectures is farcical.

(Maybe the fact that the number of distinct architectures is much reduced is related to the reduced perceived value of distributions. My impression is that a lot of the 32->64bit transition was done with heavy involvement of distributions.)

Ironically, the argument "Alma & Rocky Linux are harmed by the decision" automatically concedes that what RedHat does adds value. If RedHat did not add value there would not be any harm.

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 13:17 UTC (Thu) by pizza (subscriber, #46) [Link]

> Ironically, the argument "Alma & Rocky Linux are harmed by the decision" automatically concedes that what RedHat does adds value. If RedHat did not add value there would not be any harm.

Yeah, the cognitive dissonance around that point is quite astounding.

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 15:16 UTC (Thu) by Wol (subscriber, #4433) [Link]

I think the cognitive dissonance everywhere is quite astounding - "Wah wah wah why won't Red Hat do all this worthless work for me for free ..."

Cheers,
Wol

McGrath: Red Hat’s commitment to open source

Posted Jun 30, 2023 3:31 UTC (Fri) by ayay (guest, #144420) [Link]

"I think the cognitive dissonance everywhere is quite astounding - "Wah wah wah why won't Red Hat do all this worthless work for me for free ...""

It's more like:
1) they don't *want* to deal with an organization that's so small as to not be worth their time - fine.
2) we proceed to use CentOS, as allowed by the rules they have laid out.
3) things change for them, the rules change - again, that is fine! - but then they proceed to throw the word "freeloaders'" around, as if we were not following the rules they've laid out by their own free will.

Why is it that, when it benefits *them* (market share, patch submissions, "goodwill" or whatever) it's all charitable and noble, and yet, when it doesn't, everybody is just too willing to bleed our most magnificent benefactor dry, such treacherous snakes that we have now suddenly become?

To be fair, other than that, I'd say McGrath was fairly clear. As much as I wish they`d see the value in dealing with others that are not huge corpos eye-to-eye, at least I can appreciate he being refreshingly forthcoming.

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 16:53 UTC (Thu) by mb (subscriber, #50428) [Link]

>I think that seriously underestimates the effort involved in integration.
>The effort it takes to go from "works on my machine" and "works on everybodies machine"
>is about a factor of three, at least.

Did you even think about that before posting it here?

The step "works on my machine" includes developing the whole application, designing the program architecture, designing the code structure, writing the code, writing the documentation, writing the unit tests, writing the system tests, etc, etc.

The step "works on everybodies machines" includes fixing portability issues and packing the application, running some system tests, maybe write some distro specific docs. That's basically it.

McGrath: Red Hat’s commitment to open source

Posted Jun 29, 2023 18:06 UTC (Thu) by pizza (subscriber, #46) [Link]

> The step "works on everybodies machines" includes fixing portability issues and packing the application, running some system tests, maybe write some distro specific docs. That's basically it.

If writing the software represents 90% of the effort, "Works on everybodies machines" represents the second 90%.

It also represents ongoing effort, as it actually requires testing on different hardware (== cpu arch, graphics card, etc) and software combinations ("stock OS release, fully updated as of time X, and likely points in between. For each OS target!) and of course all of that has to be pre-emtively repeated each time you generate a new release or the OS(es) update a component or driver.

McGrath: Red Hat’s commitment to open source

Posted Jun 30, 2023 14:43 UTC (Fri) by jeleinweber (subscriber, #8326) [Link]

That second 90% is spot on.

In Fred Brook Jr's classic "The Mythical Man-Month" he estimated the difference in work between an internal-use-only project and a commercially supported and saleable one at roughly 9X.

To take a personal and trivial example, once upon a time a project I had developed and was supporting got a brilliant suggestion from an end-user. Development time to implement her request was 10 minutes. Project effort to notify two different organizations, get two sets of management approval, document the change, and train two different sets of end-users on the resulting new and additional procedure was 4 hours. I considered this 24X difference between development time and project time to be quite reasonable, and was thrilled that I managed same-day turnaround. The suggestion, as implemented, probably eliminated several weeks worth of busywork for the involved organizations just in its first year. Even at 24X non-development overhead, the ROI was amazing. Thinking that the development work - which is incredibly important - represents the bulk of the effort would be naive. This was paid work, not volunteer.


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK