Last weekend in Portland, Oregon, the Software Freedom Conservancy hosted a new conference called the Free and Open Source Software Yearly.
And long-time free software activist Bradley M. Kuhn (currently a policy fellow/hacker-in-residence for the Software Freedom Conservancy) hosted a lively panel discussion on “the recent change” to public source code releases for Red Hat Enterprise Linux which shed light on what may happen next. The panel also included:
- benny Vasquez, the Chair of the AlmaLinux OS Foundation
- Jeremy Alison, Samba co-founder and software engineer at CIQ (focused on Rocky Linux). Allison is also Jeremy Allison – Sam Slashdot reader #8,157.
- James (Jim) Wright, Oracle’s chief architect for Open Source policy/strategy/compliance/alliances
“Red Hat themselves did not reply to our repeated requests to join us on this panel… SUSE was also invited but let us know they were unable to send someone on short notice to Portland for the panel.”
One interesting audience question for the panel came from Karsten Wade, a one-time Red Hat senior community architect who left Red Hat in April after 21 years, but said he was “responsible for bringing the CentOS team onboard to Red Hat.” Wade argued that CentOS “was always doing a clean rebuild from source RPMS of their own…” So “isn’t all of this thunder doing Red Hat’s job for them, of trying to get everyone to say, ‘This thing is not the equivalent to RHEL.'”
In response Jeremy Alison made a good point. “None of us here are the arbiters of whether it’s good enough of a rebuild of Red Hat Linux. The customers are the arbiters.” But this led to an audience member asking a very forward-looking question: what are the chances the community could adopt a new (and open) enterprise Linux standard that distributions could follow. AlmaLinux’s Vasquez replied, “Chances are real high… I think everyone sees that as the obvious answer. I think that’s the obvious next step. I’ll leave it at that.” And Oracle’s Wright added “to the extent that the market asks us to standardize? We’re all responsive.”
When asked if they’d consider adding features not found in RHEL (“such as high-security gates through reproducible builds”) AlmaLinux’s Vasquez said “100% — yeah. One of the things that we’re kind of excited about is the opportunities that this opens for us. We had decided we were just going to focus on this north star of 1:1 Red Hat no matter what — and with that limitation being removed, we have all kinds of options.” And CIQ’s Alison said “We’re working on FIPS certification for an earlier version of Rocky, that Red Hat, I don’t believe, FIPS certified. And we’re planning to release that.”
AlmaLinux’s Vasquez emphasized later that “We’re just going to build Enterprise Linux. Red Hat has done a great job of establishing a fantastic target for all of us, but they don’t own the rights to enterprise Linux. We can make this happen, without forcing an uncomfortable conversation with Red Hat. We can get around this.”
And Alison later applied a “Star Wars” quote to Red Hat’s predicament. “The more things you try and grab, the more things slip through your fingers.” The more somebody tries to exert control over a codebase, the more the pushback will occur from people who collaborate in that codebase.” AlmaLinux’s Vasquez also said they’re already “in conversations” with independent software vendors about the “flow of support” into non-Red Hat distributions — though that’s always been the case. “Finding ways to reduce the barrier for those independent software vendors to add official support for us is, like, maybe more cumbersome now, but it’s the same problem that we’ve had…”
Early in the discussion Oracle’s Jim Wright pointed out that even Red Hat’s own web site defines open source code as “designed to be publicly accessible — anyone can see, modify, and distribute the code as they see fit.” (“Until now,” Wright added pointedly…) There was some mild teasing of Oracle during the 50-minute discussion — someone asked at one point if they’d re-license their proprietary implementation of ZFS under the GPL. But at the end of the panel, Oracle’s Jim Wright still reminded the audience that “If you want to work on open source Linux, we are hiring.”
Read Slashdot’s transcript of highlights from the discussion.
SFC’s Kuhn: I’ve often called the business model, “If you exercise your rights under GPL, your money is no good here.” The argument that Red Hat makes for their GPL compliance is, “All we’re doing is saying ‘We don’t want a business relationship with people who exercise their rights under GPL.'” And it’s hard to find in the GPL any section that says “You have to maintain a business relationship with somebody…”
SFC’s Kuhn: But I think the interesting thing is, what do we do about the intimidation part of it? The agreements that Red Hat puts forward have the right to audit every single customer. At any time, if you’re a customer of Red Hat, they can come into your enterprise — you agree to this, if you want their services — and they can look at every server and see whether or not you’re running a copy of RHEL that has a subscription. And if you are running copies of RHEL that don’t have a subscription, you have a choice to start paying them more money, or not be their customer any more. And a lot of people are in fear about this. So how do we deal with this, as a community that wants to rebuild this stuff, If the folks who have the source code are afraid to give it to us because they might lose their business relationship.
Oracle’s Wright: I’d go even further … What their agreement says — and to be clear, I’m not going to come up here and accuse Red Hat of breaching an agreement, violating the GPL or anything else. But what their agreement says is it’s a material breach if you distribute this code. It doesn’t just say we can terminate the business relationship. By saying it’s a material breach, there are other implications — like potential damages and other things. Right?
Like I said, I’m not going to accuse them of anything, but I think it’s kind of funny that they say that people who are rebuilding don’t add value, when Oracle has many years of kernel contributions that they’re including in RHEL and MySQL and Java. But besides that, I think there are other copyright holders — not us, because I think frankly this crowd wouldn’t like us to be an enforcer, even if we thought that was the right thing to do — but there are other copyright holders, maybe sitting on this stage, or maybe watching out here, that might have an opinion about this.
Audience question: Would you consider adding some features that RHEL doesn’t do, such as high-security gates through reproducible builds?
AlmaLinux’s Vasquez: 100% — yeah. One of the things that we’re kind of excited about is the opportunities that this opens for us. We had decided we were just going to focus on this north star of 1:1 Red Hat no matter what — and with that limitation being removed, we have all kinds of options.
Samba/CIQ’s Alison: Yeah, sure. One of the things that I’ve been working on in the last few months is FIPS certification. If you don’t know what that is, you’re very lucky; if you do know what it is, my commiseration. We’re working on FIPS certification for an earlier version of Rocky, that Red Hat, I don’t believe, FIPS certified. And we’re planning to release that. We got the go-ahead to release that as open source. So all the changes for FIPs certification for Rocky will be published… Obviously it won’t be upstream, because Red Hat’s not going to take that back, but it will be available for people who want to do FIPS certification. God help you.
Oracle’s Wright: The OpenSSL folks have now released an open FIPS module. So that’s kind of huge.
Samba/CIQ’s Alison: Sure, but not for this version. We’ve backported that to an earlier version.
Audience question: Are you planning to expand upstream contributions?
Oracle’s Jim Wright: So, we’re hiring a ton, right? We’re going to be hiring a lot, effectively, to have our own compatible distribution. Now as to what’s upstream, obviously we upstream the vast majority of our work to the kernel tree. And frankly I’m not sure that Red Hat would even want our upstreams. And it would be difficult to manage under the circumstances.
SFC’s Kuhn: And if Jim at Oracle does hire you, tell them you won’t work for ’em unless he lets you keep your own copyrights on your contributions to open source. [Laughs]
Samba/CIQ’s Alison: I live upstream… The stuff I write is built upstream, and Red Hat is downstream from me. And as CIQ grows and has more contributors, then yes, more work is going to go on upstream as the business grows.
AlmaLinux’s Vasquez: As the one that doesn’t have a company, we are already involved in Fedora, right? The community that is around AlmaLinux is a bunch of people who have been involved in the entire ecosystem for a very long time. So there’s no question of whether or not we’re going to continue or expand… Whoever joins AlmaLinux contributes wherever they want to, whenever they want to. And we certainly continue to encourage people to contribute upstream. For sure.
[An audience question came from Karsten Wade, a one-time Red Hat senior community architect who left Red Hat in April after 21 years.] I was the architect who was responsible for bringing the CentOS team onboard to Red Hat, and all of that deal, and then Engineering Manager and was on the board for a while — Red Hat liason and other junk. So here’s the question:
You all talked about various versions of digging around in source in a very disparaging manner. And It strikes me that it’s possibly disingenuous. And so I’m asking you to — like, not to get into the technical weeds, but to really consider this. I’m familar with the rebuild process of what CentOS has gone through. CentOS has always been a clean-room rebuild, without knowing what was in the build tree around it. So when they do the rebuild, they just run a rebuild, and then whatever doesn’t work, you go back and manually figure out, and start making guesses based off of Fedora. So it’s always been steps removed, right? It’s — everyone else has insisted that CentOS and RHEL were the same thing. And so finally people just said, “Well it’s the same thing, or it’s good enough.” Right? So what we’re looking at now is the source is there. It’s a couple of steps removed. It’s not in the source RPM.
Now whether source RPM is a GPL-required artifact or not — I don’t know, right? But the —
Former Red Hat community architect Wade: — the source is still there, but the.. Well, okay. So my question to you is, isn’t all of this thunder doing Red Hat’s job for them, of trying to get everyone to say, “This thing is not the equivalent to RHEL.” Right?
AlmaLinux’s Vasquez: Yeah, it makes perfect sense. But I would like to kind of say — like, we’re not afraid of digging around in source code. Right? That’s why we’re doing what we’re doing.
Samba/CIQ’s Alison: It’s make-work. It’s like when Red Hat stopped publishing the kernel patches. It’s make-work. People will figure it out. Why do it? “Oh, yes, we’re going to make your life more difficult.” Thank you, congratulations, you’ve wasted a bunch of people’s time. Great. Okay, now can we get on with contributing and working together?
Oracle’s Wright: To go not too far, but one step into the weeds — half a step into the weeds?
Saying that some piece of code was extracted from one thing and put into another thing — and that that other thing that you put it into, all the source is available? — I think is a logically specious conclusion.
When you backport something from one package to another, that does not mean that the thing you backported it to has all the code. A lot of times modifications are made in backporting. So the argument that the code is all out there, I think is just factually incorrect.
Former Red Hat community architect Karsten Wade: It’s always been that case, though, Jim. That’s the point. My point is that if the goal of Red Hat is to say “You’re thing is not the same as RHEL,” right? Then you’re proving the point. By going out and making all that noise and saying, “Now you’ve made it so much harder and so different, our thing can’t be the same as RHEL.” It never was. The sources that run from the build system, and all the packages in the build system, were never available. CentOS was always doing a clean rebuild from source RPMS of their own. And then they’d build those from disk.get. I mean it’s been this long. So yes it’s true, it’s like the patches — it’s make-work, it’s making it more difficult. So aside from it being more difficult… Are you not doing Red Hat’s job for them by making so much thunder and noise about how this is so different and such a big break of trust and such a big thing, instead of just saying “Oh, well the source is over here now. Thanks. We’ll just build from there. Have a nice day.”
SFC’s Kuhn: So I have to respond to Karsten’s point. The first is — and I told Karsten this back when he was bringing CentOS into Red Hat. That my big concern with CentOS being integrated into Red Hat was coming from the perspective of somebody that spent most of their career enforcing the GPL. The reason I, for a good 12-year period, didn’t worry about whether RHEL was complying with the GPL or not, was because CentOS, as an independent project, was getting something that all the CentOS developers were telling me was relatively easily constructed — with some work, as you point out Karsten — and was a match for a rebuild of Red Hat from the sources that were released due to GPL requirements on Red Hat. So that watchdog aspect of CentOS was what was most interesting to me — because I’m not a CentOS or a RHEL user. Or an Alma user or a Rocky user, sorry to say. I’m certainly not an Oracle Linux user. I’m a Debian. But I want to be sure that folks living RHEL/CentOS enterprise Linux space are getting the things they’re right to get under GPL. And CentOS was that watchdog.
Now I have two other watchdogs to talk to, Alma and Rocky. (I’m not counting you Jim. Sorry.) And they’re telling me, “Hey, it’s hard right now for us.” And then I get worried, as a GPL enforcement. I’m like, wait. If the people who are trying to exercise the rights under GPL are telling me, “It’s hard right now to exercise our rights,” I get worried as an enforcer.
Then I look at another aspect of it, which is kind of what Jim was getting to his with quoting from Red Hat’s statement about open source. Which is I always had viewed Red Hat as a company that wanted to be a top-tier open source company, and from my point of view, if you just barely make it into being compliant with the GPL, I give you a C. It’s a passing grade, but when I was at school at least, I think most people in this room when they were in school, they really worked hard to get the A not the C. And I’m very, very sad to see that Red Hat wants no more A’s in GPL compliance. They’re going to settle for straight C’s.
Samba/CIQ’s Alison: And to be honest, none of us here are the arbiters of whether it’s good enough of a rebuild of Red Hat Linux. The customers are the arbiters of is this good enough for our purposes. And customers who really need absolute and complete fidelity? Buy Red Hat. That’s what I would say. Go out there, give them money, get the real thing. You know, if you can live with something that’s close, then there are alternatives.
Oracle’s Wright: This is sort of an important point. People ask why we’re doing this, and the answer is because customers require it in substantial part by virtue of other projects that target compatibility. Right? They only want to build and test on a single system. Some of them are open source, some of them are proprietary products that the customers are using. And so why do it? The reason is that customers — and it doesn’t have to be paying customers — end users require it.
Audience question: With Red Hat pushing the community away, what the odds of creating a new open enterprise Linux standard that distributions can follow?
AlmaLinux’s Vasquez: I think, to answer the direct question? Chances are real high. Right? This is a very new thing — we’re, what, three weeks into it? So I think everyone sees that as the obvious answer. I think that’s the obvious next step. I’ll leave it at that.
Samba/CIQ’s Alison: Remember, enterprise Linux is what the customers say it is. And so if the customers say something that’s close to Red Hat but not exactly Red Hat is good enough, then that’s what we will be. If the customers say, “No, it has to be a rebuild, bug-for-bug compatible, then that’s what we’re going to try and be. We’re going to try and meet the market needs. We’re going to try and do what the users require. Because, I mean, that’s the whole point of this thing, is to produce freedom for the people using, developing, creating, using the software. The maximum amount of freedom.
Source link
