Fedora Cloud — Edition, not an Edition, and the future

Project Discussion
#30

And, in the spirit of following my own advice: I think, at this point, there is a quite compelling case to make sure Fedora Cloud image in its traditional form is well-represented, well-maintained, and well-presented to users. That’s because:

fedora-updates-waffle-variant-for-release-34-persistent
fedora-updates-waffle-variant-for-release-34-persistent4800×2700 214 KB

… it represents 15.2% of all persistent (installed for more than one week) Fedora Linux 34 systems observed to date, and …

fedora-updates-waffle-variant-for-release-34-ephemeral
fedora-updates-waffle-variant-for-release-34-ephemeral4800×2700 230 KB

… a whopping 30.5% of the “ephemeral” systems. (Those that show up initially but never report as being installed for more than a week.)

So, that’s a lot! It’s important! I mean, it’s obviously greater than any number of systems from any other deliverable we have, but it’s also greater than all of the non-Workstation desktop releases put together, even for persistent installs.

(Note that CoreOS, Silverblue, and IoT numbers are low because they just started getting the countme feature. They will be higher for F35 numbers.)

The huge difference in persistent vs. ephemeral for Cloud and Server does speak to a different use case for sure. But that 15.2% persistent systems makes me wonder still. Is it really better for those to be Cloud, or should we bring some of that cloud knowledge and expertise into the Server Edition? (Likewise, 5.7% of ephemeral systems are Fedora Server Edition? That’s a lot. Might those be better steered towards Cloud?)

Here is where I really need your expertise. I was really deep in doing cool stuff with Cloud at my previous non-RH job. When I joined the company, I was, like, OMG all these people who have worked at RH for 10 years have no idea how the computing landscape has changed. Now, I am those people. So, I need you to help me understand how the persistent systems installed with Fedora Cloud represent use cases different from those outlined for Fedora Server. (Which, really, is no slouch at these levels; Fedora Server is always going to be in a unique position against LTS/LTM distros and that’s fine.)

Or to put it another way, I actually fairly frequently get the question: “I don’t get it — why isn’t Fedora Cloud just, like, Fedora Server available to me in Amazon / Azure / GCE / DigitalOcean?” Help me answer that question with confidence.

#31

As I read through this thread, it appears to me there are two main ideas presented:

  1. A very detailed history lesson of how we got here.
  2. Those who work on and use the cloud build artifacts do not see Fedora valuing/promoting that work in the same way as other build artifacts.

Just scanning our web sites and documentation, anything we label is Cloud is inconsistent when compared to Server or Workstation. And we know how we got here. To me this shows that we have learned a lot since Fedora.next was first announced and we as a project are in a much better position to define use cases and build artifacts.

Speaking as a Council member, I do not understand what a Council vote would gain us now except to say “yes, it’s time to look at the Cloud builds again”. We already know that, so let’s make that a focus for Fedora 36 rather than rehashing the past.

What I would like to see for the Fedora 36 cycle is for the Cloud group to:

  • Evaluate the group’s name and change it if needed.
  • Evaluate the group’s scope and targeted use cases, revise those if needed.
  • Define the non-Server and non-Workstation builds to deliver in Fedora 36.
  • Work with infra, docs, and other teams to get consistent naming and terminology across Fedora. Just like you can say Server and Workstation now, we need new well-defined terms for anything from the Cloud group.

There is probably more here too, but I guess my main thinking here is let’s make the changes we want to see based on what we have learned.

3 Likes
#32

Oh, this reminds me – maybe too deep of a tangent, but: at some point, in the very distant past, the Cloud SG agreed* to call the cloud image officially Fedora Cloud Base image. Since then, Edition or no, we have been horribly inconsistent with “Fedora Cloud image”, “Fedora Cloud Base image”, “Fedora Cloud Base Image” (capital i), “Fedora Cloud Base” (no “image”), and in at least one unfortunate instance, “Cloud Based Image”.

We should definitely take the opportunity to clean that up — pick and actually consistently use a standard name. At the time, we were contemplating a lot of “cloudy” images, and we wanted something specific, and we felt that “Base image” conjured up the desired idea of “this is a minimal image for building on”. I am not at all attached to this nomenclature.

* note: URL contains “atomic-wg”, but at the time it was actually “cloud”; the pagure repo was renamed as part of all of the above.

#33

Re: history

I definitely think there is value in understanding how we got here and why, both to ensure everyone’s on the same page, and to troubleshoot what went wrong so we can strive for improving process in the future, and hopefully avoid ending up in similar situations. Thanks Matthew (and everybody else) for taking the time to dig up references and provide context around past decisions. I do agree that at this point we should focus on the future, and that is Fedora 36.

Re: naming

I’m partial to keeping Fedora Cloud called as it is now, mostly because I don’t really see what the benefit would be in changing the name. We should definitely clean up and update the documentation so that it’s used consistently everywhere and strive to clarify possible ambiguities.

The way I see it, Fedora Cloud is the Edition and the Fedora Cloud Base image is one of our deliverables. There’s plenty of other deliverables we could produce to cover more specialized usecases and demonstrate best practices if there’s interest.

Re: why Cloud and not just Server

I touched on this before, but fundamentally Cloud and Server are different products targeting different usecases. We’ll put together a proper writeup on this, given that it seems one of the main points of confusion.

1 Like
#34

Yeah, cool.

I’m open to some format other than “PRD” for this, although there are some essential elements of that I’d like to keep. I’m going to create a new thread for anyone interested in diving into the details of what this should look like going forward.

#35

And here is that thread:

#36

Hmmm. I either don’t understand this, or I disagree, or both. :slight_smile:

To me, an Edition is a deliverable. To resort to the dictionary, “an edition = a particular version of a product”. An edition with multiple deliverables doesn’t make sense to me. A Team can have multiple deliverable, one of which might be an Edition.

You said earlier “I see an Edition as a primary deliverable for Fedora that addresses a board market segment.”, which agrees with my understanding.

But I am not sure how to put the two together.

#37

Ah, so in this example, the Server WG produces the Server Edition ISO, as well as the Server netinstall ISO, and the latter is not an edition?

I’m not sure if this strict “an edition is exactly one deliverable” make sense for the Cloud team (WG/SIG) though since there could be several very closely related images just for targeting different cloud providers?

#38

Okay, so there’s some complication. :slight_smile: In this case, the “Edition” is the installed thing, which is (functionally) the same despite different installation paths. For cloud, I can see an image for GCE and an image for AWS having different deployment profiles and maybe even slightly different contents (cloud-init or whatever) all still being the same Edition. (In fact, I have strong belief lightly held that they should be!)

But what I mean is: if there’s a Base Image, but also a “MapReduce Image” and an “AI Image” (or whatever), that collectively doesn’t seem like it can be what we currently mean by Edition at all. But maybe there’s a different way of thinking about it that would fit the cloud problem space better than trying to shoehorn in the Edition naming anyway.

1 Like
#39

So, what I think restates what you are saying above is that even though there may be a lot of variation between what is and is not possible at the platform layer, a Cloud install would be considered the same.
I hold the same belief, that is that it should be considered the same edition across all platforms.

I would love to be working on extending the efforts of the NeuroFedora team (naming one in particular) to make a deep learning Image, but I would still want it to be a Cloud edition since that is what will dictate it’s suitability for the various targeted environments. If the targeted workload can’t be used in a hybrid manner, then it might be better identified as a spin for a specific platform ecosystem.

1 Like
#40

Sorry, I didn’t mean spin. I should have said that it was a platform specific solution.

#41

I’ve posted a Correction of Errors report to the Cloud-SIg issues Issue #352: Cloud Working Group Correction Of Errors Report - cloud-sig - Pagure.io to review what we have learned so far. .

#42

A Team can have multiple deliverable, one of which might be an Edition.

Btw, this one also confuses me. If Edition is a “big thing”, one of the larger Fedora efforts, then I would see it as a joint effort of multiple teams.

So rather than Teams having multiple deliverables, which may include editions, shouldn’t editions be the “umbrella” things for multiple teams to contribute?

#43

Oh! So, yeah, there’s another key historical thing here. See Fedora Working Groups: Call for Self-Nominations - devel - Fedora Mailing-Lists. From that:

Each group will have at least one FESCo member, who will act as a
liaison to FESCo and as a representative of the group at FESCo meetings.

We would like each group to also have representation from other major
areas of Fedora - Quality Assurance, Infrastructure, Release
Engineering, Documentation, Design, Websites, Ambassadors, Feature
Wrangler, Marketing.

This is part of what we meant in creating “Working Groups”, a term we hadn’t used previously in Fedora. The idea was that this team brought together representatives from across the project. In practice (a lot because of struggles in non-engineering groups like docs, websites, ambassadors, marketing) this hasn’t kept up. Maybe it’s worth re-emphasizing this as we go forward, and in particular figuring out how we as Council can better support that.

#44

I’m not reading this whole thread because woo boy, is it huge, but just to note one key thing - there really isn’t any ulterior motive here. I inadvertently caused this whole thing by filling out F35 validation tests at the last moment. I noticed the ‘self-identification test’ for Cloud had not been done, so I did it, and because I’m good at useless trivia, instead of just seeing that /etc/os-release said “something about Cloud” and marking it as passed, my brain stuck on it saying “Cloud Edition” when I was pretty sure we’d swapped in Atomic Host for Cloud as the “edition” years ago. Which is, in fact, what happened. So I filed a bug. All of this is just fallout from that.

This could’ve come up at any point in the last few years, but because no-one with my nitpicky mind happened to run this validation test and question the “edition” word, it didn’t.

2 Likes
#45

sorry, just a further clarification - my bug was literally about taking two words out of the /etc/os-release file. It does not imply that any further change took place or is intended to take place. Nothing at all about the F35 release would’ve happened differently if I hadn’t filed that bug. It’s not as if Cloud was going to be an “edition” for F35 right up until the last second when some sudden decision was made that it wasn’t one. It was never planned or intended to be an “edition” of F35, as it was not an “edition” of F34 or F33 or any other release since - per the Wayback Machine - F25 (F25 was the first release where we listed Atomic instead of Cloud).

1 Like
#46

On another topic, I do think there’s kind of a clear mismatch here if you think about cloud in general.

What’s the most important cloud? Love it or hate it, it’s EC2.

What do you get if you go to EC2, say you want to deploy a VM, and search for “Fedora”? You get the Cloud base images. (You also get the release day Cloud base images, which is kind of terrifying, but that’s another discussion). These are in the obviously-preferred “AWS Marketplace” box and are “free tier eligible”. If you search that box for “coreos”, you get nothing at all from Fedora (you get some flatcar stuff).

To find CoreOS images, you have to search the “community AMIs” dumping ground, which is where they are. If you search “fedora” in that box you get one vaguely current FCOS image, plus a bunch of terrifyingly outdated Atomic Host and Cloud Base images. You have to search “coreos” to get more than this.

From this I conclude that a) EC2’s search is awful and badly needs some better smarts to not show wildly outdated and insecure base images, but more importantly, b) it doesn’t match up that on the one hand we clearly sort of treat Fedora CoreOS as if it’s our primary ‘thing’ for a cloud environment - that’s why it got the spot on the editions roster that was originally occupied by Cloud itself - but on the other hand, the most-prominent Fedora images in the most-used cloud are not FCOS images, but Fedora Cloud images.

1 Like
#47

You make a valid point. Why is it like this today? I can try to explain:

  • The marketplace process in the past has been somewhat manual IIUC.
  • Fedora CoreOS updates 3 streams approximately every two weeks (sometimes more than that).
  • The only way to sustainably keep FCOS up to date in the AWS marketplace is to use automation.
  • We started down this path and I believe everything is in place (waiting on @davdunc to implement the automation in AWS).
  • Fedora Cloud releases once every 6 months.
  • @davdunc updates the marketplace entries for Fedora Cloud when it comes out.

You are definitely following the “new user clicky around and find stuff” path and I’d say it’s a gap we need to close. But I also think a good chunk of our user base will follow the docs and pull the AMI from the website or analyze our published stream metadata (which the website is populated from) to get the AMIs.

1 Like
#48

A post was split to a new topic: Are we still rebuilding/updating Cloud Base images regularly?

#49

After reading the thread, my take is: the Cloud/Atomic WG/DustyM have been producing images intended to be Cloud Edition images. Formally, there is no Cloud Edition. Nevertheless, those images are hugely popular. People involved want to continue producing Cloud images, and want them to remain significantly different in content and format from Server and Fedora CoreOS and other editions. People involved are happy with the “Cloud” name. MatthewM and AdamW want the formal status to be clarified and process to be followed.

I think it makes sense for Council to reassert Cloud as an edition. The deliverable may be called “Cloud Edition” or “Cloud Edition Base”, let’s just make this consistent. It should also be made visible on getfedora.o. The WG should be renamed back to Cloud to reduce confusion. The WG should review the PRD and refresh if necessary.

This sounds about right. It’s mostly clarifications/formalizations/cleanups, rather than any major shifts.

6 Likes