|Why Fedora needs an Updates Policy
||[Feb. 27th, 2010|10:18 am]
A huge thread-o-doom on Fedora and updates and what should be done and why the policy is horrible has sprouted on the fedora-devel list (yes, it's now called email@example.com, but I don't care.) But wait... there is no draft policy yet so how can it be horrible? Oh, yes some of the brainstorming around it
Ok, so if there is no draft policy to comment on, maybe we can look at the existing updates policy and comment on what works and what doesn't. It's right... hm where did we put that... Oh yeah, we don't have a policy at all! So what does that mean? How are updates getting out if there is no policy?? REALLY REALLY SCARY THOUGHT OF THE WEEK: Our policy for updates is basically whatever I feel like pushing that day.
Yes, you read that right. We have no updates policy and which updates get pushed on any given day is up to the person pushing them. That happens to be me. I really dislike this. Why do I dislike this? Why would I not want to bask in this god-like power over three whole Fedora releases? Oh, let me tell you why...
1) I have nothing but my own personal experience and judgement on what to allow into Stable. When I first started pushing updates, I would try and send back those that lacked update information, seemed trivial, etc. When I did this, I got one of two results: 1) people would fix it and resubmit, 2) people would ignore it entirely and resubmit. Eventually I gave up because I do not scale across thousands of updates and I can't personally scrutinize every single one that shows up. Also, this lead to...
2) I have no grounds to stand on when rejecting an update. In practice this hasn't really been an issue since I don't try and reject updates (unless they are truly broken or breaking the push), however there were several times where I would have liked to not push a huge number of trivial updates into stable. Had I removed them from the push, I would have had no policy to back my actions. It would just be me passing judgement, and while I think I have fairly sane judgement I certainly don't want to be in that position. I also know that I cannot possibly be an expert on every package Fedora ships, so my knowledge of the large majority of packages is likely not sufficient to weigh in on whether foobar should be bumped from 1.2 to 2.0 in a stable release.
3) We have hugely differing viewpoints on what a 'stable' release seems to mean across our maintainers. Some view Stable as "once the release is GA, it's very small targeted bugfix and security updates; new versions/development goes on in rawhide." Others view it as "once the release is GA, updates exist to provide a rolling-release style distro while keeping it fairly stable." My personal opinions lean towards the first view, however it would be both arrogant and actively hostile for me alone to decide that is what 'Stable' means for Fedora.
4) There is no oversight of the person doing updates pushes. If I decide that I really don't like KDE, I could just not push those updates. If I decide that I don't like new packages being added to stable releases, I could just not add them via updates. There is no policy, rule, or by-law that defines acceptable updates so it's entirely in the eye of the beholder. Now if I did something like that I'm sure I would be removed from my capacity as a rel-eng member, however people would actually have to notice first. Given the number of updates, the large amount of time it takes to actually do a push, and other factors, it could go unnoticed for quite a while. THIS IS EXTREMELY WRONG. (Note: For the record, I have NEVER rejected/refused to push an update based on personal likes/dislikes of a package or due to personal conflicts with the package maintainer.)
So one might wonder what my current push decisions come down to, since it's all on me right now. Essentially, it's push everything that is submitted. Occasionally I'll reject an update because it's broken or has negative karma that the maintainer seems to be ignoring, but by and large I just push the updates that are submitted. I realize that this can be lots of churn for not only our users but also our mirrors, so probably once or twice a week I do a bugfix and/or security only push (as classified in the bodhi update). Is this good for Fedora? Is it what our users and maintainers want? I have no clue. I HATE that I have no clue.
If we continue to lack an updates policy that can be referred to, I am pretty sure we'll continue to have huge rants on the lists, users confused about what a stable release is to Fedora, and a generally crappy update experience. (Yes, I said crappy. Yes, that's my opinion.) I think the recently introduced critpath mechanism for Fedora 13 will help us as a distro here, but it is not a replacement for an actual policy.
I am going to continue to encourage FESCo and other leadership to look into the updates issues. I would very much like to see a policy drafted, submitted for feedback, discussed, and decided upon before Fedora 13 is released. In the meantime, I will continue to try my best to not totally screw up our distros. Here's to hoping I succeed...
I agree with you that this "no policy" isn't a policy, and it's been reflected in the updates every now and then.
You seem to have the things reasonably clear in your mind, would it be possible that you draft an update policy to pass it on to FRESCo?
May be now it's the perfect time to introduce an update policy, just to take advantage of the new changes in the release cycle.
Cheer up! Your works is very appreciated.
2010-02-27 11:47 pm (UTC)
Re: Launch yourself a draft
I've thought of drafting one myself before. In the end, I always decide against it because I think there is an odd sort of conflict of interest. I'd be happy to work with FESCo (or whomever) as they work through this though.
Of course you have last word on it, but even there was a odd sort of conflict of interest (although I don't agree), it would be a good starting point from a guy that has the point of view from someone in the trenches.
This post proves that you have thought about the issue and you can contribute to make things better.
2010-02-27 06:06 pm (UTC)
Well, I think we already have an solution to that - to not screw up all of our distros. It's called servicepack, witch already supported through packagekit. Simply, easy, and works. What we need only is an monitioring system around it, and something goes wrong then still keeps usable the system - because of rollback..... Also for policy - I think there must be an testing process where at the end comes to servicepack... What do you think?
2010-02-27 11:51 pm (UTC)
Re: One solution
Something like "Fedora 12.1, 12.2, 12.3" etc? Yes, I think that could help some. The problem we have in doing that right now is that we simply don't have enough people.
If we wanted to do point releases or service packs, we'd want to gather the important packages, get them rolled into an updated iso, get that passed through QA, etc. We don't even really have enough people to get the development repo formed into a release as it is. Doing what are essentially staged mini-releases is more work and burden and we'd need quite a few people dedicated to getting that done that stick around for a while.
Personally, I happen to think the Fedora Unity team does this to a very large degree already. I'd like to see their efforts leveraged.
2010-02-28 10:09 am (UTC)
Re: One solution
My girlfriends XP machine likes to lose its sound every couple of weeks and yesterday I was fixing it for her and looking around at all the 'crap' scattered all over her drive and left behind and/or related to her most recent service pack. It appeared to me that the all the software in the service pack acted as one big blob, completely tied to each other, dependent on each other in a unhealthy way, adding unnecessary complexity. It gave me a good feeling about my Fedora box and how each package is separate and deals with its own dependencies in what I perceived to be a much 'cleaner' way.
Maybe service packs kind-of might possibly be acceptable for installing a fresh Fedora install $x time after a release ( a possible replacement for re-spins ) but using service packs for normal updates gives me a really bad feeling. And on the other hand I don't believe a service pack is providing a solution to the problem at hand-a lack of a Fedora updates policy.
2010-02-28 03:32 am (UTC)
Here's an example of one :)
2010-02-28 10:42 pm (UTC)
Re: Here's an example of one :)
But IMHO that's exactly the kind of policy we don't want. :-)
And by the way, they give special preferential treatment to proprietary software in their partner repos which can be updated to the latest version for any reason, or even for no reason at all, whereas Free Software is usually not updated. Yuck!
2010-02-28 06:43 pm (UTC)
Its working pretty well without a policy
That isn't to say that a policy isn't needed, because it would be good to have an update policy. I however like the rapid pace of updates and version churn in Fedora and I think the codification of an update policy would be slanted to always favor more conservative updates.
I like that Fedora updates KDE every time there is a new release from the KDE project. I like how I can get newer versions of things as they appear... and yes it will sometimes lead to breakage, but that was one of the charms of Fedora. On the other hand it seems that some packages are constantly updated, like every other week. That may be an exaggeration but sometimes it feels like that.
Ideally there would be a conservative updates repo and a newest-stuff repo... but I'm sure that would be more work than your already overworked group of Red Hat employees and Fedora volunteers would want to take on... and I don't blame them.
Given the rapid 6 month development cycle of Fedora and the limited lifespan of any given release... the better answer, if stability is the considern, would be to lengthen the development release cycle... but no one wants to do that, right? Another solution would be to have stated LTS releases every couple of releases, but again... that idea has been batted around several times and dismissed.
It seems many wish something would fall between the rapid development cycle of Fedora and the slow development cycle of RHEL. I don't see how that is going to happen.
Not having an update policy and the recent complaints about it will be something that is heavily criticized by those from other distros and the Linux press... but it doesn't mean that the system you have been working with and the decisions you have been making haven't been working well enough. Package makers are supposed to submit their stuff to testing, people are supposed to test and provide feedback, and only when a package is deemed sufficiently ready should it be considered. I think it is better to leave it up to the package maintainers themselves on what version of a piece of software they want to release... unless of course is an underlying package that disrupts things above it... and you have tried to address that by identifying core/critical packages and putting more rules on their being updated.
I would hope any update policy Fedora comes up with would retain the current flavor of Fedora with rapid and constant updates... rather than being stuck with older releases of things when upstream has fixed a lot of bugs and released newer versions with additional features. If you don't retain that quality then it will just encourage the development of yet more third-party repositories with newer software and just make an even bigger mess. This gets back to the seeming constant desire for Fedora to define itself and who it is targeting... and then potentially limiting itself to those more strictly defined goals. I for one like it fast and loose... but I'm just a user. :)
> I realize that this can be lots of churn for not only our users but also our mirrors,
> so probably once or twice a week I do a bugfix and/or security only push (as
> classified in the bodhi update). Is this good for Fedora?
No, sorry. I don't see what this is supposed to accomplish at all. Mirrors need to update anyway, and normally do this automatically via rsync cronjobs; whether updates are bugfix-only or not does not change anything for them, and they'll have to pull those enhancement updates sooner or later anyway. Users can just not update as often if they don't want updates that often, nobody forces them to update immediately as an update is pushed. But for users with slow connections, getting the updates spread across several days is actually a godsend compared to huge blocked updates taking hours to download. For those users, the more often we push, the better.
An additional risk with selective pushes is that maintainers may expect updates queued at the same time to go out together, even if they're not part of a group. Pushing only some categories of updates breaks that assumption and can lead to broken packages.
I also disliked the F13-only pushes this weekend, I don't see why my F11 and F12 updates of the same packages which I queued at the same time can't go out at the same time if updates are being pushed anyway.
IMHO you should really do only full pushes, and as often as possible. I don't see whom doing delayed block pushes helps. Those who want one big block a week can just choose to only update once a week, problem solved. That doesn't mean we can't push more often on our end (and make all the other users happy while not hurting the former).
2010-03-01 01:27 am (UTC)
Re: bugfix-only pushes
>An additional risk with selective pushes is that maintainers may expect
>updates queued at the same time to go out together, even if they're not part
>of a group.
An expectation is not a guarantee. If they want packages to go out at the same time, they need to be bundled. Anything else is a gamble on their part.
>I also disliked the F13-only pushes this weekend, I don't see why my F11 and
>F12 updates of the same packages which I queued at the same time can't go out
>at the same time if updates are being pushed anyway.
I'm going to be very blunt and clear about this. I don't care one bit what you like and dislike. At all. If you don't understand something, fine. Ask away. However the fact that you dislike it is irrelevant.
To answer your point, I don't have all day on the weekends to sit by a computer and babysit a full update push. Something inevitably fails that I have to fix manually and resume. The F-13 pushes are small and not yet to the point where they take hours and hours to compose, so they are easy enough for me to do and keep Branched rolling.
2010-03-01 07:41 am (UTC)
Updates of DEs
"I like that Fedora updates KDE every time there is a new release from the KDE project. I like how I can get newer versions of things as they appear... and yes it will sometimes lead to breakage, but that was one of the charms of Fedora."
New releases of desktop environments (besides bug fix releases) shouldn't be pushed to stable, imho. Last week I had to update 200 packages or so, because KDE has been updated to ver. 4.4.
If you really want to have the latest version of your favourite DE, you should perhaps take a look at the http://kde-redhat.sourceforge.net/ repository e.g..
2010-03-03 08:46 pm (UTC)
Re: Updates of DEs
This is also a point I don't really get. The only DE which gets updated from one major release to another is KDE. GNOME, XFCE,... stays with the major version from relase date.
Afaik the "KDE update policy" was introduced with KDE4.0 as an justification to include this raw version (excuse: "you will get KDE4.1 once it is released"). But it seems that since then this "update policy" become the standard for KDE packages.
I#M not sure if it is good to give KDE this special position in the long-run.