jwboyer (jwboyer) wrote,

Failing Fast

Users of the Fedora exploded kernel git tree might have noticed that it has been stale for a couple of weeks now. What they might not know is why that is, and when it is going to be fixed. The answer is somewhat complicated and I'll try and summarize here.

The Fedora kernel team recently tried shifting to using the exploded tree as the canonical location for Fedora kernel work. The benefits and ideas were written here, and most of those still stand. So I went to work on some scripts that would make this easier to do. The results weren't terrible. Things worked, kernels were still built, and the exploded tree was spit out (albeit at a different location). By some measures, this was a success.

Except it really wasn't. Some of the motivation behind the change was to increase participation and transparency around the Fedora kernel. However, while we worked through the new process we quickly realized that it was much more cumbersome to actually produce kernel packages. Since the primary output of our team is packages, that seemed like a pretty bad side effect. Similarly, transparency wasn't decreased but the change made things much more confusing. Changes in the exploded tree weren't synced 1:1 back to the Fedora package repo, so it appeared like big code drops there instead of discrete commits. Telling people to look at the exploded tree was an option, but in Fedora's package-centric world it was a bit out of place.

So what do you do when you've spent time making a change and it isn't working out? Well, you scrap it. There's little point in pushing on with something that even the primary author finds to be awkward. The payoffs weren't likely to materialize in any sort of timeframe that would make it worth the continued effort there. Was the time wasted? In my opinion, no. It answered some questions we've had for a while, and showed us that both the tooling and processes Fedora uses aren't really amenable to working with exploded sources.

Unfortunately, the exploded tree has been stagnant since we changed back. However, I don't think it will remain that way forever. We tweaked some of the things we do to build the kernel package such that taking that content and creating an exploded tree will be easier. With a bit more scripting, it should be possible to almost automate the creation on every successful build. I'll be working on that off and on for the next few weeks. Hopefully by Flock I have something cobbled together to get things back on track again.

It's been a while since I've had what I would consider a pretty big failure. I make mistakes all the time like everyone else, but those are generally small and on short-term things. In a discussion with someone, they asked me if I was bothered by this being a big bigger. I'm not. Personally, I don't care if I fail spectacularly or not as long as I learn something from it. I think I did, so the exercise was worthwhile to me. Failing fast is a really good way to work through some complicated scenarios, and is far better than staying stagnant out of fear of failure. Time to move on to the next idea!

(A couple of notes:)

It was pointed out on the list that RHEL tooling can build from an exploded tree, but it seems it does this with some odd hacks that we'd likely try and avoid in Fedora. The workflow between RHEL and Fedora kernels are also massively different. I would love to see some more commonality between the two, but not at the expense of forcing ill fitting process on either one.

Pagure, is where the exploded tree was hosted in the interim. We tried it with the idea that making multiple committers there would be easier than kernel.org. It is a neat service, but I'm not sure we'd really use many of the features it was designed around. The only complaint I heard was that browsing via the web interface was slow.
Tags: fedora

  • Flock Krakow 2016

    The annual Fedora contributor's conference, Flock to Fedora, wrapped up last week. From everything I've seen and heard, it was a smashing success.…

  • Time for an Alternative

    I've been doing kernel development or maintenance for a large portion of my professional career. It started at my previous employer and continued for…

  • When is a kernel bug not a kernel bug?

    Think of this scenario: You're sitting at your shiny Fedora install and notice a kernel update is available. You get all excited, update it through…

  • Post a new comment


    default userpic

    Your reply will be screened

    Your IP address will be recorded 

    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.