It all starts with a rant
Last week Linus ranted a bit about PAE kernels:
For those that don't know, PAE (physical address extension) is an addition to 32-bit x86 CPUs that allow them to address more than 4GB of memory. That sounds like a good idea on paper, but it is kind of hacky. The address lines allow addressing up to 64GB of memory, but the virtual memory space is still limited to 4GB. It causes the page tables to be larger which impacts performance. It also, as Linus mentions, causes lots of weird corner case behavior for memory management.
Fedora and PAE
Fedora has been building PAE kernels since Fedora Core 6, which is quite a while now. However, not everyone needs PAE support so we also still build a normal i686 kernel. In fact, most people that have 32-bit CPUs aren't running with more than 4GB of memory anyway, so why bother building PAE at all? Well, namely PAE brings in NX.
NX is another x86 feature that stands for 'Never Execute'. Namely, it is a way for the operating system to mark a page as containing either code for execution, or storage for data. It's a hardware assist for helping with things like buffer overflow exploits, etc. Now, there have been software implementations of this for a while and Fedora carried one for a long time called execshield. Fedora carried the execshield patches on the non-PAE i686 kernel for quite some time, however it's an additional out-of-tree patch and since it is an emulation of what NX does in hardware, it's naturally going to be slower. So for quite some time, anaconda has had code to detect if the CPU supports PAE and it will install kernel-PAE by default if so.
So if kernel-PAE exists, and it is used by default if the CPU supports it, what is the problem? Well, the problem is that PAE is a technology that makes a crappy situation on 32-bit CPUs tolerable and it totally irrelevant on 64-bit CPUs. For whatever reason, there are people that still install the 32-bit version of Fedora on their new shiny 64-bit machine. Maybe they like retro-computing. Maybe they're stuck using proprietary software that is only available in 32-bit versions. Maybe they really hate how python sucks on 64-bit because OMG pointers. Maybe they're masochists. I don't know, and it doesn't really matter. What does matter is that when this happens on machines with a larger amount of memory, they've worked themselves into those corner cases that Linus mentioned. Plus, it's just plain inefficient and it bothers me.
(I'll also note that people using actual 32-bit CPUs with PAE and large amounts of memory is another issue, but I can't fix everyone. If they want to do that, then they're going to be stuck with the problems.)
A possible solution
After lamenting and discussing this with a few other Fedora developers and the kernel team, Matthew Garrett suggested just building another kernel flavor for 32-bit Fedora called kernel64 which is a 64-bit kernel packaged in an i686 RPM. The intention would be to have this available in the 32-bit Fedora repositories, and for new installs of 32-bit Fedora on x86_64 machines to use this. Now, this doesn't mean tricking the users and installing the 64-bit version of Fedora when they think they're installing the 32-bit version. It would be for the kernel only. The rest of the install would still be i686 binaries and libraries, and things should operate normally as 32-bit processes.
What's old is new?
Those of you familiar with PowerPC at all will probably recognize this. When Fedora had ppc as a primary architecture, we did something similar. We put both 32 and 64 bit kernels in the ppc/ repository, and installed the .ppc64.rpm version by default on a 64-bit CPU. There were also a bunch of other .ppc64 packages in that repository, as Fedora didn't ship a split out ppc64 version of the repositories at all and for the most part 32-bit userspace made sense even on 64-bit CPUs. 32-bit PowerPC doesn't suffer from the lack of registers that x86 does.
That was done until Fedora 16 and the Fedora PowerPC secondary arch team split out ppc and ppc64, and now ppc is entirely unsupported anyway. Oh well. So much for that sidebar.
What would it take?
In order to get a 64-bit kernel in the 32-bit repositories, we'd either need to do some kind of special mash magic like ppc used to and bring in the .x86_64.rpm version from the 64-bit repositories, or we'd need to actually build a 64-bit kernel as a .i686.rpm. To do the latter, we'd need a 32-bit gcc that accepts -m64. Currently, the gcc in Fedora doesn't support that at all. Assuming that could be done, then building it shouldn't be too hard.
That leaves anaconda and yum. We'd need to patch anaconda to do the 64-bit kernel install on 64-bit CPUs. We'd probably also need to convince yum that it's OK for uname to return x86_64 and still have a completely 32-bit userspace. I seem to recall yum having some special handling on ppc for this, so it's possible it wouldn't be too hard.
Are you going to do this?
I'm glad you asked! I have no idea. It's something that might actually turn out to be a win for people, but it is hard to say without forging ahead and doing it. That's partly why I'm writing about it first. I also don't know how difficult the anaconda or yum parts would be. I wouldn't be at all upset if Fedora stopped shipping i686 separately, but I just don't think that's going to happen any time soon. Maybe this is worth the effort in the meantime. If so, I'll have to cook up a Plan for it in Fedora 20 or 21.