What to expect in OpenBSD 5.0 onwards

With OpenBSD-current tagged as 5.0-beta it's time to take a closer look at the upcoming release and the processes that make the OpenBSD project work.

Before you start getting all worked up about an upcoming dot-zero release, I'll tell you right away: Don't. Or maybe just a little bit. Release numbering in OpenBSD simply does not work the way most people expect. For this upcoming release, slated for general availability via FTP and other download methods on November 1, 2011 (and CD in pre-orderers' hands up to about two weeks earlier), 5.0 was simply the next available version number increment. OpenBSD releases every six months like clockwork, with the version number incremented by exactly 0.1 each time.

That does not mean that there is nothing to be excited about this time around, only that the OpenBSD approach is about guided and well planned evolution rather than revolutionary changes where large chunks of code are thrown away and replaced with new, untested code with bugs to be explored and exploited until a future dot-something-else release is finally considered stable.

The snapshots, as always available from your friendly local mirror, turned 5.0-beta during the early hours of today (July 19th 2011) CEST, and upgraders will notice two significant improvements before they're done running the installer (which in most other respects is fairly identical to what I described in an earlier article).

The most visible set of changes are near the end of the process:


As the illustration shows, once the install sets are in place, the upgrade program will offer to run sysmerge(8), the program that's specifically designed to help you merge any required changes into your configuration files with minimum fuss and disruption. Once you have chosen to either run the merge or skip it, the upgrade program will offer to fetch updated versions of any non-free firmware detected on your system at first boot.

When that first boot is finished, you will find a number of improvements are in place. You can find an overview of the changes at on the Daily changelog page over at the OpenBSD web site. We'll get into some of the more significant ones in this article.

But before we go into details of those changes, let's take a look at how the OpenBSD development process works. The Goals page on the project web site lays out the main project goals, while the Security page goes into somewhat more detail about the OpenBSD approach to security (including code audits and general all-tree bug sqashing), with a very worthwhile Further reading section at the bottom of the page. If you want to go into even more detail (short of reading source code and commit logs), the Papers page has a large selection of presentations and papers by various OpenBSD developers.

There's a lot of good material linked from that page; among the more recent favorites are Damien Miller's Recent developments in OpenSSH, Henning Brauer and Sven Dehmlow's Puffy At Work - getting code right and secure, the OpenBSD way and Theo de Raadt's The OpenBSD release process: A success story.

The last one is Theo's own first-hand account of how the project has been able to consistently deliver high quality releases every six months since the project started more than a decade ago. It's well worth your time flipping through the entire presentation, but for an outsider the possibly most enlightening slides are the ones contrasting the traditional release process with the OpenBSD one.

In particular, pay attention to the two slides describing the OpenBSD release calendar, slide 16 and slide 17. If you do the math, you see every six-month cycle is divided into roughly four months of intense development followed by roughly two months of stabilization before the release is cut.

Anything that will not fit within four months of hacking without breaking the tree (leaving the system in an unusable state) will not make it in. Any large changes need to be carefully planned and introduced via a number of substeps, some purely preparatory, others introducing user-visible changes. One visible effect of this is that truly incompatible changes such as the NAT and redirection syntax change that occured in OpenBSD 4.7 (prompting among other things an extensive rewrite of a certain book) are exceedingly rare, and if you're capable of reading the commit logs, you will see that it took several years of preparation before the switch happened.

It's all about clear thinking and proper planning, and changes will be introduced gradually and as they fit into the overall system. With all this as the background, it makes sense that 5.0 is just another number. Now let's take a peek at what changed since OpenBSD 4.9 and why it makes sense for both enthusiasts and others to start testing snapshots.

The changes listed here are my particular favorites, what you yourself will consider important may be different, depending on your specific use case:

BIGMEM on by default on amd64 - this is literally a big one. The amd64 architecture has true 64-bit capability, but the ghosts of hardware designs past keep hauting us, with legacy devices that continue to require a lot of dark magic to be handled correctly and safely. Excpect Ariane van der Steldt (ariane@) to be presenting at EuroBSDCon with the gory details. This commit happened fairly early in the OpenBSD 5.0 cycle.

Disk UID (DUID) support in all storage related parts such as mount, and by extension fstab (and DUID-style fstab enabled by default for new installs), so disk device renumbering will not be such a headache the next time you add or remove disks.

The proxies (ftp-proxy(8), tftp-proxy(8)) now use divert(4) sockets for performance, meaning that your rdr-to rules for those proxies need to be rewritten to divert-to rules.

Rewrite of the old /etc/security (a shell script) to the new security(8) by Andrew Fresh and Ingo Schwartze, a much needed refresh of a crucial component.

Various IPv6-related improvements in PF and other parts of the networking code, meaning that traffic using the newer generation protocol is now a bit more manageable. Just like everywhere else, the IPv6-related work is still in the early stages of seeing full scale use, and more likely than not future releases will have more news in this area.

The beginning of the end for ALTQ signalled by the introduction of always-on priority queues available as PF per-rule options, as noted in a previous article. This is an example of the longer term, incremental and well planned change like the ones I mentioned earlier. The internals of queueing and traffic shaping in OpenBSD is about to change in the long run, an it's even conceivable that the current ALTQ grammar will at one point use the newer code, but existing ALTQ setups will continue to work in OpenBSD 5.0.

Further package system improvements Marc Espie (espie@) continues his refinement of the package handling with an overhauled pkg_delete(8) that has several important improvements, including the -a option (click the link to read the man page online).

The Daily changelog page contains a lot more changes than the ones I've highlighted here. Among the things I've mostly skipped are added support for new hardware and a host of platform-specific fixes and enhancements. And of course it's likely that the list of changes will grow visibly over the next few weeks via commits caused by you testing and user feedback.

The exact point in time when a release is cut and shipped off to production is never pre-announced. The best indicator is to look for the commit message that changes the -current version string back to N.m.-current again after a brief period as N.m. A little later, the OpenBSD Orders page will allow preorders, and if you get your order in soon enough, you'll have your CDs and other swag before the official release date.

In about six months' time, you will see blog posts and other news items announcing the change to OpenBSD 5.1-beta, and we will be gearing up for yet another OpenBSD release. In any case, the best way to support the project (that produces, among other widely used software, OpenSSH, more likely than not by a wide margin, your remote login system) in addition to contributing code, testing and direct donations is to go to the OpenBSD Orders page and order one or more items.

Update: The sysmerge run from upgrade feature was backed out in a last minute commit by deraadt@, but it's possible it will return in time for the 5.0 to 5.1 upgrade cycle.

Update 2011-09-09: Pre-orders have started, the 5.0 release now has both a release page and a detailed list of changes since previous release. Expect more details to emerge over the coming days and weeks, and please do go order some items to help fund further OpenBSD development!