If It Runs OpenBSD, It Has To Be Important

If we are starting to see targeted attacks on OpenBSD systems, the world has become more interesting.

A lazy Sunday turned a tad more interesting when Dragos Rui, a person usually involved in high-quality security work (or somebody with access to his twitter account) tweeted this:

Analysis of recently deleted data on compromised server has turned up an OpenBSD boot sector trojan worm? Volunteer deconstuctors?

There have been no further updates as of this writing, and without access to the actual files in question, we can not offer any real meat on the actual payload, so any preliminary conclusions will be uncertain at best. But it is possible to come up with a threat assessment based on general knowledge of what it would take for a boot sector based OpenBSD exploit to make it into the wild.

OpenBSD is used in a wide range of environments (see the OpenBSD at work page for some examples, the list is by no means exhaustive), and since it is commonly perceived to be one of the most secure general-purpose operating systems available, it should be no surprise to find OpenBSD systems in mission critical roles.

So the motive for trying to forge a way in, past a system that is generally considered trustworthy, is obvious. If you hit the right targets, the potential payoff could be huge.

Then it becomes interesting to consider how you would go about to compromise a system running OpenBSD. If we leave aside social engineering such as bribing the sysadmin or stealing the passwords database for the moment, the traditional, and obvious, way to compromise a system is to find an exploitable, unpatched bug and use that to take control of the system.

The OpenBSD source tree is available to the world (in fact, OpenBSD was the first project to make its CVS repository available via anonymous read-only CVS), but writable only by small group of developers. The developers tend to concentrate on the in-development -current, but they do produce a small number of patches to the released version that become available via the patches page. If you want to study the OpenBSD development process more closely, there are a number of good articles and presentationas available via this page, but the important message is that insisting on code correctness and continuous code audits have yielded quite solid results. Exploiting any bugs you would be lucky enough to find in OpenBSD is harder than elsewhere.

But apart from patches to -stable, OpenBSD users usually do not update their systems by recompiling source. Neither do we usually run automatic system upgrades via an upgrade service. For packages, $ sudo pkg_add -u (assuming your PKG_PATH is set to something sensible) provides binary upgrades. For base system upgrades (the only time the boot sector code is likely to be updated), we tend to run the installer in upgrade mode and point it to a set of known good file sets. If the installer finds that a file set does not match the expected SHA256 checksum, it will warn you in no uncertain terms. The same messages will turn up if you, like me, tend to install snapshots on at least some systems as they become available, but sometimes forget to copy the correct bsd.rd into the right place.

The lesson here is that to taint an OpenBSD system, your most likely way in would be to replace a full set of installation files on your likely target's usual mirror, complete with checksums that makes the installer report your modified file sets as genuine. This has two obvious implications: one, you should never install anything on a mission critical system unless you find identical files on several different mirrors, and you should make sure that checksums from one mirror match the files on another. Watching the errata page for the release you are running is a good idea, too, of course.

This gives the proper context to the suspected exploit Dragos reported. The suspect still has some interesting properties. The boot sector code is very small and any changes would be relatively easy to spot once the system is running, but the code there runs early in the boot process and is there mainly to fetch other code that makes sure the system boots. The only useful modification here for an attacker would be to modify the boot loader code to load something that replaces the OpenBSD kernel and performs actions whatever the attacker wants.

The result of modifying the early stage boot code would more than likely be a trashed system unable to boot, but an appropriately skilled attacker who managed to insert code in the right places might be able to pass off a subtly modified system as a genuine one and keep it running for long enough to matter. That's why it will be very interesting to hear whatever real information becomes available about this suspected OpenBSD-targeting attempt.

If we are starting to see attack attempts specifically targeting OpenBSD systems, it could be an indication that at least some criminals have achieved a level of skill, or at least reached a new level of ambition. I have a strong feeling that the OpenBSD developers' efforts have paid off and creating a workable exploit will be very hard, but in the meantime, now is not the time to be slacking, even if all your critical system run OpenBSD. But you have a head start.

---

If you are curious about the status of the The Book of PF, second edition, preorders have started, and a PDF version is available right away. Physical copies will start shipping as soon as they exist, likely around November 10th. (Update 2015-03-05: For fresh reading material on PF, you're better of with the newer Book of PF, third edition, which became available in late 2014.)

EuroBSDCon 2010: The Finest Software Tool Is Alive And Well

From mainframe replacements to firewall appliances, the BSD family of systems is a toolbox flexible enough to baffle insiders and newbies alike. EuroBSDCon 2010 was good fun.

I arrived in Karlsruhe on Thursday night, and ran into Erwin Lansing, Mark Linimon and a few other FreeBSD devsummit attendees at the perfect moment to tag along to dinner at a sort-of-greek place. Forgettable food, but fortunately the dark beer was quite drinkable and the various FreeBSDers made for good company.

Then up relatively early on the Friday. My own path through the conference started with the by now fairly familiar PF tutorial, which as you may be aware, is a close relative of The Book of PF, really soon now out in its second edition. Off the top of my head I'm not sure how many times I've given some version of the PF tutorial, but BSD-DK members will find that this edition of the slides borrows heavily from the somewhat swifter paced introduction they saw in Copenhagen this August.

Among my seven attendees were several who had hoped to be able to catch early copies of The Book of PF, second edition, which I had strongly hinted in the tutorial description would be available by now.

That did not happen, unfortunately, but it's getting very close -- last week entering final-really-final corrections to the index and the laid out book itself took up a good part of my non-office time, and the last word from my contact at No Starch was that the complete and final PDF would land in my inbox for final approval before going to the printers. Amazon.com now lists a likely delivery date of November 15th, which I for one think is a rather realistic guesstimate.

The tutorial went roughly as expected, unfortunately without live demos (demo equipment has a tendency to break badly during air transport or soon afterwards just in time muck up your presentation), and produced just enough good questions that it's likely useful to keep up the effort to maintain the tutorial. Slides are available from roughly where you would expect them.

After the session I ran into Thordur Bjornson (thib@) in the bar-cum-waiting area downstairs at the hotel, waiting for various other OpenBSD developers to arrive. This year's conference had a pleasantly larger than usual number of OpenBSD topics on the program, with some rather interesting talks scheduled for the first day. After a few beers we had reached critical mass with the arrival of among others Theo (deraadt@), Henning Brauer (henning@) and Felix Kronlage (fkr@) we found food and beer at a conveniently local eatery. Then back to the hotel bar for (slightly better) beer and mingling with arriving conference attendees and organizers.

The conference proper started on the Saturday with a "Software tools" themed keynote by Poul-Henning Kamp. PHK is always witty and fun to listen to, and he took us through a number of fresh perspectives on how, even though the world has changed dramatically in several ways and the BSDs still manage to kick ass, we need to keep up the effort to stay relevant.

FreeBSD jails have been a major attraction for quite a while, and I took in the two back to back jails talks by Bjoern A. Zeeb and james Britton. Both talks were fun refreshers on jails, with each presenting a preview of what may turn up in FreeBSD 9 jails code, plus of course tidbits like Bjoern's anectdote about setting up a million jails on the same physical server.

I was intending to attend thib@ and oga@'s OpenBSD on large memory systems talk, but was unfortunately diverted into a meeting that lead to me becoming slightly more involved in future EuroBSDCons. More details on that at a later time, the meeting ran for long enough that the next talk I did catch was reyk@'s iked(8) talk.

If you're on OpenBSD 4.8 or newer, man iked will give you the full story. If you're not that fortunate, it's nice to know that OpenBSD 4.8 gives you a new key exchange daemon for IPsec, up to date with the latest versions of all relevant protocols and able to handle all the nasty little details for IPsec communication with operating systems this column would rather not mention. A good talk about a very useful program, and during the questions part, Theo de Raadt pulled out OpenBSD 4.8 CD sets for attendees who had not already preordered to buy. It's almost a month until the official release date, but the CDs do exist and are likely on their way to early preorderers.

Henning's talk about the state of the OpenBSD networking code was good fun and to the point as always. No shocking new revelations for those who have followed the subject closely like yours truly, but do look out for this talk's slides when it hits the openbsd.org papers section along with the other EuroBSDCon 2010 presentations, hopefully soon.

The social event was conveniently placed in the hotel restaurant and bar area, where something called "Phönix Disco" had set up their equipment, including earth mover size speakers and a mirror ball hanging from the ceiling. Inbetween the dining noises and music, techie talk (at my table, OpenBSD internals, with a helping of ACPI insanities) could be heard. At some point they turned on their laser strobes, which made me queasy enough that I retired to my room soon enough to be in reasonable shape for the early Sunday morning sessions.

The 09:15 sessions on the Sunday offered a choice between Dru Lavigne's BSD Certification talk (highly recommended for your next event if you haven't taken it in already, she not only writes well, she's a brilliant presenter as well), and "Hacking NanoBSD for fun and profit" by Patrick Hausen, who has twisted the NanoBSD setup (originally intended for tiny machines) to serve as a basis for maintaining a hosting environment consisting mainly of regular-sized and capable servers. I'm already quite familiar with the bsdcertification.org efforts (and I recommend getting involved if you aren't already), so I decided to try Patrick's talk which turned out to be very enjoyable and presented some good ideas that could very well be carried over to other BSDs.

One other interesting talk that morning was Hans-Martin Rasch's "From Mainframe to FreeBSD", chronicling the gradual and successful migration by a subscriptions and mass mailings company from their legacy mainframe based system to an all-FreeBSD setup, of course shedding costs in the pretty serious range along the way.

The next time slots had a "Quo vadis ZFS" talk by Martin Matuska, that fortunately contained not the usual "see how great ZFS is" but rather focused on the challenges involved in using ZFS code, technical and legal as things stand today. The FreeBSD project seems to have concluded that the way ZFS is included in their code base does not pose legal problems in itself, but there could be other submarine issues. Apparently the NetApp vs Sun patent suit had ended with one patent invalidated and a settlement whose terms were not disclosed plus the remaining two patents under reexamination. Two unresolved patent issues would be enough to scare me off, but then again the ZFS feature set is perhaps too tempting to not take a few risks for.

Next up were espie@'s back to back talks about the *amazing* work he's been doing with OpenBSD packages. Before lunch, "the long road to pkg_add -u ... and beyond" which took us through the background and the design choices and evolution that took us to the point where upgrading your packages with pkg_add -u can be reliably expected to work. All I can say is that it was an excellent presentation about top-notch work that has made the life of OpenBSD users everywhere a lot better. The after lunch part, "efficient distributed package builds in OpenBSD" took us to the slightly more esoteric part of the world that contains the magic that makes sure the binary packages you expect to have at the other end of your pkg_add command actually exist. Another very enlightening talk, and this set of talks was certainly my favorite at this conference.

For further information on the topics mentioned in this column, the EuroBSDCon web site is the natural place to start. The OpenBSD developers' presentations will appear in the papers section of the OpenBSD web site, while my slides, as usual are available from the NUUG site.

Update 2015-04-02: This article refers to the second edition of The Book of PF. The third edition of that title (linked in the previous sentence) became available, with significant updates, in late 2014 and is overall a better resource for learning about networking and PF on all PF-capable systems.