Sunday, July 1, 2012

Keeping Your OpenBSD System In Trim: A Works For Me Guide

Keeping your OpenBSD systems in trim is dead easy. Occasional reboots are inevitable, but then that's where our famous redundancy features really shine, right? Read on for notes on upgrading your system. (Most of the steps here are relevant for new installs too, but do visit the Install Guide if you're new to OpenBSD.)

My upgrades always start with the same couple of commands:

$ cd ~/upgrade
$ ls -l
$ ncftp eu-openbsd
ncftp ...penBSD/snapshots/amd64 > dir

The first two commands of this sequence show me the date of the latest snapshot I downloaded, and the next two give me the date on what is on the OpenBSD European mirror site.

The ncftp bookmark eu-openbsd in this case expands to

ftp://ftp.eu.openbsd.org/pub/OpenBSD/snapshots/amd64/

which is appropriate for a North European like myself with an AMD64 based system on hand. If you're not in Europe, it's likely you're better of with some other choice of mirror. Check the Getting Releases page on the OpenBSD site for a mirror near you. And of course, do pick the correct architecture for your system.

If the files on the mirror server are newer than the ones I have locally, I'll download them with

ncftp ...penBSD/snapshots/amd64 > get *

If there are no updates, well, that means I'll just check back later.

And of course, if you do not have the ncftp package installed, this command will work on any OpenBSD box with a full base system installed:

$ ftp -ia ftp://ftp.eu.openbsd.org/pub/OpenBSD/snapshots/`uname -m`/{index.txt,*tgz,bsd*,INS*}

(thanks, Pedro Caetano!)

One of the things that makes doing OpenBSD upgrades so amazingly easy is the sysmerge(8) program.  Much inspired by FreeBSD's classic mergemaster(8), this program is, as we shall see by executing the following commands,


$ which sysmerge
/usr/sbin/sysmerge
$ file `which sysmerge`
/usr/sbin/sysmerge: Korn shell script text executable

actually a shell script that makes extensive use of sdiff(1) to highlight the differences between your installed version of a configuration file and the one from the source tree or your install sets, and offers to merge (hence the name) your current customizations into the newer version files and install them on the spot if you like.

Do take a peek with at what the script does:

$ less /usr/sbin/sysmerge

possibly complemented by looking up the sdiff(1) man page if you want. But more about that later.

If you run regular upgrades like I tend to, with snapshots only days or at most a few weeks apart, the differences sysmerge(8) detects are likely few and trivial (but a word of caution: it usually pays to check the Following -current page from time to time).

If you do longer jumps, such as between releases, you will almost certainly find more differences, and in rare cases the changes in the configuration file baselines will be large enough that you will need to take some time out to edit the files by hand. In those cases, studying the upgrade notes relevant to your versions (and all intermediate ones if you do a long jump -- go to http://www.openbsd.org/faq/ and choose the Upgrade Guide link at the top of the left column) will be well worth your time.

The next step is to copy the fresh bsd.rd to the root directory of my boot disk:

$ doas cp bsd.rd /

With the bsd.rd in place, the next step is to reboot your system. Either choose your window manager's reboot option or simply from a shell prompt type

$ doas reboot

When the boot> prompt appears, type b bsd.rd and press Enter.

The familiar bsd.rd boot messages appear (much like the ones illustrated in my earlier installation piece, The Goodness Of Men And Machinery), and the first question you need to answer is:


(I)nstall, (U)pgrade or (S)hell?

Since we're upgrading, we type u (upper or lower case doesn't matter) and press Enter.

The next question up is the choice of keyboard layout:


Choose your keyboard layout ('?' or 'L' for list)

Here I type no for my Norwegian keyboard layout, if you want a different one you can get a list of available choices by typing L instead, and pressing Enter.

Once you've chosen your keyboard layout, the upgrade script prompts you to specify where your root file system is located. The upgrader's guess is only very rarely wrong, so more likely than not you'll just press Enter here.

The upgrader performs a full file system check on the designated root file system, and proceeds to configure any network interfaces it finds configuration files for in the designated root file systems etc directory. You will see the output of DHCP negotiations or similar.

When the network configuration is done, the upgrader prompts you to decide whether you want to perform a full file system check on the other partitions it finds.  The default choice here is no, so if you press enter here, the upgrade script simply runs a fsck -p on each of the file system listed in the system's fstab. You'll see the output for each one of these relatively lightweight checks.

Next, the upgrade script asks where to find the install sets:

Location of sets? (disk http or 'done') [disk]

Here, since I already downloaded the install set files to a local directory, the natural choice is disk. The upgrade script has already mounted all partitions from the system's fstab under /mnt/, so the files I downloaded to /home/peter/upgrade are now available under /mnt/home/peter/upgrade, and that's what I type in response to the prompt for where the sets are to be found.

Unless you have prepared a site.tgz for your site there is normally no reason to add or subtract sets from the default list, so pressing Enter will do nicely. The sets are copied and extracted, and when the extraction is done, you confirm that the upgrade is [done], and when the # shell prompt appears, type

# reboot

to let the system boot into the upgraded operating system.

Watch the system boot, and if you look closely, you will notice that on first boot the updated sysmerge(8) program runs and does the obvious things that do not require manual intervention, and if there are non-obvious things left, a message and an email to root alerts you of the need to do a manual sysmerge(8) run.

I tend to do a sysmerge run anyway after upgrade, if only to see it complete silently:

$ doas sysmerge

That's it! There is a chance that there have been updates to packages you have installed (such as ncftp in my case), so I tend to do a package upgrade pretty soon after rebooting into a freshly upgraded system. The basic update command is pkg_add -u, but I tend to want more verbose output and prompts and generally choose this variant:

$ doas pkg_add -vuV

This command will most likely just work, fetching packages from the site and directory specified in /etc/installurl which is automagically created and maintained by the install program. If the installer somehow guessed wrong, you need to adjust the contents of that file and restart pkg_add (fiddling with the environment variable PKG_PATH as was the procedure in previous releases will not do any good). During the relatively short time when snapshots identify as the release but actual -release packages are not yet available (as in the recent 6.1 release cycle) you may have to adjust the pkg_add command to include -D snap, like this:

$ doas pkg_add -D snap -vuV

For upgrading from one snapshot to the next, there is really not much more to the process. You will occasionally want to run

$ doas pkg_delete -a

to remove packages that were installed as dependencies (mainly libraries) but are no longer needed. In addition, you may want to install and run sysclean to help you indentify other files such as obsolete configuration files that are no longer needed in your current configuration.

If you're upgrading from one release to the next, it makes sense to check the Errata page for any patches you need to apply to the release you're running.

An if you've read this far and found this interesting or useful, please head over to the OpenBSD Store Orders page and buy a few items such as CD sets (up to 6.0 only, sorry), T-shirts, posters and books, or follow the instructions on the OpenBSD site's own Donations page to make a donation to the project.

Update 2017-04-17: Minor updates to adjust to the post-6.1 world, such as no more CDs. Thanks to Marc Espie for (as always) valuable input.

15 comments:

  1. Good article, do packages get security updates within the same release?

    ReplyDelete
  2. A small typo, it's

    sudo pkg_add -vui

    ReplyDelete
    Replies
    1. Fixed, thanks!

      I can't imagine how I managed to make that error, but I guess it happens every now and then.

      Delete
  3. Thank you for this nice article!
    Please let me drop my suggestion to use base ftp to download the necessary files instead of ncftp:
    ftp -ia ftp://ftp.openbsd.org/pub/OpenBSD/snapshots/amd64/{index.txt,*tgz,bsd*,INS*}

    ReplyDelete
    Replies
    1. Thanks! I amended the text slightly (and made the one-liner even more portable).

      Delete
  4. I agree with Pedro. I didn't have ncftp installed, and your howto article would be more fullproof if it could be run on any OpenBSD installation by writing it to use ftp instead of ncftp. Either way, I got the gist, and thank you for this article! :)

    ReplyDelete
  5. Why do you bother starting the process with sysmerge if your going to run it again after installing the install sets?

    ReplyDelete
    Replies
    1. Running your installed sysmerge before running the upgrade will catch any differences between the *etc.tgz sets and the installed files, so your configuration at fresh reboot will most likely be good.

      However, there's a chance (how big depends on how far you are jumping) that the new sysmerge will include information that the older version did not have. For most snapshot-to-snapshot jumps, the second sysmerge run doesn't make any changes.

      Delete
  6. So if I understand it correctly, the main benefit of running sysmerge before and after the update is one less reboot (assuming the second sysmerge didn't make any significant changes.)

    ReplyDelete
  7. Peter,
    This was great. It's a little different with OpenBSD 5.5, but as usual the man pages are helpful. Your article helped me "connect the dots." Thanks for pointing out ncftp.

    ReplyDelete
  8. Perhaps add an instruction to check http://www.openbsd.org/faq/current.html before doing the upgrade ?

    ReplyDelete
  9. Do you still roll your updates this way, or have things changed with more recent versions?
    Thanks,
    -Ben

    ReplyDelete
    Replies
    1. The basic steps are the same, but I notice I should do the sudo -> doas thing and update a few other details.

      Watch this space!

      Delete
  10. My update procedure for snapshots is similarly simple: I download/verify bsd.rd, boot from it, perform a network install, and on reboot run “sysmerge && pkg_add -u”.

    Here’s the script I use to fetch and verify bsd.rd:

    #!/bin/sh

    cd $(mktemp -d)
    if [ -f /etc/installurl ]; then
    < /etc/installurl read mirror
    else
    mirror=https://ftp3.usa.openbsd.org/pub/OpenBSD
    fi
    ftp "$mirror"/snapshots/$(uname -m)/{bsd.rd,SHA256{.sig,}}
    key1=$(ls /etc/signify/openbsd-??-base.pub | tail -2 | head -1)
    key2=$(ls /etc/signify/openbsd-??-base.pub | tail -2 | tail -1)
    (signify -C -p $key1 -x SHA256.sig bsd.rd ||
    signify -C -p $key2 -x SHA256.sig bsd.rd) || exit 1
    mv bsd.rd /

    ReplyDelete

Note: Comments are moderated. On-topic messages will be liberated from the holding queue at semi-random (hopefully short) intervals.

I invite comment on all aspects of the material I publish and I read all submitted comments. I occasionally respond in comments, but please do not assume that your comment will compel me to produce a public or immediate response.

Please note that comments consisting of only a single word or only a URL with no indication why that link is useful in the context will be immediately recycled so those poor electrons get another shot at a meaningful existence.

If your suggestions are useful enough to make me write on a specific topic, I will do my best to give credit where credit is due.