Whenever you write about security in general and SSH security in particular (and for good measure also get slashdotted for your efforts), the comments inevitably turn up a variety of useful and less useful suggestions.
The April 6th, 2012 article about the possible preparations for a new round of slow bruteforcers was no different. Some of these recurring ideas are worthy of some more attention, and a more or less complete list is as follows -
- Set up iptables with the recent module - a clear indication that the commenter did not, in fact read the article. That module implements a subset of the state tracking techniques from my PF rule set referenced in the article, but unlike the PF rules it is specific to the SSH protocol rather than a general technique. And inevitably, the actual configuration the poster presents would in fact not trigger on the kind of traffic we see from the Hail Mary Cloud with its signature slow brute force attempts. Misunderstandings of this kind keep cropping up, for no good reason.
- Disable root logins over the network. Yes, this is obviously useful, and if I remember correctly, PermitRootLogin no is the default in recent OpenSSH versions. Do remember to check what the setting is on your kit, obviously.
- Use packet filtering or other means to restrict where users can log in from. Once again, a very useful suggestion, but some sites may require shell access from semi-random locations, so practical issues (aka real life) can interfere with this one.
- Disable password logins entirely, and allow only key-based logins. Another a very useful suggestion, and one that is useful for several reasons. You do get the downside of managing your keys securely, but key-only logins are generally recommended practice and should be encouraged.
- Set up your sshd to listen on a non-standard port. Several sites report less log file noise after moving their sshd to a non-standard port. Easy to implement and possibly useful, but if a prospective attacker takes the time to do a real port scan for active services, they will find your sshd running on port 22222 fairly easily. I'd say moving your sshd to listen on a non-standard port postpones rather than solves the problem.
- Use denyhosts to automatically add hosts with too many failed logins to a blacklist. Yes, an excellent idea in principle, but it comes with one caveat: Creating a blacklist automatically leaves you in charge of maintaining its content. Blacklist maintenance -- as in keeping your lists up to date and useful, is a complex enough issue that I'm fairly confident it is worth a separate column, and I most likely will return to the issue later. (Update: I did return to the subject, in a 2013 column called Maintaining A Publicly Available Blacklist - Mechanisms And Principles)
- Why not set up port knocking? Well, as you may have guessed I'm about to tell you about a few whys not. I've been circling port knocking as an article subject for a long time, and now it seems that the time has come to outline why I think that port knocking emphatically not a good idea. Read below for my take on the subject.
I must admit that the port knocking concept fascinated me at first, but all implementations had the downside of adding yet another piece of clearly experimental software to my system along with somewhat convoluted procedures for setting the thing up, so I left it for later.
At each of the later dates when i decided to look into the topic again, I discovered new reasons not to like the method (read this thread on OpenBSD-misc for a particularly disenchanting example -- how's single-packet unconditional remote reboot or remote kill your sshd for a feature?). Finally, it was the appearance of the slow bruteforcers (also known as the Hail Mary Cloud) that made me finally decide that port knocking is neither particularly elegant or actually useful.
One of the reasons the port knocking concept seems so deceptively attractive is probably that it is fairly easy to explain (see the Port Knocking article on Wikipedia for a fairly thorough treatment). The target system runs with no services accessible to the outside, and in order to gain access a remote system must send a specific, pre-determined sequence of packets aimed at specified ports. Unless you know (or are able to guess) the correct sequence of ports, you will not be able to gain any access to the target system.
And after this explanation, the naive listener goes
"Oh, how clever, we send packets to random ports!
Nobody will ever be able to guess our combination!".
But first impressions are not always correct. First, please take one step back and explain to me what problem this is supposed to solve. If the answer is the old canard "all the insecurities of that horrible SSH protocol", I will ask you to please point out to me just what those are, and if you will, describe which parts of what I am about to describe actually adds value (that's "security" to you) in a real life scenario.
So let's go straight to the the core of the matter, and consider what actual information content an attacker would need to get right in order to gain access to the system. The hopeful would need to know or guess a sequence of TCP or UDP ports. In both cases, the range of possible ports is a 16 bit number, with a total of 65536 possible values.
Each value is a 16-bit number, with a size of two bytes, or equal to two ASCII characters or one Unicode character. Port knocking examples generally do not run to more than three packets, which means that the minimum amount of information a prospective attacker would need to get right in order to gain access is six bytes, equal to six ASCII characters or three Unicode characters.
Seen from this angle, all port knocking gets you is a somewhat cumbersome method for encoding your unicode password. In most implementations, that password would even be common to all users, with no easy way to change it.
And of course, in almost all other contexts where passwords are used, most sites dictate that you choose a personal password that is longer than six bytes. So that's at least two failures rolled into one: a password that's common to several or all users, and one that is hard or impossible to change, possibly even short enough to fail even basic guidelines.
The amount of information an attacker would need to get right, measured in number of bits or bytes is a fairly useful measure. I admit it's a fairly crude measure, but at least real data are easy to obtain, and since we are talking here about designing and maintaining systems, not writing them, how much cryptographic fun goes into generating the necessary data is not really relevant to the context. Proper cryptography is essential to maintain confidentiality and ensure the integrity of the communications, but we'll leave the details of the crypto bits for a later time.
One other factor that speaks against most implementations of port knocking is that the system runs with no ports open, and a daemon that parses firewall logs for anticipated sequences of port numbers contacted as the sole authority to determine whether access will be granted. We all know that all non-trivial software will contain bugs, so what are the chances that even a fairly simple deamon will at some point in the future be confronted with a situation that makes it terminate, making your system inaccessible (or as we say around here, "bricking your system")?
And one other thing, how do you spot an attacker? If you're looking for access attempts to closed ports, as you should be if you run a port knocking setup, how do you spot an attacker in the primal soup of noise that will anyway be arriving at your public interfaces? Do you, for example, record all sequences that could possibly be seen as unsuccessful attempts and put the source addresses in a "definitely banned" list?
I have not seen any useful answer to that one either. Doing a sudo tcpdump -nettti pflog0 action drop (that is, using tcpdump(8) to extract information on blocked traffic from my packet filtering setup, directly from the log device) on my gateway here certainly shows me access attempts to ports I didn't know much about -- what legitimate purpose, for example, is UDP port 287 actually used for? Once again it all boils down to the fact that if you rely on port knocking, you're really only implementing a cumbersome variant encoding of relatively short passwords.
On a side note (thanks to Adrian Close for reminding me), anybody who sits in the signal path between your knock target and somebody executing the correct knock sequence will be able to record the connections (using, for example, a netflow based tool -- expect to see Netflow-themed columns here in the future), making your closely guarded secret roughly equivalent to a plaintext password much like we used to see in old-style ftp setups.
Next, let us let us instead look at the thing port knocking is trying to protect, namely the sshd service. Typical SSH connection setup involves the transfer of at least the public host keys (typically in the some hundred bytes to several kilobytes range), followed by the user authentication which can involve passwords, keys or a number of cryptography based toys^H^Hols, in any case producing further transfer and validation of anything up to several more kilobytes of data (and in some setups, use of out-of-band methods like short lived keys generated by special-purpose devices) before any meaningful access is possible. The exchange and validation of host keys in itself involves more information that the attacker would have to get right in order to gain access than is involved in any port knocking setup I've seen demonstrated. And remember, host keys exchange is only one early steps of several on the way to authentication.
For the actual math on how data sizes and entropy is significant (yes, in some contexts, size does matter), see the Wikipedia entry and a somewhat shorter summary by Phil Ratcliffe.
All port knocking implementations share the problems I've outlined, and their proponents have tended to ignore or gloss over the issues rather than address them. It is an unfortunate and perhaps embarrasing fact that port knocking in practice comes down to implementing and maintaining a separate set of passwords, and in all likelihood you will be using tools that are less appropriate for the task than the ones that already come with the base system on any reasonable Unix-like system.
In an environment where we know there are large, distributed efforts underway to crack easily guessable passwords, it is naive not to assume that the same password guessing tools could be adapted to try sequences of TCP or UDP ports instead of character sequences. If there is in fact any significant number of systems that use port knocking today, it would surprise me if this isn't already happening. The way I see it, the belief that port knocking in fact offers an effective protection against attackers is a dangerous misunderstanding and probably only serves to divert your attention away from the real problems involved in keeping your systems running in a secure fashion.
If you're still not convinced, I'll show you a better way
Now, if you really, really want to implement port knocking anyway, I'll let you in on a dirty little secret: you don't need a write a new daemon or install anything besides what's already in the OpenBSD base system (or for that matter, on other PF-equipped BSD variants).
You can implement port knocking in a relatively straightforward manner via minor contortions in your PF rule set, and I know for a fact that people have done just that. The examples I was thinking of before I started writing this piece appear to have been removed from the public eye, but with a little effort using the obvious keywords, some well intended but actually quite bad advice on how to put together a seemingly elegant implementation will be on its way to you.
Be aware that you will be doing things the tools were not designed for, and more likely than not you will soon find yourself in a mess of rinky-dink workarounds that keep fighting back.
If, however, what you really want to do is create a separate line of defence with its own round of authentication required for access, your OpenBSD base system already contains a suitable tool in authpf(8). It's fairly easy to come up with a setup that lets your users log in to the authenticating gateway, using any authentication method sshd(8) supports.
You can even run your inital sshd(8) on a separate, non-standard port if you like, and a successful login at that point will only result in a set of packet filtering rules (possibly tailored to that specific user) being loaded. When the ssh session terminates, the rules are unloaded, and the system reverts to the state it had before the user authenticated.
The tools are as flexible and robust as you have come to expect from the OpenBSD system tools (even if you're not running OpenBSD at the moment, you are by a large margin more likely than not using OpenSSH as your ssh client and server, which is supplied to the world by the OpenBSD project), and you can tailor your access profiles and services to your heart's content so they fit your needs.
If you've read this far, you've probably also come to the conclusion that port knocking is really not the way to go. Other, more appropriate tools exist. But how to go about improving the situation?
My suggestions follow below, with some specific suggestions that may have a personal slant.
A Path To Self Improvement
As with any problem that is really rooted in a lack of information or knowledge, educating yourself is the key. Read up on relevant subjects, go back and determine what problem you are actually trying to solve, and then perhaps shop around for tools that will help you solve that problem.
My suggestion is to go to first to www.openbsd.org and browse the resources available there, including the online manual pages and the extensive FAQ document that actually serves quite well as a user manual.
The next logical step is to go to the Orders page and order at least one copy of the latest release (and if you're reading this not too long after publication, do pre-order the upcoming release as well). If you're too impatient to wait for the postal services' mules to load up and deliver, you could download the release (or a -current snapshot if you're feeling adventurous) from a suitable mirror site and start installing right away.
If you don't order any merchandise, you really should head over to the donations page and donate at least an amount equal to the cost of the CDs and other stuff you would otherwise have ordered. There are several fit and frightening ghosts primed and ready to come haunt you if you don't.
If you want some literature to back up the official OpenBSD documentation, you could do worse than order The Book of PF, 3rd edition and Michael W. Lucas' 2012 book SSH Mastery. Using those links here will ensure the OpenBSD project gets a larger-than-otherwise cut of the money as net profit. There are also electronic versions available for both titles (see The Book of PF home page or the SSH Mastery home page for details). If you're really short of funds, even the free, online PF tutorial that the Book of PF evolved from will get you started.
And finally, there are few places better to collect useful information about OpenBSD and its sister operating systems than the various local and regional user groups and conferences such as the upcoming BSDCan in Ottawa, Canada May 9th through 12th, 2012 or EuroBSDCon, with this year's version to be held in Warsaw, Poland, October 18th through 21st.
These events attract both regular users (some of them quite proficient) as well as a useful subset of the people who write and maintain the tools.
Whatever you end up doing, buying or downloading, please keep in mind that it's your responsibility to keep your systems in the best possible shape, and make sure you monitor them properly, via log files or other means. Also see the Pledge of the Network Admin for my (slightly humorous) take on that responsibility. Along with the better parts of the tutorial, it made its way into the book, too.
Thanks to Michael Dexter, Kristaps Dzonsons, Michael W. Lucas, Thordur Bjornsson and Henning Brauer for insightful comments and valuable input.
Copyright © 2012 Peter N. M. Hansteen
If you found this article inspiring, irritating, useful or otherwise moving, please let me know via the comments field just below here, or if you like, drop me a line at peter at bsdly dot eu.
Note: A Better Data Source Is Available
Update 2013-06-09: For a faster and more convenient way to download the data referenced here, please see my BSDCan 2013 presentation (also in article form as The Hail Mary Cloud And The Lessons Learned) which summarizes this series of articles and provides links to all the data. The links in the presentation and the article point to a copy stored at NUUG's server, which connects to the world through a significantly fatter pipe than BSDly.net has.
Posts
