Looking back at the past week, the most interesting communication received here was not about meteorites hitting Russia (although some of the dashcam videos were really exciting), but rather about an old and very simple prediction of mine that appears to have come true.
It came via an email message from a friend and somewhat infrequent correspondent, who wrote on February 15 2013,
"[...] Here's something mildly curious, though. Between now and Dec 26, the current start of my authlog, the below tried to get in via port 2222 on my openbsd box. I'm surprised both that there are not many yet at the same time that there are any at all."And the log excerpt he sent me looked like this:
Jan 11 16:02:01 net5501 sshd[3170]: Invalid user admin from 66.135.32.170
Jan 11 17:58:43 net5501 sshd[2611]: Invalid user aadriano from 66.135.32.170
Feb 10 17:16:33 net5501 sshd[23679]: Invalid user alex from 66.133.188.86
Feb 10 17:18:32 net5501 sshd[20330]: Invalid user cactiuser from 66.133.188.86
Feb 10 17:20:31 net5501 sshd[19432]: Invalid user www-data from 66.133.188.86
Feb 10 17:24:28 net5501 sshd[29455]: Invalid user user from 66.133.188.86
Feb 10 17:26:27 net5501 sshd[82]: Invalid user tomcat from 66.133.188.86
Feb 10 17:28:26 net5501 sshd[31011]: Invalid user test from 66.133.188.86
Feb 10 17:30:25 net5501 sshd[20057]: Invalid user temp from 66.133.188.86
Feb 10 17:32:23 net5501 sshd[19848]: Invalid user service from 66.133.188.86
Feb 10 17:34:22 net5501 sshd[31209]: Invalid user samba from 66.133.188.86
Feb 10 17:38:20 net5501 sshd[12045]: Invalid user postgres from 66.133.188.86
Feb 10 17:40:20 net5501 sshd[1085]: Invalid user oracle from 66.133.188.86
Feb 10 17:42:19 net5501 sshd[12896]: Invalid user nagios from 66.133.188.86
Feb 10 17:44:19 net5501 sshd[10681]: Invalid user mysql from 66.133.188.86
Feb 10 17:46:19 net5501 sshd[27897]: Invalid user guest from 66.133.188.86
Feb 10 17:48:19 net5501 sshd[13400]: Invalid user tom from 66.133.188.86
Feb 10 17:50:18 net5501 sshd[5132]: Invalid user ftp from 66.133.188.86
Feb 10 17:54:15 net5501 sshd[28789]: Invalid user jboss from 66.133.188.86
Feb 10 17:56:14 net5501 sshd[20435]: Invalid user backup from 66.133.188.86
So just a sampling of likely service user IDs on a random system, right?
The reason why this is significant is that my friend, like a few others, tired of the apparently never ending password guessing attempts that were hitting the sshd (also known as the Secure Shell Daemon) listening on the standard port 22, and moved the service to listen on a different port instead.
The immediate effect in almost all cases was a much quieter authentication log, because the password guessers weren't yet trying to bother you on non-standard ports.
Those of us who could for one reason or other not be bothered to change the config to have our sshds run on other ports and using OpenBSD on anything Internet-facing soon found ourselves using the state-tracking features of OpenBSD (introduced in the May 2005 3.7 release). I wrote about it in the PF tutorial, specifically the Turning away the brutes part, and of course a slightly embellished version made it into The Book of PF. Please note the sentence somewhere down the page,
"You might consider moving the service to some other port, but then again, the ones flooding you on port 22 would probably be able to scan their way to port 22222 for a repeat performance."The news is that this is now happening. The only surprise here to my mind is that signs of that exact activity has not been reported earlier. I put that down to mainly two factors. One is that the number of sites running their ssh service only on non-standard ports is probably quite small, another is that people who run services in general pay way too little attention to their logs.
To those of us who run our packet filtering with a default deny policy and drop anything that's not relevant to services we actually run, those attempts to access ports where we do not run anything are simply part of the background noise, so we drop and ignore unless we're looking for patterns in the connection attempts we drop. So we wouldn't necessarily notice the attempts either.
And for all I know (I really can't resist a dig at the high port refugees) it could be that for the most part, those who have moved their main login service to a non-standard port pay less attention to their logs than the rest of us, now that they have obscured the entrance to the kingdom to their own satisfaction if not so many else's.
As I've argued elsewhere, obscuring your login service via non-standard ports or even a requirement to try several ports in sequence really only buys you security equivalent to lengthening your password by two characters per port. After all, TCP and UDP port numbers are limited to 65536 distinct values, within the scope of a 16-bit value. Moving your service to a non-standard port means that the would-be intruder has to guess a two-byte value. Introducing a sequence of ports buys you the added obscurity of two bytes per port.
I have reason to believe that my correspondent here has done away with passwords altogether for logins to his systems, and therein lies the real solution: SSH offers out of the box the opportunity to allow only key based authentication, and with a little extra effort, you can implement more sophisticated authentication schemes. One useful reference for ssh in addition to the manual pages (starting with this one, all of the expected OpenBSD quality) is Michael Lucas' recent book SSH Mastery.
Stay safe, please move from passwords to passphrases, and apply those to your keys only, never to be transmitted in the clear over the net where you never know who could be listening in.
For anyone else seeing this kind of activity, as always I'll be very happy to hear from you, via comments or email.
Good luck and good night.
Thanks to Lars Noodén for valuable input for this column.
Note: A Better Data Source Is Available
Update 2013-06-09: For a faster and more convenient way to download the data referenced here, please see my BSDCan 2013 presentation The Hail Mary Cloud And The Lessons Learned which summarizes this series of articles and provides links to all the data. The links in the presentation point to a copy stored at NUUG's server, which connects to the world through a significantly fatter pipe than BSDly.net has.
Posts
