Sunday, October 23, 2011

You're Doing It Wrong, Or, The Return Of The Son Of The Hail Mary Cloud

Do Linux system administrators still in this day and age run with PermitRootLogins yes in their sshd configurations? Do they also allow password logins? Do they ever attempt to keep their systems up to date and reasonably secure?

Apparently the answers are yes, yes, and no, at least for some. The evidence is slowly accumulating in the authentication logs on one of my servers, published via the The Hail Mary Cloud Data Page. There are several reasons why these attempts stand out, but it kind of helps that the number of users with sensible or indeed legitimate reasons for shell access to this particular server is quite limited.

I've ranted about this before, famously but not exclusively in a series of slashdotted and much-syndicated blog posts such as this one. For the TL;DR crowd, here's the summary:

If you're allowing root logins from the great elsewhere, you're doing it wrong.

If you've been allowing root logins from the great elsewhere, I wouldn't be surprised it's one or more of your boxes doing the distributed password guessing.

If you can't remember the last time you checked that your system is up to date and properly configured, you're doing it wrong.

So nothing really new to see here, it's only yours truly seeing his hope of never seeing this silliness repeated dashed, again.

If you're interested in background information about the Hail Mary Cloud phenomenon, please do read the previous posts (A low intensity, distributed bruteforce attempt (December 2, 2008), A Small Update About The Slow Brutes (December 6, 2008), Into a new year, slowly pounding the gates (December 21, 2008), The slow brutes, a final roundup (January 22, 2009) and The slow brute zombies are back (April 12, 2009) as well as the one referenced earlier.

Good night and good luck.

Update 2011-10-27: The alphabetic stage has started, see refreshed data for details.

Note: A Better Data Source Is Available
Update 2013-06-09: For a faster and more convenient way to download the data referenced here, please see my BSDCan 2013 presentation The Hail Mary Cloud And The Lessons Learned which summarizes this series of articles and provides links to all the data. The links in the presentation point to a copy stored at NUUG's server, which connects to the world through a significantly fatter pipe than BSDly.net has.

3 comments:

  1. Hmmm ... by any chance, is PermitRootLogin turned on by default in the standard ssh distributions? I just noticed it being turned on at my old ubuntu laptop ... I never turned it on. Then again, I think ubuntu by default sets no root password ... if one needs root, one is supposed to sudo.

    Professional sysadmins may be blamed for "doing something wrong", but the regular Ubuntu user, only using the web browser and hardly anything other, cannot really be blamed for "doing something wrong". I think it's really very wrong if the default configuration allows root login ...

    ReplyDelete
  2. My opinion is that all sshd's should have password logins turned off, not necessarily root logins turned off. Hopefully ssh worms can't brute force a pubkey login. :)

    ReplyDelete
  3. This funny script will keep those dicks out without making too much hassle for the occasional guy who forgot their password.

    Pipe your log through it, set up PF to block the corresponding list and purge the list of IP's at your preferrable time. I use 3 days.

    #!/usr/bin/perl

    use strict;
    no strict 'subs' ;
    my ($c,$ip,$domain);

    use Sys::Syslog qw( :DEFAULT setlogsock);
    use Net::Nslookup;

    setlogsock('unix');
    openlog("bruteforce",'pid',LOG_AUTH);

    open (STDIN);
    while () {


    if ($_ =~ /ftpd.*FTP LOGIN FAILED FROM (.*\.[a-z]{2,4}),[a-z0-9]*/) {
    $domain = $1;
    $c = `/usr/bin/tail -n 30 /var/log/auth.log |grep -c $domain`;
    if($c > 5) {
    $ip = nslookup($domain);
    system "/sbin/pfctl -t ssh_abuse -T add $ip";
    syslog('info', "Added $ip to blocklist from ftpd");
    }
    }
    if ($_ =~ /ftpd.*FTP LOGIN FAILED FROM ([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)/) {
    $ip = $1;
    $c = `/usr/bin/tail /var/log/auth.log |grep -c $ip`;
    if($c > 5) {
    system "/sbin/pfctl -t ssh_abuse -T add $ip";
    syslog('info', "Added $ip to blocklist from ftpd");
    }
    }
    if ($_ =~ /error: PAM: authentication error for illegal user.*from ([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)/) {
    system "/sbin/pfctl -t ssh_abuse -T add $1";
    syslog('info', "Added $ip to blocklist from sshd");
    }
    if ($_ =~ /sshd.*PAM: authentication error for .* from ([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)/) {
    $ip = $1;
    $c = `/bin/cat /var/log/auth.log |grep -c $ip`;
    if($c > 3) {
    system "/sbin/pfctl -t ssh_abuse -T add $ip";
    syslog('info', "Added $ip to blocklist from sshd");
    }
    }
    if ($_ =~ /sshd.*PAM: authentication error for .* from (.*\.[a-z]{2,4})/) {
    $domain = $1;
    $c = `/bin/cat /var/log/auth.log |grep -c $domain`;
    if($c > 3) {
    $ip = nslookup($domain);
    system "/sbin/pfctl -t ssh_abuse -T add $ip";
    syslog('info', "Added $ip ($domain) to blocklist from sshd");
    }
    }
    if ($_ =~ /ipop3d.*Login failed.*\[([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)\]/) {
    $ip = $1;
    $c = `/usr/bin/tail /var/log/auth.log |grep -c $ip`;
    if($c > 5) {
    system "/sbin/pfctl -t ssh_abuse -T add $ip";
    syslog('info', "Added $ip to blocklist from ipop3d");
    }
    }
    if ($_ =~ /ipop3d.*Login excessive login failures.*host=\[([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)\]/) {
    $ip = $1;
    syslog('info', "Added $ip to blocklist from ipop3d");
    system "/sbin/pfctl -t ssh_abuse -T add $ip";
    }
    }
    closelog;
    close (STDIN);

    ReplyDelete

Note: Comments are moderated. On-topic messages will be liberated from the holding queue at semi-random (hopefully short) intervals.

I invite comment on all aspects of the material I publish and I read all submitted comments. I occasionally respond in comments, but please do not assume that your comment will compel me to produce a public or immediate response.

Please note that comments consisting of only a single word or only a URL with no indication why that link is useful in the context will be immediately recycled so those poor electrons get another shot at a meaningful existence.

If your suggestions are useful enough to make me write on a specific topic, I will do my best to give credit where credit is due.