A few weeks before I left Datadok, some real user accounts there, including mine, were joejobbed. That is, somebody, somewhere started sending spam with the From: or return address set to a real, live mail account in one of our domains. There were several incidents, the backscatter output of one of the more recent ones is preserved here.
Of course there were a handful of the "please prove to me that you are a human" messages from challenge-response systems too, but more about that later.
I normally keep the composed, sceptical attitude you will have come to expect from system administrators, but at fairly random intervals I turn into a regular Goody Two-Shoes. So faced with a collection of bounces for messages I certainly did not send, I decided to take some time to contact the people responsible for the machines that sent the spam. Owners of spam sending machines (as distinct from their Pwners) generally are not aware what the machines are doing, and at times I turn into that soft-hearted guy who just wants to help.
As you will see from the collection, not all bounces contain enough of the original message to track down where the spam was actually sent from. Then there were others where it was possible to dechipher where the message came from, and I made a canned message and sent it off to whatever addresses seemed likely from the whois output.
You should not be surprised to hear that quite a handful of those produced regular bounces (yes, postmaster@ and several other RFC2142 mandated mailboxes are non-existent at several sites), but what struck me was that of those that did not bounce there were several that instead produced "Hi, I'm the challenge/response robot at site.nx, please follow my instructions to prove you are human" responses.
Seriously, folks, does anybody in this day and age actually believe that these systems work? It's possible that challenge-response adherents' claims that their system stops all incoming spam are true, but seen from the outside, the only guaranteed effect is that your site will produce backscatter, and lots of it. It would perhaps have been interesting to know just how many of the undeliverables I've squirted into my spammer bait list were actually produced by challenge-response systems, but unfortunately, we have no way of knowing.
There are a few other things that challenge-response systems do not protect you against, such as plain old stupidity and malicious clicking by those who receive your challenge-response backscatter. I take the spam bounces in my collection as proof of that.
So challenge-response people, yes, I agree that you had no way of knowing whether Hampe Ivancevic, Hamada Kirkegaard, Caixia Kluge, Bitte Kutschinski, Anant Johaneson, Sumol Przygocki or a few others actually worked for me at the time you sent your challenge. I will however promise those fine people that if they exist, if they have the right qualifications and I am in a position to offer them employment, I will consider their applications in all seriousness.
The rant about certain blacklists and certain unwise ways to use them will have to wait until later. There are other deadlines to keep, the house still needs the odd splash of paint, but I'll catch up with you all later.
Sunday, May 25, 2008
Wednesday, May 21, 2008
Fake Address Round Trip Time: 13 days
The results are in. Our adversaries really are mindless automata.
Regular readers will have noticed that I've been running a small scale experiment over the last few months, feeding one spammer byproduct back to them via a reasonably accessible web page. The hope was that I would learn a few things about spammer behavior in the process.
After collecting fake addresses in my domains generated elsewhere for a while, I started noticing that a short time after I'd put addresses on the traplist page, they would start appearing as To: addresses on what appeared to be spam message entries in my logs. After a while more, I was certain that the round trip time was down to a few days, but my notes did not include exact dates for when each individual harvested address made it into my spamtrap list. Meaning, of course, I had no way of telling just how fast the process is.
Time to cheat slightly, or, as they say in the trade, perform a controlled experiment. Instead of sticking to the original plan of strictly collecting addresses generated elsewhere and feed them back to the harvesters via the web page, I decided to deviate slightly and plant an address at a specified date, and maybe add another few for data points later. I did the obvious thing and on October 11th, 2007, I slipped put-here-2007-10-11@datadok.no into one of that day's batches of collected addresses.
Needless to say, my attention wandered from the project in the meantime. After all, in October there was still that book to finish, and after the book was done, other developments had my attention or at least drained my energy for quite a while. So it was only last Sunday it struck me that by now I should have at least some data on what actually happened. One other reason it was suddenly quite appropriate to sum up the data was that the datadok.no domain had been turned over to new owners, and it will likely move out of my sphere of responsibility in the near future.
So here's the result, the fake address round trip time:
$ grep put-here-2007-10-11@datadok.no /var/log/spamd
Oct 24 03:40:40 skapet spamd[20795]: (BLACK) 60.50.174.129:
<pepgyoygq@boisdelan.com> -> <put-here-2007-10-11@datadok.no>
That is, the first time my artificially inserted address was used as a spam target was thirteen days after I put it on the traplist page. Since then, something, somewhere, has tried
$ grep -c put-here-2007-10-11@datadok.no /var/log/spamd
300
to deliver email to our imaginary friend a total of 300 times. Data taken from the spamd in front of that domain's secondary mail exchanger, of course. As always, I would love to hear from you about any related experiences.
In upcoming columns we will see, er, actually I find myself with such a selection of tempting topics to choose from, it is really hard to decide what to cover next. But the next one will appear here shortly.
Update 2015-08-01: In a totally unrelated article, posted on the afternoon of July 24th, 2015, the string razz@skapet.bsdly.net appeared. it took only three days, two hours and forty-eight minutes (approximately) before my spamd(8) logged this attempt at delivering mail to that address:
Thanks to the script that also prepares the downloadable list of trapped IP addresses, I was alerted of this happening, and the address was duly added to the spamtraps list and the accompanying web page as part of the batch that took the list to its current count of 29135 entries.
From this accidental anecdotal evidence, we can conclude that the time from when random string containing an at sign appears on a web site to the time it's used as spam target has now shrunk to about three days.
The attempts seem to be a little less energetic this time though: greping the relevant logs turns up only 27 attempts at delivery. It's possible this is down to the fact that there are now so many more imaginary friends to choose from in that long list.
Note: The
Regular readers will have noticed that I've been running a small scale experiment over the last few months, feeding one spammer byproduct back to them via a reasonably accessible web page. The hope was that I would learn a few things about spammer behavior in the process.
After collecting fake addresses in my domains generated elsewhere for a while, I started noticing that a short time after I'd put addresses on the traplist page, they would start appearing as To: addresses on what appeared to be spam message entries in my logs. After a while more, I was certain that the round trip time was down to a few days, but my notes did not include exact dates for when each individual harvested address made it into my spamtrap list. Meaning, of course, I had no way of telling just how fast the process is.
Time to cheat slightly, or, as they say in the trade, perform a controlled experiment. Instead of sticking to the original plan of strictly collecting addresses generated elsewhere and feed them back to the harvesters via the web page, I decided to deviate slightly and plant an address at a specified date, and maybe add another few for data points later. I did the obvious thing and on October 11th, 2007, I slipped put-here-2007-10-11@datadok.no into one of that day's batches of collected addresses.
Needless to say, my attention wandered from the project in the meantime. After all, in October there was still that book to finish, and after the book was done, other developments had my attention or at least drained my energy for quite a while. So it was only last Sunday it struck me that by now I should have at least some data on what actually happened. One other reason it was suddenly quite appropriate to sum up the data was that the datadok.no domain had been turned over to new owners, and it will likely move out of my sphere of responsibility in the near future.
So here's the result, the fake address round trip time:
$ grep put-here-2007-10-11@datadok.no /var/log/spamd
Oct 24 03:40:40 skapet spamd[20795]: (BLACK) 60.50.174.129:
<pepgyoygq@boisdelan.com> -> <put-here-2007-10-11@datadok.no>
That is, the first time my artificially inserted address was used as a spam target was thirteen days after I put it on the traplist page. Since then, something, somewhere, has tried
$ grep -c put-here-2007-10-11@datadok.no /var/log/spamd
300
to deliver email to our imaginary friend a total of 300 times. Data taken from the spamd in front of that domain's secondary mail exchanger, of course. As always, I would love to hear from you about any related experiences.
In upcoming columns we will see, er, actually I find myself with such a selection of tempting topics to choose from, it is really hard to decide what to cover next. But the next one will appear here shortly.
Update 2015-08-01: In a totally unrelated article, posted on the afternoon of July 24th, 2015, the string razz@skapet.bsdly.net appeared. it took only three days, two hours and forty-eight minutes (approximately) before my spamd(8) logged this attempt at delivering mail to that address:
Jul 27 20:16:01 skapet spamd[1520]: (GREY) 183.79.28.71: <esther1jomkoma1ej@yahoo.co.jp> -> <razz@skapet.bsdly.net>
Thanks to the script that also prepares the downloadable list of trapped IP addresses, I was alerted of this happening, and the address was duly added to the spamtraps list and the accompanying web page as part of the batch that took the list to its current count of 29135 entries.
From this accidental anecdotal evidence, we can conclude that the time from when random string containing an at sign appears on a web site to the time it's used as spam target has now shrunk to about three days.
The attempts seem to be a little less energetic this time though: greping the relevant logs turns up only 27 attempts at delivery. It's possible this is down to the fact that there are now so many more imaginary friends to choose from in that long list.
Note: The
@datadok.no addresses were removed from the traplist in late 2008 when that domain was handed over to the care of a different organization.
Wednesday, May 14, 2008
Network devices that lie
Do some network devices lie about their config, or are they just talking past each other?
One of my tasks recently has been to start integrating the networks of two companies, because one of the companies bought the other.
At my end you would have guessed already that anything exposed to the Internet would be running OpenBSD, while at the other end they were strangely attached to products from a certain software marketer based in the north-western part of the USA. But for the IPSEC part, they had decided that a certain proprietary "Internet Security Appliance" would fill their needs, and at each of their locations they had identical appliances running for "VPN" purposes.
Therein lies a story. After a few working days of debugging my way to an almost working configuration, my main conclusion is, that it is quite likely that proprietary security appliance lies. That is, it will let its administrator choose a wide range of very specific configuation options, but will actually override them, at least if the device at the other end does not do the "I'm one of your kind" secret handshake.
Here's what it looks like in the logs on the OpenBSD end, the part that is supposed to be a negotiation but actually isn't and the options on the OpenBSD side matched the options given in the screendumps of the other side's GUI that I had to work with:
and so on, through the various options, each time expecting what matched the screendumps, getting something else entirely, up until the inevitable
or as they say elsewhere, Do not pass GO.
Moving on from there to actual tunnel setup reveals troublesome vendor IDs and other strangeness. This could very well be an attempt at vendor lock-in, but then the conventional wisdom anyway seems to be that the only way to set up a VPN is to have identical devices at every endpoint. There is no good reason why it should be like that, but I may be discovering the less good ones.
The main reason I'm not naming names or stating flat-out that "this device lies about its configuration" is that my analysis is not complete yet. If you have data you want to share with me on this or similar experiences, I would love to hear from you, at peter@bsdly.net. It's likely I'll write a followup based on what I hear and what happens.
During my silent period blogging wise, some notable events did occur, and I may return to them in later posts.
UKUUG Spring 2008
In late March I went to Birmingham to attend the UKUUG Sping 2008 conference - with a PF tutorial much like the one in Riga in February and a refreshed malware talk. A number of very good talks, and a number of Perl Mongers there, notably Barbie, who told me about his "Understanding Malware" talk that actually overlaps quite a bit with my paper. The social parts were good too, and yes, in Birmingham it is possible to find Real Ale.
BSD Magazine #1 is out
Not too long after the UKUUG Spring Conference, the first issue of BSD Magazine was published. Readers of this blog may find one article to be slightly familiar, but there's a nice selection of articles and reviews, covering all the BSD systems and a wide range of topics. And I should mention, they are looking for articles for upcoming issues.
OpenBSD 4.3 is out
OpenBSD 4.3 was released on schedule on May 1st, and as usual those who pre-ordered got their copies early. As always, exciting new stuff as well as a number of incremental improvements. There's interesting stuff happening post-4.3 as well, watch undeadly for updates.
Next up: FOSS Aalborg, June 4th
Our Danish friends are kind enough to give us yet another excuse to come visit. A group of BSD and Linux people have been hosting good conferences in Copenhagen for years, last year's EuroBSDCon being one.
This time it's Aalborg's turn, with a one day conference on June 4th. Yours truly will be on the speaker list. I hope to see you there, I will be bringing a small stack of books.
One of my tasks recently has been to start integrating the networks of two companies, because one of the companies bought the other.
At my end you would have guessed already that anything exposed to the Internet would be running OpenBSD, while at the other end they were strangely attached to products from a certain software marketer based in the north-western part of the USA. But for the IPSEC part, they had decided that a certain proprietary "Internet Security Appliance" would fill their needs, and at each of their locations they had identical appliances running for "VPN" purposes.
Therein lies a story. After a few working days of debugging my way to an almost working configuration, my main conclusion is, that it is quite likely that proprietary security appliance lies. That is, it will let its administrator choose a wide range of very specific configuation options, but will actually override them, at least if the device at the other end does not do the "I'm one of your kind" secret handshake.
Here's what it looks like in the logs on the OpenBSD end, the part that is supposed to be a negotiation but actually isn't and the options on the OpenBSD side matched the options given in the screendumps of the other side's GUI that I had to work with:
May 13 14:00:28 delilah isakmpd[13547]: attribute_unacceptable: ENCRYPTION_ALGORITHM: got DES_CBC, expected 3DES_CBC May 13 14:00:28 delilah isakmpd[13547]: attribute_unacceptable: HASH_ALGORITHM: got MD5, expected SHA
and so on, through the various options, each time expecting what matched the screendumps, getting something else entirely, up until the inevitable
May 13 14:00:28 delilah isakmpd[13547]: message_negotiate_sa: no compatible proposal found
or as they say elsewhere, Do not pass GO.
Moving on from there to actual tunnel setup reveals troublesome vendor IDs and other strangeness. This could very well be an attempt at vendor lock-in, but then the conventional wisdom anyway seems to be that the only way to set up a VPN is to have identical devices at every endpoint. There is no good reason why it should be like that, but I may be discovering the less good ones.
The main reason I'm not naming names or stating flat-out that "this device lies about its configuration" is that my analysis is not complete yet. If you have data you want to share with me on this or similar experiences, I would love to hear from you, at peter@bsdly.net. It's likely I'll write a followup based on what I hear and what happens.
During my silent period blogging wise, some notable events did occur, and I may return to them in later posts.
UKUUG Spring 2008
In late March I went to Birmingham to attend the UKUUG Sping 2008 conference - with a PF tutorial much like the one in Riga in February and a refreshed malware talk. A number of very good talks, and a number of Perl Mongers there, notably Barbie, who told me about his "Understanding Malware" talk that actually overlaps quite a bit with my paper. The social parts were good too, and yes, in Birmingham it is possible to find Real Ale.
BSD Magazine #1 is out
Not too long after the UKUUG Spring Conference, the first issue of BSD Magazine was published. Readers of this blog may find one article to be slightly familiar, but there's a nice selection of articles and reviews, covering all the BSD systems and a wide range of topics. And I should mention, they are looking for articles for upcoming issues.
OpenBSD 4.3 is out
OpenBSD 4.3 was released on schedule on May 1st, and as usual those who pre-ordered got their copies early. As always, exciting new stuff as well as a number of incremental improvements. There's interesting stuff happening post-4.3 as well, watch undeadly for updates.
Next up: FOSS Aalborg, June 4th
Our Danish friends are kind enough to give us yet another excuse to come visit. A group of BSD and Linux people have been hosting good conferences in Copenhagen for years, last year's EuroBSDCon being one.
This time it's Aalborg's turn, with a one day conference on June 4th. Yours truly will be on the speaker list. I hope to see you there, I will be bringing a small stack of books.
Subscribe to:
Posts (Atom)
Posts
