A Lady in Distress; or Then Again, Maybe Not

A two user domain gets bounces for seven hundred, grep and sed to the rescue, spamd saves the day

The past week moved along with only minor disturbances on the keep-systems-running front. The time consuming frustrations were generated elsewhere, and (un?)fortunately I am not at liberty to discuss the details. Incompetence was involved, next week it's somebody else's problem.

All the while, the spammer trapping experiment has been moving along at a leisurely pace.

Generally keeping the lists (both the web version and the live one) updated would cost me a few minutes' browsing of greylist dumps two or three times a day or whenever I felt like it, with a typical catch of maybe fifteen new bogus addresses to feed to the trap list each day.

For the last three or four days the haul has been smaller, with essentially no new captures yesterday, for example. Now I've found out why. They have moved on, alpabetically.

Done with bsdly.net, the dominant group of spammers moved on to generating addresses in the D domains including datadok.no and dataped.no. I'm bound to have missed a few, since the grand total by this morning had yet to reach a full thousand. By now, they seem to have reached the Es. This morning I noticed the overnight greylist dumps were bigger than usual.

The reason: ehtrib.org, the domain we set up mainly for my wife's use (read: her email), appears to be the current home of made up From: addresses, with roughly seven hundred accumulated by the time I was done with morning routines of breakfast with coffee and browsing the overnight incoming mail.

That is by far the largest addition to the flypaper list ever.

Fortunately, with only two active addresses in the domain (I'm not telling what either other one is) it's fairly trivial to extract the bogus ones.

Up to now I've been integrating the noise into the traplist page manually, for now I've put this batch up at http://www.bsdly.net/~peter/ehtrib-1stbatch.

They're all in the active traplist at the gateways, of course. It's the editing into the page the spammers will slurp via unattended robot I'm putting off for a little more while I'm doing some other writing. [not any more. all there now, but the original list is preserved too]

Just why this time we are seeing this number of addresses over a short period of time, and not a handful each day over several months is an open question. One likely explanation is that one of the chickenboners fell asleep at the wheel and let the junk generator run longer than they actually intended. Time will show if this means they move on more quickly.

When I have more time, I will probably analyse the data I am accumulating at the moment and tell the tales of the silly lamer tricks the spammers try to pull.

In the meantime, following up on earlier posts, there are still a few people who Just Don't Get It:

Aug 19 13:28:03 delilah spamd[3712]: 217.159.231.230: connected 
(9/9), lists: spamd-greytrap
Aug 19 13:31:49 delilah spamd[3712]: (BLACK) 217.159.231.230:
<> -> <armrest10@datadok.no>
Aug 19 13:33:32 delilah spamd[3712]: 217.159.231.230: Subject:
Considered UNSOLICITED BULK EMAIL, apparently from you
Aug 19 13:33:32 delilah spamd[3712]: 217.159.231.230: From:
"Content-filter at linux.byroomaailm.ee" 
<postmaster@linux.byroomaailm.ee>
Aug 19 13:33:32 delilah spamd[3712]: 217.159.231.230: To: 
<armrest10@datadok.no>
Aug 19 13:34:38 delilah spamd[3712]: 217.159.231.230: disconnected 
after 395 seconds. lists: spamd-greytrap

And it looks like the published list is having the effect I was hoping for. I keep seeing quite a few of the addresses in ALLCAPS (with numbers tacked on) I put on the web page a few weeks back beginning to appear in lowercase but otherwise identical in my greylist dumps.

In other news, the PF tutorial session at EuroBSDCon is now a definite.

See you in Copenhagen, if not before!

Now for that other bit of writing. The Book of PF page now refers to the tutorial page at bsdly.net. Now let's get that baby done.

The lady is, in fact, not too distressed.

And in case you were wondering - Yes, you can use my auto-generated list of trapped hosts for your own blacklisting purposes if you like. Here it's just a supplement to Bob Beck's traplist, and most likely you're better off using the Beck/UofA list along with your own greytrapping, but if you really want to use mine, be my guest. It gets updated ten past every hour.

BSD is dying, security shot to hell, clamav wins and other tales of depravity and greed

It's been an interesting week, in several ways.

Yesterday's big item was the slashdotted report that BSD is dying, or rather, that some important security related software in among others OpenBSD may, according to a paper by University of Cambridge researcher (and FreeBSD core member) Robert Watson, be vulnerable to a previously unresearched class of vulnerabilities. This time we're talking about a really hard problem which I think hits a lot more than the ones they picked for the tests. Local privilege escalation only, so not the third remote hole in OpenBSD after all. The paper is well worth reading, and if you're a little short of time, the slides will give you the general drift and then some. The sky didn't fall this time either.

Actually totally unrelated, Jason Dixon's BSD is dying talk (see above) is worth a few chuckles. He gave it at BSDCan 2007 too.

Meanwhile, reports say that over at LinuxWorld in San Francisco, they put a ten popular antivirus packages through the paces, and according to this story the free (as in GPL) ClamAV came out on top. Nice to see that the free stuff (which we've been using for years here) is found by independent testing to be as good as we thought it was.

Continuing the "the free stuff is quite good" thread, when I found that I actually needed a Windows machine to do some work from home, I tried getting that Windows laptop to talk to my wireless network at home. Windows didn't recognize the integrated 11b wireless adapter at all, so I dug out the Atheros based DWL-AG650 I'd used with the machine and various BSDs.

No go. Windows did register a new PCCARD inserted, but did not have a usable driver available. The Control Panel showed a generous helping of question marks, with two 'Ethernet Class' devices among them, so it's quite possible that the integrated 11b unit was the other one.

I'm not one who gives up easily, so I went to the D-Link web site for a driver. They did not actually have one on tap (or at least not easily available), since the card is no longer in production, so via the well known search engine starting with G I found something that claimed to be the correct Windows driver. Which installed, but even after a reboot the card management software (why oh why a separate management app for each bit of hardware in your system?) still claimed that no compatible card was present.

A short string of unprintables and 22 minutes later, I had the machine working the-thing-that-needed-windows via Rdesktop on Ubuntu, remote controlling a machine at the office. The moral of the story: If you need Windows, you're better off with Linux and Rdesktop.

Certainly worth a read is the short paper by Sun's Jon Bosak on why Sun voted not to OOXML, at http://www.streamingweb.no/v1-ooxml.pdf.
Well researched and well written, and contains such nuggets as

"On the face of it, this astonishing provision would appear to indicate that the authors of the DIS did not understand the purpose of XML,"

and

"In practice, the effect of radical underspecification is to allow behavioral details to be determined on an ad hoc basis by the dominant software."

This somehow fails to surprise me, it's the story of RTF all over again. I'be been meaning to write about Microsoft vs standards, but in the meantime Jon's paper is well worth reading.

Back to the inevitable spam update (yes, elzapp, I do sometimes blog about something besides spam), the local traplist keeps growing. I sometimes wonder if they've actually looked at what we do here - this morning's batch of fake From: addresses had proofreads49@datadok.no among them.

And accenting one of the points I made in the malware paper that we are making the spammers work ever harder to generally fail to deliver their crap, Bob Beck's traplist keeps growing and has now hit a new all-time high of 125,808 entries.

That number could grow a bit more before they're all done. I do pity those who get billed by unit of data transferred who still don't have a sensible setup in place.

And yes, the book is progressing.

UPDATE 2007-08-14: After a relatively quiet weekend spamwise (Bob Beck's list in the 65,000 to 85,000 range), activity seems to have reached another peak with a total of 141,892 entries trapped at 08:00 CEST this morning. I would have expected to see a corresponding surge in the number of new bogus addresses seen in our greylist, but they did not turn up. We can always hope that this is due to saner spam handling at sites which used to bounce spam back to the From: address.

We see your every move, spammer

My logs tell me that the spamtrap topic is a favorite, and more likely than not somebody who read the announcement will also take a peek at the traplist itself. So while I'm slowly preparing a post about something else entirely (which what I feel is actually a lot more interesting), it can't hurt to fill you in on what I've been doing to keep track of spammer behavior.

It's a quiet life, at least by surface appearances. In between the steady stream of mainly confidential tasks handled at Datadok and the odd request to bsdly.net for services of one kind or the other, I focus on getting the book done, chapter by chapter.

The traplist is slowly expanding. The collection process itself is automated for all the tedious tasks. The "Unknown user" entries from my mail server logs as a source of traplist material almost dried up, so I started looking at the greylists directly.

After sampling my greylists at random intervals for a while, a short shell script now dumps the data to somewhere safe ten past every full hour, notes the number of grey entries and TRAPPED entries, and dumps the TRAPPED IP addresses to a file which is available to the world from the traplist page. The list is comfortably short at most times. I imagine somebody with beefier bandwidth or a more widely known domain would have more hosts trapped at any time.

The file with currently trapped hosts gets overwritten each time the script runs. There is an outside chance that the other generated data might be useful in future research, and storage is cheap these days, so I keep the data around.

Observing the greylists reveal some odd things, like a certain Taiwanese host which tried, on August 1st, 2007, to send roughly a thousand messages to one address in a domain elsewhere, using generated From: addresses at every host name and IP address in our local network. They probably thought they'd found an open relay. Spamd's "250 This is hurting you more than it is hurting me." probably did not register with them as an outright rejection, much like it fools a number of web available open relay detectors.

The conclusions still stand, though. They echo the conclusions from the malware paper (*): the spammers are working harder at sending their trash mainly because we are as close as does not matter to always correctly detecting and dealing with their junk traffic.

I keep wondering if even the few minutes' worth of work a day updating the traplist is worth it, since we are catching essentially all spam anyway. Then at intervals, one or more of the generated, made up addresses from the list actually turns up in my greylist dumps.

(*) Whenever the "The silent network" paper comes up in discussions, it looks like depending on who you are, it's either way too long or too short. At twenty-few pages it's too long for the attention span of the loudmouth self-appointed SMTP experts you may encounter on web forums and mailing lists, and too short (read: not a book) to carry much weight with a decision maker who will not read much more than the executive summary anyway. Making that article morph into a book is on my list of Things To Look Into Later If Time Allows And It Still Makes Sense Then.

If you're still there after reading all this: Click the ads already. Make somebody else pay for your entertainment.

On the business end of a blacklist. Oh the hilarity.

I had planned to write about something else for my next blog entry, but life came back and bit me with another spam related episode. Next time, I promise, I'll do something interesting.

In the meantime, I've discovered that a) very few people actually use SPF to reject mail b) the SPF syntax looks simple, but is hard to get right, and c) there are still blacklists which routinely block whole /24 nets.

This morning I got a message from somebody I met at BSDCan in May, asking me to do something LinkedIn-related. Naturally, since I felt I needed some more details to do what this person wanted, I sent a short email message. That message got rejected,

SMTP error from remote mail server after MAIL FROM: SIZE=2240:
    host mailstore1.secureserver.net [64.202.166.11]:
    554 refused mailfrom because of SPF policy

which means that the SPF record

 datadok.no. IN TXT "v=spf1 ip4:194.54.103.64/26 
                     ip4:194.54.107.16/29 -all"

does not do what you think it does. Mail sent from 194.54.103.66 was not let through.

OK, the checking tool at the OpenSPF site seem to agree with secureserver.net, and I seriously can not blame them for the choice to trust SPF absolutely.

At the moment it seems my listing each host name is what does the trick. Weird. Anyway, next up in my attempt to communicate with my overseas friend, I tried sending a message from bsdly.net instead. That bounced too, but for a slightly different reason:

 SMTP error from remote mail server after RCPT TO::
    host smtp.where.secureserver.net [64.202.166.12]:
    553 Dynamic pool 194.54.107.142. 

If you look up the data for bsdly.net, you will find that valid mail from there gets sent mainly from 194.54.107.19, which is in the tiny /29 our ISP set aside for my home net when I told them I wanted a fixed IP address.

I'm not sure if the rest of the "ip=194.54.107.*" network is actually a pool of dynamically allocated addresses these days, but I do know is that 194.54.107.16/29 has not been dynamically allocated for quite a number of years.

Going to the URL gave me this picture:



This really gives me no useful information at all. Except, of course, that at secureserver.net they think that putting entire /24 nets on their blacklist is useful. Some of us tend to disagree with that notion.

Anyway, I filled in the form with a terse but hopefully polite message, and clicked Submit.

I was rewarded with this message:



If I read this correctly, they think mail from 194.54.107.19 is spam because BGNett or MTULink have not set up reverse lookup for 194.54.107.142. OR because they think the entire /24 is dynamically allocated. OR somebody in that subnet may have sent spam at one time. I can only guess at the real reason, and repeat over and over that blocking entire subnets will give you a generous helping of false positives.

Nevermind that, the SPF record which made my mail from datadok.no go through to my overseas friend included a:hostname.domain.tld for all allowed senders.

And in other news, the PF tutorial saw its visitor number 15000 since EuroBSDCon 2006 on Saturday, last count is 15220.