A nice surprise may be in line for a new visitor, and you (yes, you) can help me pick the surprise.
In late 2004, I started working on the text for a user group lecture for the BLUG meeting scheduled for the the following January.
The original manuscript was in Norwegian, but after a rather successful and surprisingly well attended user group meeting, I wrote up an English version and posted both online. With some encouragement from Greg Lehey (I'd participated in the group of volunteer reviewers for his third edition of The Complete FreeBSD), I submitted a proposal to give a half day tutorial for the 2005 AUUG conference in Sidney.
The proposal was accepted, as were several of the followup submissions to other conferences, and via a sequence of conferences and some private sessions, the document kept developing. In early 2007 I started working on turning the manuscript into a usable book. As regular readers will be aware, the much revised second edition was completed during the second half of 2010, and even that version has recently been subjected to its first update thanks to the ongoing development of the OpenBSD operating system.
The original tutorial has kept attracting a relatively steady stream of new visitors from all over the world, even though I have not added any new material to the document since I started working on the book version. New material will, rather, find its way into slides for the next session (such as the most recent one at BSDCan 2011), or will be put in the queue for possible upcoming book material.
During periods when I have had little visible output to offer, it has been interesting to see that the documents attract visitors and the occasional comment or suggestion for improvement. Then a little while back, I realized that in a not too distant future, the number of unique host names or IP addresses that have visited the tutorial tree will roll past a hundred thousand (100,000).
That particular number is possibly only significant to me, I keep the count of unique hosts accessing mainly to get an idea how many people have looked at my work. The raw number of page hits for the same location (we don't have any numbers for the early days when it was hosted at a now-defunct ISP) is fairly close to one and a half million, but I feel that number is a rather pointless statistic.
But when visitor number one hundred thousand arrives, how should we celebrate? I'm inclined to try to identify and contact the lucky visitor and offer a prize of sorts, but I have not quite made up my mind what and how. I'll welcome suggestions sent to via email (to peter@bsdly.net) with a recognizable subject.
It is worth mentioning that neither the tutorial nor this blog directly generates any revenue for me. I did for a short time have Google-supplied ads running on both sites, but for reasons that have never been quite clear to me, Google chose to terminate my AdSense account a few days before my second USD 100 transfer was due.
Showing posts with label PF book. Show all posts
Showing posts with label PF book. Show all posts
Sunday, June 5, 2011
Tuesday, November 9, 2010
The Book of PF, 2nd ed: It's Here!
Yes, it's that time of the year again -- we missed both Halloween and the OpenBSD 4.8 release, but hot on the heels of both, here it is:
The Book of PF, 2nd Edition is here, a box of author's copies turned up here just after lunchtime, and were taken well care of by Nora and Birthe.
This means, of course, that those of you who preordered will be receiving your copies shortly (mod the usual factors eloquently described by Michael Lucas here, the printer in my case is in Louiseville, Quebec), those who have reason to expect copies from my hoard here can rest assured that I'm taking them to the post office right after this. There's an illegible scrawl on some early pages, sorry 'bout that.
Better bookstores online and elsewhere will have it, or you could make it part of a bundle by ordering from the OpenBSD orders page. You will be going there for your six monthly fix anyway, won't you?
Upcoming events: The plans are not fixed yet, but you should expect me to turn up at BSD-themed events over the next few months. Look for announcements here, tweeted, or via the usual mailing lists.
NOTE: This article refers to the now outdated second edition, which has been superseded by The Book of PF, 3rd Edition, which covers changes up to and including OpenBSD 5.6. For the purpose of learning network technology in general and PF in particular, the significantly updated third edition is a better choice than the second edition. Also see the October 25, 2015 post about the arrival of my third edition author copies.
Wednesday, June 4, 2008
More than 40,000 served
Today's blog comes to you from sunny Aalborg in northern Denmark, where our Danish friends had the good sense to put together a one-day conference. Go to the web site at http://www.foss-aalborg.dk/ for details of the programme, I certainly hope the organizers will start a tradition and put on another conference soon.
For my own part, the PF tutorial (the free predecessor to the book), saw its confirmed unique visitor number forty thousand today, apparently a user somewhere in Ukraine:
$ ./mystats.sh
Wed Jun 4 14:57:26 CEST 2008
Total PF tutorial visitors : 40000
I promise I won't bother you with updates of the number of visitors again until we reach fifty thousand. I'll do my very best to have produced some other interesting material before then.
For my own part, the PF tutorial (the free predecessor to the book), saw its confirmed unique visitor number forty thousand today, apparently a user somewhere in Ukraine:
$ ./mystats.sh
Wed Jun 4 14:57:26 CEST 2008
Total PF tutorial visitors : 40000
I promise I won't bother you with updates of the number of visitors again until we reach fifty thousand. I'll do my very best to have produced some other interesting material before then.
Sunday, November 25, 2007
I Must Be Living in a Parallel Universe, Then
It's Sunday morning, and I'm having my morning coffee while getting ready for a long session of editing my OpenCON presentation. By working on adapting the presentation tailored to the tutorial I've been rediscovering just how much work went into making the book, so a long Sunday session is needed, if not more.
Then courtesy of Groklaw's news picks comes the USA today piece called Despite filters, tidal wave of spam bears down on e-mailers.
A tidal wave of spam, no less. Well, we're seeing a lot of attempts at sending, like the sequence here (text link, formatting it would take too long) that I captured from the xterm running a tail -f on my spamd log a little while back. That sequence tells me, for one thing, that the naive spambot thinks my spamd looks like an open relay.
The other interesting thing about the sequence there is the pattern you can see in the From: addresses. It may have dawned on some of the spammers that generating random addresses in other people's domains might end up poisoning their own well, so they started introducing patterns to be able to weed out their own made up addresses from their lists. I take that as a confirmation that our harvesting and republishing efforts here and elsewhere have been working rather well.
Here the method seems to be that they take the victim domain name, prepend "dw" and append "m" to make up the local part and then append the domain, so starting from sia.com we get dwsiam@sia.com.
There is one other common variation on that theme, where the prepend string is "lin" and the append string is "met", producing addresses like linhrimet@hri.de, used just a few minutes ago to try to spam malseeinvmk@bsdly.net from the apparently Polish adress 89.228.40.80. This is of course very interesting, as is the fact that right now about two and a half thousand machines are in my spamd-greytrap list . That's where they end up, making no waves at all.
On the subject of patterns, earlier this month the address capitalgain02@gmail.com started appearing frequently enough that it caught my attention in my greylist dumps and log files.
The earliest contact as far as I can see was at Nov 10 14:30:57, trying to spam wkzp0jq0n6.fsf@datadok.no from 193.252.22.241 (apparently a France Telecom customer). The last attempt seems to have been ten days later, at Nov 20 15:20:31, from the Swedish machine 217.10.96.36.
My logs show me that during that period 6531 attempts had been made to deliver mail from capitalgain02@gmail.com via bsdly.net, from 35 different IP addresses, to 131 different recipients in our domains. Those recipients included three deliverable addresses, mine or aliases I receive mail for. None of those attempts actually succeeded, of course. With a little more time on my hands I'm sure I could have made a good regular expression to calculate to the second how much time those spam senders wasted here, too.
So where's the tidal wave? Back when PDF spam was the new horror, it actually took three weeks for one to reach me, and then only via an alias on a machine I really don't have much control over anymore. The number of spam sending machines does seem to be increasing, though.
Bob Beck's uatraps list is a good indicator, and the tendency is clear from the graph in my malware paper. The number did dip just below 100,000 addresses earlier this month, and it now seems to have stabilized in the 110,000 to 120,000 range.
From my perspective, it looks like a reasonably configured spamd is really all we need to observe the tidal wave at a safe distance and have fun all the while.
It's almost like living in a parallel universe.
Then courtesy of Groklaw's news picks comes the USA today piece called Despite filters, tidal wave of spam bears down on e-mailers.
A tidal wave of spam, no less. Well, we're seeing a lot of attempts at sending, like the sequence here (text link, formatting it would take too long) that I captured from the xterm running a tail -f on my spamd log a little while back. That sequence tells me, for one thing, that the naive spambot thinks my spamd looks like an open relay.
The other interesting thing about the sequence there is the pattern you can see in the From: addresses. It may have dawned on some of the spammers that generating random addresses in other people's domains might end up poisoning their own well, so they started introducing patterns to be able to weed out their own made up addresses from their lists. I take that as a confirmation that our harvesting and republishing efforts here and elsewhere have been working rather well.
Here the method seems to be that they take the victim domain name, prepend "dw" and append "m" to make up the local part and then append the domain, so starting from sia.com we get dwsiam@sia.com.
There is one other common variation on that theme, where the prepend string is "lin" and the append string is "met", producing addresses like linhrimet@hri.de, used just a few minutes ago to try to spam malseeinvmk@bsdly.net from the apparently Polish adress 89.228.40.80. This is of course very interesting, as is the fact that right now about two and a half thousand machines are in my spamd-greytrap list . That's where they end up, making no waves at all.
On the subject of patterns, earlier this month the address capitalgain02@gmail.com started appearing frequently enough that it caught my attention in my greylist dumps and log files.
The earliest contact as far as I can see was at Nov 10 14:30:57, trying to spam wkzp0jq0n6.fsf@datadok.no from 193.252.22.241 (apparently a France Telecom customer). The last attempt seems to have been ten days later, at Nov 20 15:20:31, from the Swedish machine 217.10.96.36.
My logs show me that during that period 6531 attempts had been made to deliver mail from capitalgain02@gmail.com via bsdly.net, from 35 different IP addresses, to 131 different recipients in our domains. Those recipients included three deliverable addresses, mine or aliases I receive mail for. None of those attempts actually succeeded, of course. With a little more time on my hands I'm sure I could have made a good regular expression to calculate to the second how much time those spam senders wasted here, too.
So where's the tidal wave? Back when PDF spam was the new horror, it actually took three weeks for one to reach me, and then only via an alias on a machine I really don't have much control over anymore. The number of spam sending machines does seem to be increasing, though.
Bob Beck's uatraps list is a good indicator, and the tendency is clear from the graph in my malware paper. The number did dip just below 100,000 addresses earlier this month, and it now seems to have stabilized in the 110,000 to 120,000 range.
From my perspective, it looks like a reasonably configured spamd is really all we need to observe the tidal wave at a safe distance and have fun all the while.
It's almost like living in a parallel universe.
Sunday, October 28, 2007
Of Course, It Had To Be A Webshield
In an earlier blog post, I mentioned that I would buy a round of drinks the first time I saw an attempt to deliver a message with both the From: and To: addresses already on my spammer baiting list.
In fact it happened very soon afterwards, and as luck, misfortune or just plain old incompetence would have it, that message apparently came from a WebShield appliance not too far from here:
Oct 17 23:03:52 skapet spamd[20795]: 194.54.96.18: connected (6/4)
Oct 17 23:04:03 skapet spamd[20795]: (GREY) 194.54.96.18:
<capitulations7@datadok.no> -> <capitulations7@datadok.no>
Oct 17 23:04:03 skapet spamd[20795]: 194.54.96.18: disconnected
after 11 seconds.
Oct 17 23:19:21 skapet spamd[20795]: 194.54.96.18: connected (4/3)
Oct 17 23:19:32 skapet spamd[20795]: (GREY) 194.54.96.18:
<capitulations7@datadok.no> -> <capitulations7@datadok.no>
Oct 17 23:19:32 skapet spamd[20795]: 194.54.96.18: disconnected
after 11 seconds.
Oct 17 23:30:30 skapet spamd[20795]: 194.54.96.18: connected (4/4),
lists: spamd-greytrap
Oct 17 23:34:14 skapet spamd[20795]: (BLACK) 194.54.96.18:
<capitulations7@datadok.no> -> <capitulations7@datadok.no>
Oct 17 23:35:58 skapet spamd[20795]: 194.54.96.18: From:
Webshield.SMTP.V4.5.MR1a.Mail.Service@vs4.bgnett.no
Oct 17 23:35:58 skapet spamd[20795]: 194.54.96.18:
To: <capitulations7@datadok.no>
Oct 17 23:35:58 skapet spamd[20795]: 194.54.96.18:
Subject: Returned Mail: Error During Delivery
Oct 17 23:37:00 skapet spamd[20795]: 194.54.96.18:
disconnected after 390 seconds. lists: spamd-greytrap
Oct 17 23:57:18 skapet spamd[20795]: 194.54.96.18:
connected (6/6), lists: spamd-greytrap
I sent the operators at that site a polite message right away, pointing out the misconfiguration. Two weeks later I have seen no response other than the automatic acknowledgement, but it looks like the machine has managed to get itself automatically whitelisted in the meantime. So perhaps they found the button that actually does something.
Since my last blog post I have completed the book, and I expect the last bit of proofing to be done during the coming week. Then a few other necessary processes, and physical copies available for mid December if all goes well. With the cover in place, it looks like it's become attractive and popular over at amazon.com in its various categories. The BSD category there looks pretty No Starch dominated at the moment.
That can not be a bad thing. It's been a real pleasure working with the people at No Starch Press. If you think you want write a tech book, they should be on the list of publishers to contact with your proposal.
While all this was happening, the spammer baiting operation seems to have reached a critical mass of sorts. With roughly 7,200 addresses in the spamtrap list there are several hundred bait addresses for each real one in those domains taken together, so it's extremely unlikely that the spammers will ever get a chance to try delivery to a real address before they hit the tar pit. Over the last couple of weeks, my gateways have had anywhere between 2,500 and 4,000 hosts in the local spamd-greytrap, and anywhere from 0 to about 300 spambots pushing bytes into the tar pits at any time. It's fun to watch (some of the bots labor through the bait list from top to bottom), and the net effect is, well, we're not seeing much spam.
I think I've mentioned it before, but it bears repeating: To naive spammers and the tools they use, spamd looks like an open relay. Spamd never actually delivers any messages, but this
GREY|201.250.57.147|sofia|<vdaegkoxgk@bonana.com>|
<brad.james.anderson@jhg.com.au>|1193105605|1193127205|1193127205|1|0
says that whoever operates 201.250.57.147 (according to whois, likely located in or near Buenos Aires, Argentina), is unable to tell the difference between an open relay and spamd's 451 and subsequent "this is going to hurt you more than it hurts me" messages.
Another variation on that theme is what I think is some sort of amateurish relay testing, which typically produces anywhere from five hundred to a thousand greylist entries of the type
GREY|59.35.4.51|UATIM-F7E7949C7|<adgjnq@194.54.103.104>|
<ariel5268@yahoo.com.tw>|1193084672|1193113472|1193113472|2|0
GREY|59.35.4.51|UATIM-F7E7949C7|<xaehkn@rosalita.datadok.no>|
<ariel5268@yahoo.com.tw>|1193084675|1193113475|1193113475|2|0
GREY|59.35.4.51|UATIM-F7E7949C7|<qswyd@brutha.datadok.no>|
<ariel5268@yahoo.com.tw>|1193084691|1193113491|1193113491|2|0
GREY|59.35.4.51|UATIM-F7E7949C7|<nqtw@monalisa.datadok.no>|
<ariel5268@yahoo.com.tw>|1193084733|1193113533|1193113533|2|0
where the From parts are made up of host names and IP addresses in our local net, including in this case, the host name for one of our laser printers. Those floods have tended to swell the bait list a bit, even if I strip out the invalid @<IP address> ones.
Spamd makes the naive relay testers think we have a whole network of open relays, and we harvest the noise they generate to lead the spambots to the tarpit. That's pretty close to a hands-off spammer repellent for us, and a serious auto-LART for the spammers.
OpenCON is sneaking up on us in a month's time, and we're heading for Venice with a refreshed tutorial session. See you there!
PS - [non-IT PS coming up] Bergen's football (soccer) team SK Brann has just won the national league for the first time in 44 years. With one game to go before end of season they are so far ahead in points there is no way any other team will be able to catch up. The town is predictably going gaga over the event, and we joined the thousands at the central Festplassen square for the city sponsored celebration tonight. I'm surprised how many songs have been written about that team and how everybody around me seened to know every last word of the lyrics. Good fun, ending with fireworks.
In fact it happened very soon afterwards, and as luck, misfortune or just plain old incompetence would have it, that message apparently came from a WebShield appliance not too far from here:
Oct 17 23:03:52 skapet spamd[20795]: 194.54.96.18: connected (6/4)
Oct 17 23:04:03 skapet spamd[20795]: (GREY) 194.54.96.18:
<capitulations7@datadok.no> -> <capitulations7@datadok.no>
Oct 17 23:04:03 skapet spamd[20795]: 194.54.96.18: disconnected
after 11 seconds.
Oct 17 23:19:21 skapet spamd[20795]: 194.54.96.18: connected (4/3)
Oct 17 23:19:32 skapet spamd[20795]: (GREY) 194.54.96.18:
<capitulations7@datadok.no> -> <capitulations7@datadok.no>
Oct 17 23:19:32 skapet spamd[20795]: 194.54.96.18: disconnected
after 11 seconds.
Oct 17 23:30:30 skapet spamd[20795]: 194.54.96.18: connected (4/4),
lists: spamd-greytrap
Oct 17 23:34:14 skapet spamd[20795]: (BLACK) 194.54.96.18:
<capitulations7@datadok.no> -> <capitulations7@datadok.no>
Oct 17 23:35:58 skapet spamd[20795]: 194.54.96.18: From:
Webshield.SMTP.V4.5.MR1a.Mail.Service@vs4.bgnett.no
Oct 17 23:35:58 skapet spamd[20795]: 194.54.96.18:
To: <capitulations7@datadok.no>
Oct 17 23:35:58 skapet spamd[20795]: 194.54.96.18:
Subject: Returned Mail: Error During Delivery
Oct 17 23:37:00 skapet spamd[20795]: 194.54.96.18:
disconnected after 390 seconds. lists: spamd-greytrap
Oct 17 23:57:18 skapet spamd[20795]: 194.54.96.18:
connected (6/6), lists: spamd-greytrap
I sent the operators at that site a polite message right away, pointing out the misconfiguration. Two weeks later I have seen no response other than the automatic acknowledgement, but it looks like the machine has managed to get itself automatically whitelisted in the meantime. So perhaps they found the button that actually does something.
Since my last blog post I have completed the book, and I expect the last bit of proofing to be done during the coming week. Then a few other necessary processes, and physical copies available for mid December if all goes well. With the cover in place, it looks like it's become attractive and popular over at amazon.com in its various categories. The BSD category there looks pretty No Starch dominated at the moment.
That can not be a bad thing. It's been a real pleasure working with the people at No Starch Press. If you think you want write a tech book, they should be on the list of publishers to contact with your proposal.
While all this was happening, the spammer baiting operation seems to have reached a critical mass of sorts. With roughly 7,200 addresses in the spamtrap list there are several hundred bait addresses for each real one in those domains taken together, so it's extremely unlikely that the spammers will ever get a chance to try delivery to a real address before they hit the tar pit. Over the last couple of weeks, my gateways have had anywhere between 2,500 and 4,000 hosts in the local spamd-greytrap, and anywhere from 0 to about 300 spambots pushing bytes into the tar pits at any time. It's fun to watch (some of the bots labor through the bait list from top to bottom), and the net effect is, well, we're not seeing much spam.
I think I've mentioned it before, but it bears repeating: To naive spammers and the tools they use, spamd looks like an open relay. Spamd never actually delivers any messages, but this
GREY|201.250.57.147|sofia|<vdaegkoxgk@bonana.com>|
<brad.james.anderson@jhg.com.au>|1193105605|1193127205|1193127205|1|0
says that whoever operates 201.250.57.147 (according to whois, likely located in or near Buenos Aires, Argentina), is unable to tell the difference between an open relay and spamd's 451 and subsequent "this is going to hurt you more than it hurts me" messages.
Another variation on that theme is what I think is some sort of amateurish relay testing, which typically produces anywhere from five hundred to a thousand greylist entries of the type
GREY|59.35.4.51|UATIM-F7E7949C7|<adgjnq@194.54.103.104>|
<ariel5268@yahoo.com.tw>|1193084672|1193113472|1193113472|2|0
GREY|59.35.4.51|UATIM-F7E7949C7|<xaehkn@rosalita.datadok.no>|
<ariel5268@yahoo.com.tw>|1193084675|1193113475|1193113475|2|0
GREY|59.35.4.51|UATIM-F7E7949C7|<qswyd@brutha.datadok.no>|
<ariel5268@yahoo.com.tw>|1193084691|1193113491|1193113491|2|0
GREY|59.35.4.51|UATIM-F7E7949C7|<nqtw@monalisa.datadok.no>|
<ariel5268@yahoo.com.tw>|1193084733|1193113533|1193113533|2|0
where the From parts are made up of host names and IP addresses in our local net, including in this case, the host name for one of our laser printers. Those floods have tended to swell the bait list a bit, even if I strip out the invalid @<IP address> ones.
Spamd makes the naive relay testers think we have a whole network of open relays, and we harvest the noise they generate to lead the spambots to the tarpit. That's pretty close to a hands-off spammer repellent for us, and a serious auto-LART for the spammers.
OpenCON is sneaking up on us in a month's time, and we're heading for Venice with a refreshed tutorial session. See you there!
PS - [non-IT PS coming up] Bergen's football (soccer) team SK Brann has just won the national league for the first time in 44 years. With one game to go before end of season they are so far ahead in points there is no way any other team will be able to catch up. The town is predictably going gaga over the event, and we joined the thousands at the central Festplassen square for the city sponsored celebration tonight. I'm surprised how many songs have been written about that team and how everybody around me seened to know every last word of the lyrics. Good fun, ending with fireworks.
Saturday, September 29, 2007
Always a pleasure to be wasting your time, guv
This week has been a little unusual around the BSDly household. So far I've generally been doing my regular job in the daytime (with longish office hours), only working on the book evenings and weekends. That the arrangement would lead to "Exhaustion is my middle name" status was obvious to everyone except me, but I finally saw where it could be going. So for a little more than the past week I've been working on the book full time.
The state of perpetual exhaustion has had some not too happy consequences. Of course the general progress on the book suffered, but it also lead to me missing the monthly BLUG meeting in August. Of course much of that particular day I had spent persuading somebody not too bright that it indeed had to be a reconfiguration they said had never happend at their end which ended up breaking things at our end, and I was just too tired and missed what I assume was a well executed lecture on networking basics by Vegard Engen (of RFC1149 implementation fame).
This week with only one job I needed to tackle, I was there for an enjoyable one and a half hours of Bacula, well presented by Bård Aase (aka elzapp). Off to Henrik (the regular BLUG pub) for a few beers afterwards, and with Johan Riise volunteering to put together a 'Unix and time' lecture for next month, the BLUG calender seems to be in order after all, with Jill Walker doing the end of semester talk in November, on whatever interesting stuff she has been up to lately. Unfortunately it looks like the last Thursday of November is close enough to OpenCON that I'll likely miss Jill's session.
In the meantime, there are signs that the greytrapping and my bait list is working. Looking over the spamd logs today I found quite a few entries like these:
which looks strikingly like the Spanish lottery scam spammers patiently and methodically working their way through my list of bait addresses, all the way from top to bottom, at roughly 3000 addresses it's going be a while. All I can say is, we are extremely pleased to be wasting your time, senor.
Also while the girls were off to the Raptus comics festival (an annual event, and one of the big things here in Bergen), I found enough trash backscatter to non-existent bsdly.net addresses that it's likely that the same weekend spambot operators who spewed their spam with @ehtrib.org and @skapet.datadok.no addresses earlier (both times at weekends) have now discovered bsdly.net and are doing their damnedest.
Why they prefer to generate a few hundred fake addresses and use them all in one go is beyond me. The other groups seem to generate only a handful of new addresses each every day, and for good measure at least one of them sort of reuse the generated addresses by using a forward and a reverse (such as in this morning's preserved greylist dumps, there was a potterv76@datadok.no as well as the reverse 67VRETTOP3@datadok.no). This lot just dumps all they have in one go, mainly contributing to swelling that file in my home directory with the totally unprintable file name which is the temporary storage before they go to into the traplist and on to the bait page.
Distractions of that kind from my main task is never entirely welcome, but with a larger influx of new addresses to be added to the bait list I made some small changes to make the maintenance of that page a bit more sane, rediscovering server-side includes and redirects along the
way. And the data I keep collecting may become the basis for other projects later.
Anyway, it is increasingly clear that the spammers are including the generated fake addresses in their "known good" lists. Consider the spambot at 210.111.190.216 (apparently in Korea), which insists on delivering to an address somebody generated in early July:
I have no real opinion on the validity of the From: addresses, but the address they are trying their best to deliver spam to here never actually existed, of course. The first record of it at datadok.no was this bounce from a Russian site:
Dumping their trash back at them is good for a laugh, and I am quite amazed how shortsighted the spambot operators appear to be. They get yelled at for spamming, so to avoid detection, they start using fake addresses. This in turn means they have no feedback whatsoever on the quality of their address lists, and with well pissers like me in action, they are getting less effectitive each day, reducing themselves to background noise in the network.
Now with this blog post done I will go back and finish the edits on the logs chapter. With the early parts of the book about to enter the layout phase while the last bits get written over the next few days, there is a chance that there will be a physical copies of the book to pass around at OpenCON. Not quite there yet, but the fulltime push is certainly helping. The preface with a list of thanks is part of what is entering layout; I think a few people who did not expect to be in there will soon have a pleasant surprise.
Also this week, the PF tutorial saw its unique visitor number 19,000 since EuroBSDCon 2006 on Thursday morning (September 27th). We certainly hope at least some of them will come back for the book.
The state of perpetual exhaustion has had some not too happy consequences. Of course the general progress on the book suffered, but it also lead to me missing the monthly BLUG meeting in August. Of course much of that particular day I had spent persuading somebody not too bright that it indeed had to be a reconfiguration they said had never happend at their end which ended up breaking things at our end, and I was just too tired and missed what I assume was a well executed lecture on networking basics by Vegard Engen (of RFC1149 implementation fame).
This week with only one job I needed to tackle, I was there for an enjoyable one and a half hours of Bacula, well presented by Bård Aase (aka elzapp). Off to Henrik (the regular BLUG pub) for a few beers afterwards, and with Johan Riise volunteering to put together a 'Unix and time' lecture for next month, the BLUG calender seems to be in order after all, with Jill Walker doing the end of semester talk in November, on whatever interesting stuff she has been up to lately. Unfortunately it looks like the last Thursday of November is close enough to OpenCON that I'll likely miss Jill's session.
In the meantime, there are signs that the greytrapping and my bait list is working. Looking over the spamd logs today I found quite a few entries like these:
Sep 29 15:29:23 skapet spamd[20795]: (BLACK) 84.76.177.159: <royaleuromillion2007@yahoo.es> -> <211hgsreliart7@datadok.no> Sep 29 15:29:32 skapet spamd[20795]: (BLACK) 84.76.177.159: <royaleuromillion2007@yahoo.es> -> <00b27f18@datadok.no>
which looks strikingly like the Spanish lottery scam spammers patiently and methodically working their way through my list of bait addresses, all the way from top to bottom, at roughly 3000 addresses it's going be a while. All I can say is, we are extremely pleased to be wasting your time, senor.
Also while the girls were off to the Raptus comics festival (an annual event, and one of the big things here in Bergen), I found enough trash backscatter to non-existent bsdly.net addresses that it's likely that the same weekend spambot operators who spewed their spam with @ehtrib.org and @skapet.datadok.no addresses earlier (both times at weekends) have now discovered bsdly.net and are doing their damnedest.
Why they prefer to generate a few hundred fake addresses and use them all in one go is beyond me. The other groups seem to generate only a handful of new addresses each every day, and for good measure at least one of them sort of reuse the generated addresses by using a forward and a reverse (such as in this morning's preserved greylist dumps, there was a potterv76@datadok.no as well as the reverse 67VRETTOP3@datadok.no). This lot just dumps all they have in one go, mainly contributing to swelling that file in my home directory with the totally unprintable file name which is the temporary storage before they go to into the traplist and on to the bait page.
Distractions of that kind from my main task is never entirely welcome, but with a larger influx of new addresses to be added to the bait list I made some small changes to make the maintenance of that page a bit more sane, rediscovering server-side includes and redirects along the
way. And the data I keep collecting may become the basis for other projects later.
Anyway, it is increasingly clear that the spammers are including the generated fake addresses in their "known good" lists. Consider the spambot at 210.111.190.216 (apparently in Korea), which insists on delivering to an address somebody generated in early July:
peter@skapet:~/www_sider$ grep 210.111.190.216 /var/log/spamd Sep 29 15:58:07 skapet spamd[20795]: 210.111.190.216: connected (5/4) Sep 29 15:58:21 skapet spamd[20795]: (GREY) 210.111.190.216: <jim.vance@presentsmadeeasy.com> -> <careersogt2083@datadok.no> Sep 29 15:58:22 skapet spamd[20795]: 210.111.190.216: disconnected after 15 seconds. Sep 29 15:58:35 skapet spamd[20795]: 210.111.190.216: onnected (4/3) Sep 29 15:58:49 skapet spamd[20795]: (GREY) 210.111.190.216: <tbaker@groupecdb.com> -> lt;careersogt2083@datadok.no> Sep 29 15:58:50 skapet spamd[20795]: 210.111.190.216: disconnected after 15 seconds. Sep 29 15:59:03 skapet spamd[20795]: 210.111.190.216: connected (5/3) Sep 29 15:59:17 skapet spamd[20795]: (GREY) 210.111.190.216: <wotan@4vsi.com> -> <careersogt2083@datadok.no> Sep 29 15:59:18 skapet spamd[20795]: 210.111.190.216: disconnected after 15 seconds. Sep 29 15:59:30 skapet spamd[20795]: 210.111.190.216: connected (6/5), lists: spamd-greytrap Sep 29 16:03:14 skapet spamd[20795]: (BLACK) 210.111.190.216: <sylviacastleman@alltypecalligraphy.com> -> <careersogt2083@datadok.no> Sep 29 16:04:59 skapet spamd[20795]: 210.111.190.216: From: "Marguerite Casey" <sylviacastleman@alltypecalligraphy.com> Sep 29 16:04:59 skapet spamd[20795]: 210.111.190.216: To: <careersogt2083@datadok.no> Sep 29 16:04:59 skapet spamd[20795]: 210.111.190.216: Subject: 100mg x 60 pills US $ 129.95 buy now Sep 29 16:06:04 skapet spamd[20795]: 210.111.190.216: disconnected after 394 seconds. lists: spamd-greytrap
I have no real opinion on the validity of the From: addresses, but the address they are trying their best to deliver spam to here never actually existed, of course. The first record of it at datadok.no was this bounce from a Russian site:
Jul 12 23:38:52 delilah spamd[29851]: (GREY) 81.177.34.190: <> -> <careersogt2083@datadok.no>
Dumping their trash back at them is good for a laugh, and I am quite amazed how shortsighted the spambot operators appear to be. They get yelled at for spamming, so to avoid detection, they start using fake addresses. This in turn means they have no feedback whatsoever on the quality of their address lists, and with well pissers like me in action, they are getting less effectitive each day, reducing themselves to background noise in the network.
Now with this blog post done I will go back and finish the edits on the logs chapter. With the early parts of the book about to enter the layout phase while the last bits get written over the next few days, there is a chance that there will be a physical copies of the book to pass around at OpenCON. Not quite there yet, but the fulltime push is certainly helping. The preface with a list of thanks is part of what is entering layout; I think a few people who did not expect to be in there will soon have a pleasant surprise.
Also this week, the PF tutorial saw its unique visitor number 19,000 since EuroBSDCon 2006 on Thursday morning (September 27th). We certainly hope at least some of them will come back for the book.
Saturday, September 8, 2007
Wanna help science? Study your greylists' innards!
If somebody, say five years ago, had told me that I would be spending a little time, every day, studying data about what invalid addresses some unknown miscreants are making up in my domains, I would have thought them to be slighly off their rockers.
Yet here I am, actually maintaining a publicly available list of addresses which do not stand a chance of becoming valid, ever. It all started with a log data anomaly - I noticed an increase in the number of failed delivery messages to non-existent addresses in our domains. I had expected that the bounces to invalid addresses would appear for a short period only, but for one reason or the other it looks like it's here to stay, with some dips and peaks like the ehtrib.org flood.
The list is apparently working as intended too. These addresses are on my local greytrap list, and I have started seeing addresses I put in there as all uppercase turn up in my logs in all lowercase variants. Fun to watch, sort of.
Anyway, the supply of new bogus addresses proved to be larger than I had expected. So to get a handle on just what is happening I ended up doing periodic dumps of the live greylist data. This is really easy to do if you're using spamd as your greylister, your basic command is
$ sudo spamdb | grep GREY
and you redirect to a file, pipe to mail, or whatever you like.
Now if you're a bit like me, looking for patterns in the noise like this makes you feel a little weirder than usual and possibly lead you to think of a Clive Barker novel (specifically the bits about the dead letter file in The Great and Secret Show) and you wonder why this is worth doing at all. After all there is precious little spam that actuall reaches my users, so like I said earlier, for us spamd users it really looks like spam is a solved problem. I guess I'm just a bit fascinated by the pure irrationality of the spammers' behavior.
From the data I collect here in my tiny corner of the world to browse when time allows there may be useful information lurking somewhere.
Typical entries show things like the host 202.152.33.43 tried to send with a From: address jcejft@charter.com to dkqvujfn@datadok.no and sdenuuu@datadok.no. Using a few common networking commands we see that there is no reason why charter.com email should come from the IP range belonging to idola.net.id, and as the admin of datadok.no I know these two addresses have never been deliverable. Most likely the admin at charter.com can tell you if that from address is deliverable, but I keep wondering how much of the spam out there is stuffed into the pipe with bogus From: and To: addresses both. Or in other words, purely useless noise, never to be delivered anywhere.
On a side note, with one or more of the spammer operations trying to sneak through using sender and recipient addresses in the target domain, I assume it is just a matter of time before I see a tuple with both sender and recipient addresses already in my spamtraps list. When that happens, I think I will feel inclined to let my friends have a round of refreshments on my tab.
It's obvious that there are a handful of spammer operations that have decided to use datadok.no (and to a lesser extent, dataped.no and ehtrib.org) From: addresses on the spam they send, apparently in an attempt to cover their tracks. I will probably never know why they decided to do that, but I wonder why they keep it up and for that matter how many other domains are seeing this, with bounces from strange places, directed at non-existent, fairly obviously generated bogus addresses.
So if you are seeing similar stupidity in your logs and if you are running a sensible greylister such as spamd, I would be interested in hearing from you so we can compare notes.
Out there in meatspace, EuroBSDCon 2007 is coming up. I'll be there with the PF tutorial on Wednesday. This Friday's deadline for an updated manuscript had totally slipped my mind (I blame the book and a few other, less rational, factors), but hopefully the 24 who signed up for the session will find it useful anyhow - there will be new bits and as much interesting stuff as I can manage. I'll be around for the rest of the conference too, but unfortunately I'll have to give the Legonland trip a miss.
Be seeing you in Copenhagen! The book is getting closer to finished, I promise!
Yet here I am, actually maintaining a publicly available list of addresses which do not stand a chance of becoming valid, ever. It all started with a log data anomaly - I noticed an increase in the number of failed delivery messages to non-existent addresses in our domains. I had expected that the bounces to invalid addresses would appear for a short period only, but for one reason or the other it looks like it's here to stay, with some dips and peaks like the ehtrib.org flood.
The list is apparently working as intended too. These addresses are on my local greytrap list, and I have started seeing addresses I put in there as all uppercase turn up in my logs in all lowercase variants. Fun to watch, sort of.
Anyway, the supply of new bogus addresses proved to be larger than I had expected. So to get a handle on just what is happening I ended up doing periodic dumps of the live greylist data. This is really easy to do if you're using spamd as your greylister, your basic command is
$ sudo spamdb | grep GREY
and you redirect to a file, pipe to mail, or whatever you like.
Now if you're a bit like me, looking for patterns in the noise like this makes you feel a little weirder than usual and possibly lead you to think of a Clive Barker novel (specifically the bits about the dead letter file in The Great and Secret Show) and you wonder why this is worth doing at all. After all there is precious little spam that actuall reaches my users, so like I said earlier, for us spamd users it really looks like spam is a solved problem. I guess I'm just a bit fascinated by the pure irrationality of the spammers' behavior.
From the data I collect here in my tiny corner of the world to browse when time allows there may be useful information lurking somewhere.
Typical entries show things like the host 202.152.33.43 tried to send with a From: address jcejft@charter.com to dkqvujfn@datadok.no and sdenuuu@datadok.no. Using a few common networking commands we see that there is no reason why charter.com email should come from the IP range belonging to idola.net.id, and as the admin of datadok.no I know these two addresses have never been deliverable. Most likely the admin at charter.com can tell you if that from address is deliverable, but I keep wondering how much of the spam out there is stuffed into the pipe with bogus From: and To: addresses both. Or in other words, purely useless noise, never to be delivered anywhere.
On a side note, with one or more of the spammer operations trying to sneak through using sender and recipient addresses in the target domain, I assume it is just a matter of time before I see a tuple with both sender and recipient addresses already in my spamtraps list. When that happens, I think I will feel inclined to let my friends have a round of refreshments on my tab.
It's obvious that there are a handful of spammer operations that have decided to use datadok.no (and to a lesser extent, dataped.no and ehtrib.org) From: addresses on the spam they send, apparently in an attempt to cover their tracks. I will probably never know why they decided to do that, but I wonder why they keep it up and for that matter how many other domains are seeing this, with bounces from strange places, directed at non-existent, fairly obviously generated bogus addresses.
So if you are seeing similar stupidity in your logs and if you are running a sensible greylister such as spamd, I would be interested in hearing from you so we can compare notes.
Out there in meatspace, EuroBSDCon 2007 is coming up. I'll be there with the PF tutorial on Wednesday. This Friday's deadline for an updated manuscript had totally slipped my mind (I blame the book and a few other, less rational, factors), but hopefully the 24 who signed up for the session will find it useful anyhow - there will be new bits and as much interesting stuff as I can manage. I'll be around for the rest of the conference too, but unfortunately I'll have to give the Legonland trip a miss.
Be seeing you in Copenhagen! The book is getting closer to finished, I promise!
Subscribe to:
Posts (Atom)
Posts
