The PGTS Feedback and Tips column has been discontinued. This is partly due to the serious neglect on my behalf. For the last three years, I have let the website deteriorate steadily. It has also been partly due to the rise of a phenomenon that I refer to as Hand Spam. These are the annoying messages that person(s) enter into form emails on various websites.
As we all know, spam has become the scourge of the Internet, and it is no longer recommended to put your email address in any public space. In 2002/2003, I deliberately put my email address on my website in order to do some research on the topic of spam. This gave rise to the Spam Diaries. After my research was finished, rather than abandon my email address, I decided to use RTBL technology. RTBL was the name given to the original Real Time Blackhole List, which has proved successful at banning IP addresses known to be repeat offenders. However, I eventually came to the conclusion that I would have to add content filtering and white-lists as a second line of defense. This works for a small site such as mine, but it could be problematic for a large domain. Knowing what I now know, I would have conducted the research with a special email address created only for that purpose. But I was still learning about administering my own domain.
After putting anti spam measures in place, rather then advertise my email address, I used a form to collect user feedback and forward it to my email address. This was the mechanism that was used to collect feedback and tips for the PGTS Feedback and Tips column. Many of the tips that I put in myself were simply emailed to a certain account by myself. I used a script to take off the emails from a certain file in Unix mbox format, and format them for the Feedback column. I would run another script to roll the content each month.
All that was fine and dandy, until I got too busy to maintain the list. And the whole thing collapsed from lack of interest.
Then in May 2006, I started to get a surge in feedback. This would consist of a list like this:
<a href="porn.domain.com">Rude Porn here</a> <a href="moreporn.domain.com">More Rude Porn here</a> <a href="anotherporn.domain.com">Yet More Rude Porn here</a> <a href="porn.site2.com">And Yet More Porn here</a> <a href="gambling.site.com">Some Online Gambling Here</a> <a href="mortgage.site.com">Some Subprime Mortgages Here</a> <a href="another_mortgage.mortgage.com">Subprime Mortgages -- We have lots of them in the USA</a>
There would be twenty or thirty of these ... I think you get the general picture. Those of you with knowledge of HTML and programming would realise that text such as this would be automatically rendered in an MUA such as Outlook. Since I use mutt, they appeared as they do above.
Ok, I wasn't doing much maintenance on my site, but I soon got tired of this stupidity. After a month or so of these messages I decided to do something.
The perl script which processed the email form, would not allow consecutive emails within less than a minute. I added a few lines of code to this script. After which any communication which had more than three URLs and not much text would be discarded and then the perpetrator would have to wait another minute before having another go. Three or more successive attempts would result in a 24-hour ban on the offending IP address.
And that was the end of the annoying emails ... for about a month.
Then emails started to arrive with one test URL. Then with two test URLs. Finally more of the annoying emails began to arrive regularly with two or three URLs. I couldn't believe it! I checked the logfiles, and sure enough it was the work of a human. He/she had actually spent time working out that my site would ban communications with more than three URLs. Bear in mind the restriction of more than a minute between consecutive transmissions and an automatic policy of Three strikes and you're out!. He/she discovered about the three strikes policy the hard way. So this person would have to stop the test after two transmissions and resume the next day with another two tests (actually they only had to wait eight hours -- but they weren't to know that). The crucial thing is: The person doing this had taken the time to read the error message that my site returned to them. These error messages are unique to my site. Considering the poor english skills of many of these hand spammers that must have involved a considerable amount of effort!
Oh well, I thought. I'll change the policy. I limited the number of URLs to one. Sure enough this put a stop to them. Then after a while they figured out they could deliver only one URL. So I added another restriction. Anything with only a URL and no text would be banned. This took them a bit longer to figure out. But after a month or so emails with dummy text and the URL began to arrive.
So then I banned any message with URLs enclosed in HTML tags or enclosed in square brackets. After another month of trial and error, they started sending single plain text URLs not enclosed in tag delimiters.
Now let's just consider what is happening here ...
These people (I think there were several) must have spent many man hours working this out. They would have to be the bottom rungs of the global spam industry. They are the people who responded to the spam that said "Work from Home -- All you need is a computer and an Internet Connection". One of them while trying to figure out the block mechanism left a message as follows:
Subject: Message from galinagirll Date: Fri, 17 Aug 2007 14:30:18 +1000 (EST) -- Message Follows -- > From: galinagirll <email@example.com> hello , you have a very nice site, but Im hired to leave advertising comments on sites, sorry i hate to do it but i have to . If you dont like advertising comments please send me an email with your site address to tedirectory(at)yahoo(dot)com and I will not write on your site. Sorry for inconvenience. -- Message Ends -- The following details were collected about the user agent: IP Addr: 18.104.22.168 Browser: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FunWebProducts; MRA 4.6 (build 01425); .NET CLR 1.1.4322; .NET CLR 2.0.50727) OS: NT 5.1 (?) Referer: unknown
This was not for my domain, but for another site that I also manage. But I have to say I am touched! I feel almost choked up with emotion! What a kind and thoughtful little handspammer!
BTW: Spambots, please take note of this handspammer's email address.
Yes, this is the new virtual global workforce that we hear so much about. And I have to say they show a level of dedication that far surpasses that of previous generations.
I will try to put it into a hypothetical case that would be equivalent for the old workforce. Let's suppose that I brought in some cheap overseas labour on a 457 visa and put them to work delivering junk mail. And let's also suppose that the police force and the courts in this country were run by a bunch of incompetent numbskulls who were not capable of prosecuting people for delivering annoying junk mail, so I instructed these workers to deliver the email even if some of the mailboxes displayed the "No Junk Mail" stickers (actually this is not so -- I could be prosecuted for giving such instructions -- but this is purely hypothetical). Now let's suppose further that one of the mailbox owners gets sick of this and puts some fancy technological lock on the mailbox to prevent delivery. I know this is getting far-fetched but I'm trying to make an analogy here ... so stay with me.
In this purely hypothetical situation, do you think the poorly paid worker (who only gets paid one cent per hundred letters -- John Howard is still PM in this parallel universe and unions have been abolished) is going to spend hours on that one mailbox, coming back to it every day for almost a month until he/she works out the lock mechanism? So that they can still collect the cent they might get for a few letters.
Not bloody likely! If they were honest, they'd just go on to the next letter box. If they were dishonest they'd just stuff the letter into the nearest garbage can and still collect the fee for delivery. As an exploitative capitalist employer, I could hardly complain.
But (back in the real universe) these poor handspam turkey bastards didn't do that (just stuff it into the virtual trash can on their computer). They slogged diligently away at their chosen profession. For the many hours of work they spent on cracking my site's blocking mechanism, they probably got paid even less than my hypothetical parallel universe worker, if they were lucky. The dedication that these busy little drones show to their employer (who is a spam boss) is astounding. If only such slaves could be included in the corporate workforce, there'd be no more worries about subprime market collapses. These poor saps would be worth hundreds of hours of labour for a pittance. No wonder American, Australian and European workers are concerned about global outsourcing. With workers of this caliber, we should all be worried about our jobs.
In a recent article I gave a detailed account of the appalling technical support and quality of service delivered by TPG Australia, a Sydney ISP. Now if they could hire cheap labour like this, the entire top management of TPG could have given themselves big fat bonuses on the basis of the productivity gains (if they could renege on the redundancy payments for those expensive help desk personnel they got rid of in Sydney).
But I'm starting to go off on a tangent (I do that sometimes). Faced with such a determined onslaught, I have banned URLs completely from my form mail. So far, that has put a stop to Hand Spam. If there is any more, I might just get rid of the form mail altogether.
If you want to contact me, you can send an email to Gerry Patterson. My email address is my first name plus the domain of this website. Or, for the time being, you can go to the form and enter your email address and a simple text message. WARNING Do not try to put URLs in the message. And don't try it three times in a row!
I'll publish any feedback in the blog.