Beating comment spam: Improving the Honey Pot
Mikey 31 comments
Update: We are no longer using this method as it, after more than a year of successful use, seems to be beaten regularly now.
Back in January I posted an article outlining our honey pot method of beating comment spam, aptly titled "how we beat comment spam". It's been more than six months since we first put it into practice, and so far we are still at a 100% success rate. There have been the occasional deliberate manual spammers, but they are unavoidable.
There is some legitimate criticism regarding that method, the most obvious being accessibility. We hope to address this and other issues with a new slightly more polished implementation. Another thing we did not even consider at the time was form fillers like roboform, which was pointed out by James last week. This has now been rectified by changing the field name to something other than 'lastname'. I will use 'muffins' for this article as I can not think of an instance where that might already be used by a form filler.
If you aren't sure what a honey pot is or you want to read about how we did it, read the previous article. But for the sake of comparison I will show it again below, albeit briefly, or you can jump past it.
Current honey pot implementation.
We have 4 fields, one of which is hidden with CSS. Spam bots fill in all the fields by default, including the hidden one. If the form is submitted with anything inside the hidden field, then the post is simply rejected. The accessibility issue is obvious – screen readers will still ‘see' the field, and unsuspecting users might fill it in and consequently have their comments discarded.
New implementation.
We have a simple three-part work around to the problem of people accidentally filling on the hidden 'muffin' field.
1) Place the honey pot field (muffin) after the submit button, as it is assumed that once you get to a submit button in a form, there are no fields after it to be filled in. But what if someone tabs once extra into the field by accident, and presses a key? Submitting the comment would see it rejected as spam. To fix this...
2) Place some JavaScript on the 'muffin' field which simply sends the cursor back to the previous field on focus.
document.form1.comment.focus();" />
...which is document . the name of the form . the name of the field to shift focus to . focus();
OK so it's not perfect as it relies on JavaScript being enabled. But that is a judgment call for each web master to make. In our case, stats show that practically all our visitors have javascript enabled.
3) As an additional indicator, we place a text notice hidden by a css class above the honey pot field (as an accessibility courtesy):
please leave the following field empty
Hiding the honey pot field without using 'visibility:hidden'.
Instead of using 'visibility:hidden' in the class for the honey pot field, we leave it visible. Don't panic I'm not done yet. We styled it so it blends in with your web site. In the case of this sites it is simply white borders, white background, and white text.
.access text {
border:solid 1px #fff;
background-color:#fff;
color:#fff;
}
Take it a step further if you want by making the height of the field 0px (to minimise any affect on your design).
That addresses any issues of people visually spotting it, and more importantly spam bots looking for 'visibility:hidden' or 'display:none' and interpreting it as suspicious (a potential honey pot), and deliberately ignoring it.
Conclusions.
This is a slight variation on the previous method which to date has still blocked all comment spam on this site, but there is no guarantee it will work for everyone. We understand some spam bots are smarter than others, and we also concede that someone clever enough could make a bot to specifically target this site.
I am hoping our technically skilled readers will suggest ways for improvement, tweaks and what not, or point out anything that may be a problem.
Acknowledgments: Kudos to Mark, Muzza and Aaron for feedback. And thanks to everyone who provided comments on the previous article.
Update 01
For the last few days or so we have been getting comment spam. Some of it is obviously manual in an attempt to be funny in light of this particular subject matter, but we can not be certain of the large majority. In any event we delete them. This particular subject has brought us a lot of exposure and there is no reason to believe that spammers have not picked up on it and made the necessary changes to target our site (or maybe I am being paranoid?).
Update 02
It appears honey pots are merely a temporary solution, most suitable for (smaller?) sites that fly under the spam bot radar. I conclude this from the fact we have seen our best web site traffic ever lately (nearly double from previous months) and our honey pot article seems to have been picked up by a number of programming and security web sites, of which I have no doubt are frequented by spammers for information.
Update 03
At the time of writing we had this method running on this site, but by the time you read this we will be working on (or have already completed) implementing a more drastic solution.
Update 04
RIP Honey pot. We're using our own custom built CAPTCHA system now with 100% success.
Georgio R.
Monday 6th August 2007 | 08:42 PMNot too shabby at all gentlemen. Is it a viable alternative to captcha? I think one of the most important things to generating discussion on a blog is to not force registration and to not force deciphering captchas.
PY
Monday 13th August 2007 | 04:18 AMGood Ideas. It was working for a while, but now spam bots (at least one...) retrieve the URL just after the post of the comment.
Therefore I think they 'intelligent' enough to guess what is the honey pot field, or not to send it anymore (just a guess).
PY
Friday 17th August 2007 | 12:05 AMTo avoid that, I did implement a small javascript event when user is typing in a mandatory field, that fills out the hidden field with a specific keyword.
It seems to work so far, since Spam bot don't interpret javascript... yet.
Sergey
Saturday 18th August 2007 | 03:34 AMSpam bots keep getting better and better.
Bots already open sites with IE, they wait before submit.
They not looking to the field names. Bots submit form in all combinations (they try to submit email, name, year to all available inputs, then name, email, year) and message to the textarea field. There are other ways to protect, but what if no javascript? what if no css? Visitor will be confused...
I proud to introduce you ActiveSpamProtection. My friend Victor uses flash to protect sites from spammers. And now you can see it on his new site.
It will take too much time for any programmer to write bot that will able to work with flash (popular ActiveX application). Programmers are too lazy. So putting 'Flash Form' on your page you will save your blog for several years.
Also ActiveSpamProtection is so nice looking!
http://dogs.triwe.net/dog-care1.php
No more stupid CAPTCHA
No more additional questions.
For example, on the page I'm showing you:
if you did not fill textarea, the dog will remind you to fill it.
Please let me know (in comments on this blog) what you think about it.
cam
Sunday 19th August 2007 | 11:15 PMPerhaps putting the honeypot field after the submit button is a mistake - if spambots simply read the raw HTML they may stop at the rather than the
cam
Sunday 19th August 2007 | 11:16 PMand that was stop at the [input type=submit] rather than the [/form]
SaleBum
Monday 20th August 2007 | 06:53 AMVery cool idea. I am actively involved with blogging every day and this is the first I have seen this method. Well done!
SaleBum
Don Buelke
Friday 26th October 2007 | 02:33 AMI'm a neophyte on web design with only VERY rudimentary understanding of web code and designed my site with a "cookbook" program. Do you have a cookbook version of your Honeypot that I can install? I'm having trouble following your instructions.
Hi Don,
Unless I am mistaken it looks like you have a static web site (no database) so implementing this solution would not be possible. Furthermore I couldn't see a comment form on your site anyway. Unless it is for a different site you want it for?
PS: Love the view from your office window. I have a view of a wall :-)
frustrated
Sunday 28th October 2007 | 11:32 AMHello,
I run a small site that's recently been attacked by some kind of spambot via its comment form. I've done a lot of research to see what I should do about this and really like the idea here, however I don't know much about php. I can code html and css, but just use a free formmail.php type program to process the form. Could anybody tell me in super simple terms how to add a function to the php file so that forms are not processed if a certain field is not empty? I've found bits here and there, but haven't been able to adapt the php file with any success. Thanks!
Mark
Sunday 28th October 2007 | 12:01 PMWhen someone submits a form on your site to a PHP file (where form method is POST), the PHP file can access the values in the fields by using the $_POST variable.
Try this:
1. Make a field in your form, name it "test" or something.
input type="text" name="test"
2. In your PHP script, formmail.php in your case, output the value of "test".
echo "The value of test is: " . $_POST["test"];
3. If that seems to work, get rid replace the echo with an "if" statement.
if ($_POST["test"] != "") {
// send the mail
} else {
// don't send the mail
}
In that last step, I'm checking to make sure that "test" is empty. If it is, send the mail. Otherwise, don't send the mail. You have to insert the code that actually sends the mail within the first set of curly braces there.
Hey Mark.
You made a mistake.
In your code the send-block should be within the SECOND set of curly braces.
You used != instead of == ...
So long,
Danny
John
Monday 25th August 2008 | 10:41 PMThank you for making the effort to battle spam. I noticed by disabling all css using firebug the "Please leave the following field empty" field is visible, using page info shows: field id hidden value 676.
I think the best approach is what you have done, being active and frequently changing the code.
sam
Tuesday 27th January 2009 | 06:42 AMVery nice way of handling it indeed. I'm just using wordpress, so I can't really use this.. but interesting nonetheless.
David
Tuesday 28th July 2009 | 10:16 AMI am not sure that this particular picture is very good as there are at least four legitimate answers to the question "Who is this". Peter, Toby, Spider, Spider-man. Apparently, the answer Toby (should it have been Tobey that's the correct spelling) won't work. The pic was of Tobey MacGuirre playing Peter Parker dressed as Spider-man. Other pictures also expect you to answer in a dumb way, character not actor.
Elizabeth K. Barone
Thursday 13th August 2009 | 11:44 PMThought this was a great article - until I scrolled down to leave a comment and found a Captcha (Who is this? with a picture of Kermit and an input field) anyway. It's kind of hard to be creditable if you're not even using your own method.
...in response to this comment by Elizabeth K. Barone. Amazing how often someone will comment without reading the full article. Notice the updates 2 and 3 at the end:
Update 2: It appears honey pots are merely a temporary solution.
Update 3: At the time of writing we had this method running on this site, but by the time you read this we will be working on (or have already completed) implementing a more drastic solution.
The more drastic solution is the one you already used below.
Have a nice day.
kakooljay
Tuesday 29th September 2009 | 12:18 PMHow about this idea: If CAPTCHA is a burden on the user [which it is] & spam bots are smart enough to interpret CSS & look for hidden fields etc, why not make the honey pot field VISIBLE to users? Just include a note like: LEAVE THIS FIELD EMPTY UNLESS YOU ARE A BOT :)
That might solve the problem, at least until bots start learning NLP...
jek123
Tuesday 26th January 2010 | 11:14 PMI have php form. It is working but I want to reduce spam. How do I insert into it a capture?
Thanks in advance.
jon_s
Wednesday 7th April 2010 | 09:43 AMHi
I loved this method of stopping comment spam, I used an adapted version (just a different css method) and it worked brilliantly, so thanks for the short break in comment spam!
I'm interested to know what benefit your captcha has over a normal letters / numbers captcha?
Also, I got one captcha picture on your site, of a frog, or a toad, not sure which. Would that have worked if I had entered frog or toad?
Likewise, the picture I see below (on captcha) is a Penguin, however, it does say I don't have to be perfectly specific, so will it work if I enter 'bird'?
No, it did not work if I entered Bird as the answer, so I did have to be specific. I failed to prove I am human.
Sorry, I'm being intentionally pedantic because although the vast majority will get the captcha correct, I feel having the website visitor type something that is not spelled out to them next to the field they need to fill in can lead to mistakes and frustrated users.
...in response to this comment by jon_s. Hi Jon,
The honey pot stopped working for us a long time ago. It was good while it lasted though, so now we have this captcha system.
It's much more reliable (no spam since) and to answer your question, it only has to be as specific as I make it. The answers to all the captchas are stored in a config file, and I am able to specify multiple answers for each. The one of the kitten for example, will also work if you type 'cat'. Likewise for the clown fish the word 'fish' will also work. The word 'toad' never occurred to me but I will include it because it's valid - thanks.
Luckily the comment doesn't disappear from the field in the event of a wrong answer which I think is an important feature.
As for captchas with normal letters or numbers, these can actually be beaten (though not often) and they are also a pain to decipher sometimes. A picture of an animal on the other hand is easy to comprehend and impossible for a bot to break- unless it has some sophisticated image recognition software.
Hope that helps.
Trans
Monday 6th September 2010 | 08:00 PM...in response to this comment by Rodney. Best CAPTCHA ever!
Joshua Kehn
Wednesday 13th October 2010 | 04:04 AMExcellent article, it's a pity though that it had to go to the graveyard. Kept under wraps it might have lived longer, but then no one will have benefited. Love the new captcha system.
nimda
Tuesday 30th November 2010 | 02:46 AMWhat if you designed the honeypot field as a Captcha? Human visitors would never see it...bot visitors would either recognize it as a Captcha, and give up (which is good); or recognize it as a Captcha and attempt a solution...not realizing that ANY answer unmasks them as spammers (which is good).
Ravi
Friday 1st April 2011 | 06:12 PMVery Helpful article.
Thanks
JeevanSathi
Ravi
Friday 1st April 2011 | 06:17 PMVery Helpful article.
Thanks
JeevanSathi
Gary
Friday 9th September 2011 | 10:57 PMHi there, love the old honeypot captcha, been trying to implement this with absolutely no success due to my lack of coding no doubt!
In your article you suggested its 4 easy steps, but it stopped at no 3, I fear no 4 is the section I really need, Google for once isn't my friend on that either!
I have all the elements in place but just cannot get the submit button to check the field. If you have time could you let me know the code or where I place the code to prevent form submission if the field is not null?
I tried placing this above submit button with no joy:
if(!String.IsNullOrEmpty(Request.form["muffins"]))
IgnoreComment();
By the way, excellent new captcha too.
Rodney
Sunday 5th August 2007 | 10:17 PMI'm still impressed to see you working on it. Whether it wins in a long term or whether it's beaten soon, I still think it's admirable to have tried your own path, rather than just Google for the same tools everyone else uses.
I think the very best thing about RustyLime is the ability for people to fire off comments quickly and without having to go through captcha and other multiple staged approachs. This lets people say what they feel, when they feel it.