I was a robot four times out of five. This sheds a new light on my existence.
On a serious tone, apart from a cool showcase of technology, do we REALLY need this? CAPTCHAs are the bane of everyone's internet time. Sometimes it works, sometimes it doesn't, sometimes I have to write strange unicode letters, when it fails I have to go back and re-input everything in the form,... And more importantly, the persons it's supposed to prevent from abusing a service are not likely to be stopped by these childish "anti-bot measures".
More importantly than the technical issues, CAPTCHA completion is already big business. Where the common scripts are known to fail, malware developers just feed the CAPTCHAs over to call centres in Africa or somewhere else cheap.
Everyone wants CAPTCHAs that minimise the annoyance to end users, and that are hard for a machine to crack. But the above business model means that, as a service designer, you either need to accept some degree of CAPTCHA fail, or somehow not annoy your valid users whilst also make solving them take unprofitably long for a remote worker to do.
SEO/internet marketers who want to go create a bunch of spammy backlinks through user profiles / blog comments on forum sites and blogs (anything that allows account creation and user generated content) to target affiliate marketing sites they own have pretty great automated tools to do this that integrate with the commercial captcha solving services.
The captcha solving aspect is around $1 - $1.50 per 1000.
You can't really prevent this and it really doesn't matter how effective the captcha is because outsourcing the human required to solve it is so ridiculously cheap.
Isn't object recognition in a not-distorted environment relatively easy? Click-and-drag is something UI automation does all the time, so that's not the hard part, either.
A couple more thoughts:
This requires a huge, human-written database (or it requires an algorithm ... and, oh wait, we already decided that if an algorithm can do it, then it's not really worth of being a captcha)
Also, a few of the images, especially the coffee mug ones, look like they could be solved by selecting the whole image.
That's a good point, but we are a long way away from an automated general object recognition system. This discounts occasional outliers such as obvious objects which take up the whole page, or ones that are clearly visible against a gradient-less background (both of which can be pruned).
Currently, we are validating inputs from a human written database, which is continuously being expanded with new user inputs and new images. One image is used for validation, and the other is used for expanding the pool.
An attacker might not even need to use object recognition.
The 'noun' (beerbottle, golfcart) is always in the same place in the DOM (so, easily targetable as text) The urls for each image appear to be constant as well. I also discovered the element with the token and the structure of the POST requests to get the pair of images as well as to send data. It might be possible to just bruteforce it with a little scripting, to send random boxes based on the size of the images and when I get "success" back, I know to associate those boxes with those image urls.
I assume I can't simply replace the images with ones I want (that would be insane) but I might be able to just bruteforce it with calls to the api until I get images I know. I would suggest that rather than exposing the actual image urls, you serve them up with obfuscated urls (though I personally wouldn't even want them to be publically accessible anyway.)
Also, maybe add a bit of good old fashioned cruft to the images themselves, to throw off attempts at storing their hashes. Maybe rotate or flip them now and then, change the colors, add random lines, filters, etc (basically what happens with text captchas.)
You might consider using photos that have more than the target object in them. It is easy as heck for a computer to determine the bounds of an object in a photo, and increasingly easy for it to know what that object is. It's certainly no challenge to a bot to determine the bounds of the "coffeemug" when it is the only object in the photo.
Also, CAPTCHA of any sort just sucks. As a user, I only will bother with a site that uses CAPTCHA if it is providing something I absolutely must have. If it's something I may or may not use, and I'm presented with a CAPTCHA (of any kind), I leave.
1. Not going to work for those with sight impairments.
2. Not going to work on mobile (in current form).
3. Its not any harder to break in an automated way then most other CAPTCHA's
I hate CAPTCHA's. With a passion. I even write automated CAPTCHA decoders for fun http://wausita.com/captcha/
If there is one thing the world does not need its another CAPTCHA. If you have something to protect all the CAPTCHA's in the world are not going to stop those who want to exploit it. They can be automated or outsourced (easily and for peanuts). CAPTCHA's should only be considered suitable for keeping random spam bots off your site, and even then adding a honeypot field is a safer bet.
If you are being targeted a CAPTCHA is next to useless.
CAPTCHAs were broken a long time ago when spammers hired real people to enter their values instead of relying on bots. In short, testing to see if the visitor is a human is protecting against spamming tactics from 2008.
I drew the box in the middle of the picture, despite what actual photo is (75% width / height of the original photo), 2 out of 15 times of trial, I get passed.
I have a background in breaking and designing CAPTCHAs at Microsoft Research; how is this un-OCRable?
If you're finding the objects automatically, you can write an algorithm to do that. If you're doing it manually, your corpus isn't going to be big enough, and I can just pay people to enumerate them.
Plus, for all of the examples I see on your page, you can just find objects of interest, which is fairly trivial, and that's probably going to be what you're looking for.
You're right about certain images having easily distinguished objects-of-interest. In the future, running a routine to weed those easy ones out will stop any being solvable by bots. And also, as with any other captcha, paying other humans to validate themselves for you will always be an issue.
We validate users' inputs against a pool of manually-entered data. Because we collect a pair of annotations for each submission, one is used to verify the user, and the other is collected against an un-annotated image. As annotations are collected for new images, they are synthesized to expand the pool of annotated images.
What he said was that if you manually preparing the images then the number of different images will be low enough that he can pay other people to solve the complete set - a one off cost.
I think the site/service is pretty slick and well-made. It worked well for me -- I'd be curious to see what kinds of boxes these people failing were drawing (or whether it's just a browser issue for them or something).
These comments are all pretty negative, and I think the criticisms are mostly valid, but I don't think you've made a bad product (though it may need some tweaking, and captchas may be on their way out now for the reasons others have posted).
I just know that I've felt awful before when receiving similar comments to these others, and I would have liked someone to remind me: you made something pretty good, and it wasn't a stupid idea.
I like the idea but the whole thing about reCaptcha is that we are helping as well, and that is really compiling.. BUT maybe if u can add advertising pictures in the captcha that payoff the blogger/developer you can get more acceptance ..
For everyone who seems to be getting false negatives, it's because you're tracing a space too large.
I was initially getting robot returns, but it seems to have a higher tolerance for not binding the entire target, rather than trying to get the entire thing and having non-target space in the enclosure.
With that out of the way...this isn't safe. Interesting idea, but not safe. All that's needed is an algorithm that finds the object existing in both images. This is trivial with your current setup because size doesn't matter, recognizing two clocks is pretty easy even if one takes up a full picture and the other takes up a corner.
Some possibilities, and problems with each:
1. If you try a more rigorous system, where there's two pictures full of images in both, then you might have a better chance. But you'll still be giving text instructions to specify which image is the one you need to trace, and this can be automated as stated above. As soon as you state the word of the image, it's going to be algorithm-able, unless you make the image so blurry or incomprehensible as to fool both machines and humans, which defeats the purpose.
2. You could specify nothing, and instead tell the human to trace two matching objects. Regardless of whether or not there's more than one matching set, you can still automate this and pass with a machine.
I think this is an interesting, and perhaps capable idea, but it's far away from having any practical or secure utility.
Going from 10-14 seconds to 5-7 seconds per Turing Test isn't a 100% improvement, it's a 50% improvement. A 100% improvement would mean each run took 0 seconds.
It's a neat idea, and a good implementation at this point, but I don't like the idea of moving towards human-verification methods that won't work for the sight-impaired.
37 comments
[ 2.0 ms ] story [ 86.9 ms ] threadSeems like a good idea, but I'm pretty sure no one will ever user a captcha that repeatedly fails human input (even worse than reCAPTCHA).
On a serious tone, apart from a cool showcase of technology, do we REALLY need this? CAPTCHAs are the bane of everyone's internet time. Sometimes it works, sometimes it doesn't, sometimes I have to write strange unicode letters, when it fails I have to go back and re-input everything in the form,... And more importantly, the persons it's supposed to prevent from abusing a service are not likely to be stopped by these childish "anti-bot measures".
I tried a couple more, with slightly better success, however one of those times I saw the same image in both panels.
And then, it broke rather oddly: http://i.imgur.com/LwYtKe2.png (time approximately 00:45 BST, firefox 21.0, Win7)
Edit: I'm using Chrome 27.0.1453.110 under Ubuntu.
Everyone wants CAPTCHAs that minimise the annoyance to end users, and that are hard for a machine to crack. But the above business model means that, as a service designer, you either need to accept some degree of CAPTCHA fail, or somehow not annoy your valid users whilst also make solving them take unprofitably long for a remote worker to do.
SEO/internet marketers who want to go create a bunch of spammy backlinks through user profiles / blog comments on forum sites and blogs (anything that allows account creation and user generated content) to target affiliate marketing sites they own have pretty great automated tools to do this that integrate with the commercial captcha solving services.
The captcha solving aspect is around $1 - $1.50 per 1000.
You can't really prevent this and it really doesn't matter how effective the captcha is because outsourcing the human required to solve it is so ridiculously cheap.
(I'm using Chrome 29 on a Chromebook.)
A couple more thoughts: This requires a huge, human-written database (or it requires an algorithm ... and, oh wait, we already decided that if an algorithm can do it, then it's not really worth of being a captcha)
Also, a few of the images, especially the coffee mug ones, look like they could be solved by selecting the whole image.
Currently, we are validating inputs from a human written database, which is continuously being expanded with new user inputs and new images. One image is used for validation, and the other is used for expanding the pool.
The 'noun' (beerbottle, golfcart) is always in the same place in the DOM (so, easily targetable as text) The urls for each image appear to be constant as well. I also discovered the element with the token and the structure of the POST requests to get the pair of images as well as to send data. It might be possible to just bruteforce it with a little scripting, to send random boxes based on the size of the images and when I get "success" back, I know to associate those boxes with those image urls.
I assume I can't simply replace the images with ones I want (that would be insane) but I might be able to just bruteforce it with calls to the api until I get images I know. I would suggest that rather than exposing the actual image urls, you serve them up with obfuscated urls (though I personally wouldn't even want them to be publically accessible anyway.)
Also, maybe add a bit of good old fashioned cruft to the images themselves, to throw off attempts at storing their hashes. Maybe rotate or flip them now and then, change the colors, add random lines, filters, etc (basically what happens with text captchas.)
Just some random ideas.
Also, CAPTCHA of any sort just sucks. As a user, I only will bother with a site that uses CAPTCHA if it is providing something I absolutely must have. If it's something I may or may not use, and I'm presented with a CAPTCHA (of any kind), I leave.
1. Not going to work for those with sight impairments. 2. Not going to work on mobile (in current form). 3. Its not any harder to break in an automated way then most other CAPTCHA's
I hate CAPTCHA's. With a passion. I even write automated CAPTCHA decoders for fun http://wausita.com/captcha/
If there is one thing the world does not need its another CAPTCHA. If you have something to protect all the CAPTCHA's in the world are not going to stop those who want to exploit it. They can be automated or outsourced (easily and for peanuts). CAPTCHA's should only be considered suitable for keeping random spam bots off your site, and even then adding a honeypot field is a safer bet.
If you are being targeted a CAPTCHA is next to useless.
If you're finding the objects automatically, you can write an algorithm to do that. If you're doing it manually, your corpus isn't going to be big enough, and I can just pay people to enumerate them.
Plus, for all of the examples I see on your page, you can just find objects of interest, which is fairly trivial, and that's probably going to be what you're looking for.
You're right about certain images having easily distinguished objects-of-interest. In the future, running a routine to weed those easy ones out will stop any being solvable by bots. And also, as with any other captcha, paying other humans to validate themselves for you will always be an issue.
We validate users' inputs against a pool of manually-entered data. Because we collect a pair of annotations for each submission, one is used to verify the user, and the other is collected against an un-annotated image. As annotations are collected for new images, they are synthesized to expand the pool of annotated images.
These comments are all pretty negative, and I think the criticisms are mostly valid, but I don't think you've made a bad product (though it may need some tweaking, and captchas may be on their way out now for the reasons others have posted).
I just know that I've felt awful before when receiving similar comments to these others, and I would have liked someone to remind me: you made something pretty good, and it wasn't a stupid idea.
Why is it that computers can drive cars on a highway, but they can't discern a human from a Nigerian spam bot?
The whole necessity should be embarrassing to programmers w/ access to a powerful server.
If they put CAPTCHA's on the Xcode Build command, the whole problem would be solved within the week.
Eat your own dogfood, developers.
I was initially getting robot returns, but it seems to have a higher tolerance for not binding the entire target, rather than trying to get the entire thing and having non-target space in the enclosure.
With that out of the way...this isn't safe. Interesting idea, but not safe. All that's needed is an algorithm that finds the object existing in both images. This is trivial with your current setup because size doesn't matter, recognizing two clocks is pretty easy even if one takes up a full picture and the other takes up a corner.
Some possibilities, and problems with each:
1. If you try a more rigorous system, where there's two pictures full of images in both, then you might have a better chance. But you'll still be giving text instructions to specify which image is the one you need to trace, and this can be automated as stated above. As soon as you state the word of the image, it's going to be algorithm-able, unless you make the image so blurry or incomprehensible as to fool both machines and humans, which defeats the purpose.
2. You could specify nothing, and instead tell the human to trace two matching objects. Regardless of whether or not there's more than one matching set, you can still automate this and pass with a machine.
I think this is an interesting, and perhaps capable idea, but it's far away from having any practical or secure utility.
Going from 10-14 seconds to 5-7 seconds per Turing Test isn't a 100% improvement, it's a 50% improvement. A 100% improvement would mean each run took 0 seconds.