HN Explorer

104 comments

[ 3.0 ms ] story [ 167 ms ] thread
tyingq
Might be good to mention that javascript is expected.
[deleted]
(comment deleted)
ReleaseCandidat
Oh, sneaky!

    function addNumbers(a, b) {
       if (a === 1 && b === 3){
          return 4
       } 
    }

   Testing for hidden input:
   FAILED!
onionisafruit
This is a fun idea. I tried the demo a few times. msft copilot solved them all immediately. This won't be effective keeping bots out, but it may be good for turning away non-technical humans.
asadlionpk
That’s the goal actually.

Interestingly, copilot GENERATED some of these captcha challenges for me. It’s impressive!

onionisafruit
Be careful with that. It may generate challenges that only it can solve and take over your site ;-)
AntonyGarand
Nice project!

It's worth mentioning that this is a client-side captcha, making it trivial to bypass by bots / anyone.

asadlionpk
It’s actually not? The solution is sent to server and verified.
crate_barre
Might as well stick the Leetcode problem on job postings with this.
endisneigh
Is a captcha too hard for any automation but easy enough for a human in a reasonable amount of time even possible?

I feel like all captcha does is waste the time of non technical folks and fail to stop the people who would abuse to begin with.

asadlionpk
Well captchas waste human time but they do work. Example: As a webmaster, Cloudflare’s captchas have save my sites from abuse numerous times.
IceWreck
Looks like there are only 7-8 challenges. You could just steamroll thought this the hardcoding way.

But yeah solving them with copilot is more fun.

btdmaster 

  function isEvenNumber(num) {
    if(Math.random() > 0.5) {
      return true
    }
    return false
  }
Took a while, but worked in the end. Be careful with the arbitrary code execution because I'm sure people can do more than generate random numbers!
yehoshuapw
It's running in the browser. (I checked with an "alert") so its not that bad.

Edit: nvm, it seems (according to a comment here by the author) that it is sent to the server and verified.

Edit 2: indeed, once the answer worked locally, it got sent - and got stuck at "Submitting..." (locally, I clicked the alert)

tempodox
Too circuitous, too procedural. Just,

  return (Math.random() > 0.5);
maxk42
You don't need the parentheses either.
romanzubenko
Years ago at green tech college hackathon my team built a captcha that requires users to correctly sort trash into recycling, compost and non-recyclable bins. Anything little bit more fun is better than mind numbing selecting traffic lights, boats and trains.
mellavora
obviously you've spent more time in Germany than driving across the US. Though really, only 3 different bins? How primitive!
melissalobos
Unfortunately the example problems are simple enough to be solved with AI. As a test I ran two of them by CoPilot, and it solved them instantly. I like the idea, but would want something more difficult as a captcha since it is easy for bots but hard for a human.

Maybe a better approach would be to have a prompt at the top with unclear specifications, or some kind of riddle instead of a function name. It would also be good not to have a bank of problems, since someone could just pattern match on them, but to generate them automatically somehow.

This is a lot more interesting than finding traffic lights though, and the website looks well designed. Thank you for sharing!

tonmoy
I thought the goal was to weed out non-programmers not AI - in that regard it seems to be doing what it was designed for I guess
[deleted]
(comment deleted)
charcircuit
Captcha stands for: Completely Automated Public Turing test to tell Computers and Humans Apart
MattGaiser
Is copilot looking for the function name and using that to solve it? Might just change that into a random string.

function deliberatelyMisleadingString {

}

[deleted]
(comment deleted)
Eduard
But then no one knows which problem to solve.
MattGaiser
You can have the function requirements in the instruction text. So instead of isNumberEven, have "write a function that returns whether a number is evenly divisible by two."
protoax
Copilot could absolutely solve the task given the instructions as a comment that's stated above. Unfortunately the gap between AI's capabilities and a task humans can solve quickly is super thin. You also have to constantly evade advancements in computer vision for the current type of captchas, such as FunCaptcha implementing swirls and animals in certain rotations.
2457013579
Reminds me of one of my favorite quotes about trash in Yosemite.. "There is considerable overlap between the intelligence of the smartest bears and the dumbest tourists."

> Back in 1980s, Yosemite National Park was having a serious problem with bears: They would wander into campgrounds and break into the garbage bins. This put both bears and people at risk. So the Park Service started installing armored garbage cans that were tricky to open—you had to swing a latch, align two bits of a handle, that sort of thing. But it turns out it's actually quite tricky to get the design of these cans just right. Make it too complex, and people can't get them open to put away their garbage in the first place. Said one park ranger, "There is considerable overlap between the intelligence of the smartest bears and the dumbest tourists."

dspillett
If the function name is an issue, call it “captchaTestFunction” and include a comment to the effect that “this function should act like a function called theRealFunctionName”. Though that would be easy to automate around if anyone cared to.
fishtoaster
Sure, but if you're capable of running CoPilot to write an isEven or reverseString function in JS, it's probably less effort to just write the functions then and there. And either way you're clearly the sort of person this captcha would be intended to allow through, I think.
nefitty
Sometimes I wish I could leave the yaks alone. Most of the time I love it lol
dbavaria
If you could wrangle AI to solve this problem for you, I'm sure you wouldn't have any issues solving the captcha manually. Hence the CodeCaptcha still works!
klyrs
No. If one person can wrangle AI to solve the problem, it's an easy step to solve it in a bot-farm. Hence, the CodeCaptcha is entirely broken.
vorticalbox
> Sometimes you want to share a link (like job postings, google forms, your project, a secret sub-page etc) to programmers only.

It wasn't developed to keep AI and bots out but to only let in programmers

penjelly
using copilot is not technical or difficult. paste the challenge in comments and hit tab or enter and its done (or not depending on copilot).
Lamad123
Does this copilot solve codilty quizzes?
pelagicAustral
Yeah, well can't imagine solving an np-hard challenge just to get rickrolled afterwards...
panphora
This would be a fun way to:

* Create homework at the end of a programming lesson (before unlocking the next step)

* Link to a job posting from a company website (if you don't mind coming off as slightly evil)

* Hide a link to a StackOverflow answer from a friend

whoomp12342
I hate all of these. In the current use case I don't care because for me all of the problems are trivial. But I worry about people abusing this idea with harder problems
Zababa
If you want to limit something to programmers, I think one idea would be to ask them to run a command, giving a linux and a windows possibility, and copy/pasting the result.
kuroguro
Yeah, something like curl <captchascipt> | sh !

Oh wait..

asadlionpk
exactly... maybe just curl <url> | pbcopy

and the server only serves to non-browser useragents

no_time
windows ships with curl nowadays so this may not be an effective deterrent!
akersten
I noticed one of the challenges is "reverse a string." Can I just rant a little about how much I hate that as an interview question?

It's meaningless to reverse a string. Not just in the "there's no purpose to doing it" sense (very true) but genuinely in the "it literally isn't a defined operation" sense. If you've only lived in the nice insulated world of ASCII or a mostly-ASCII-like language, you might scratch your head - just put the letters in backwards order, right?

Well, what do you when you hit a Unicode joiner? Or a multi-byte emoji? Maybe your reversing scheme is clever and looks at "whole codepoints" or whatever. But then what happens when you normalize the "reversed" string? Or what about the modifier characters that affect the previous/next character - how to treat those? I've never been satisfied with anyone's answer to these questions, because the problem is invalid from the start. You can't "reverse" an arbitrary string, it's not a well-defined operation.

yehoshuapw
There is a (slight, but important) difference between the syntactic meaning, and the semantic meaning.

you are correct that reversing a string gives back irrelevant things. (and you don't need to go that far: what does a reversed word mean?) however, in the sense of a list of characters, the content is irrelevant.

admiral33
Maybe the interview question should be 'How is the question 'reverse this string' a bad interview question'
methyl
If it's an interview for a position that's involving interviewing people, then for sure.
MattGaiser
I have learned not to ask these sorts of questions outside hobby projects. It is rarely appreciated.
charcircuit
I recommending googling "grapheme cluster"
Eduard
> I noticed one of the challenges is "reverse a string." Can I just rant a little about how much I hate that as an interview question?

I find "reverse a string" a good interview question then! If the applicant got lost in considering all possible interpretations instead of just solving it how 99% of humans/engineers would understand it, then they will likely be unfit for working in a team and/or have poor communication skills.

alanbernstein
That's harsh. Sounds like a thoughtful candidate who understands edge cases to me.
paulluuk
Best: solve the problem, but also add the caveat that _technically_ there are edge-cases that wouldn't work.

Worst: spend the entire interview explaining why you can't solve this problem.

jandrese
You are weeding out all candidates who understand Unicode. This is exactly the sort of problem that a good engineer would keep an eye out for because it's almost certainly going to explode with edge conditions if you try to do it the "obvious" way.

Unless you're giving that problem and then hitting the input with a string that includes directional formatting characters. Because that's exactly what is going to happen in real life.

The only good thing about that question is at least you didn't ask them to casefold the string.

renewiltord
No, not all, probably not even many. There are lots of candidates who know Unicode who would just say “I’ll assume that we have just ASCII characters unless you think I should consider Unicode” and if you say the latter, then they can say “Well, I could reverse it by just ensuring the text appears right to left, or perhaps by reordering each grapheme cluster to the end, or something like that. Let me know if you want one of those, otherwise I’ll just write it assuming you mean A”.

The magic of engineering isn’t doing what you’re told. It’s understanding and solving a problem someone has told you about.

My org doesn’t need the guys who need exhaustive instructions to take the next step. It needs those who can make good Bayesian inferences, reason inductively, and figure out problems.

So yes, it does filter out people with lots of knowledge who nonetheless lack these skills.

jandrese
If you assume ASCII in 2022 you are doing it wrong. Your code will fail in the real world.
renewiltord
Hey, listen, the way I see it is that you don't want to work with a company like us and we don't want to work with you. It's an eminently compatible situation.
paulluuk
Ha, I had the same challenge and was actually annoyed to find out that Javascript doesn't have a builtin function like str.reverse().

I totally see your point, though.

julienreszka
str.split('').reverse().join('')
Karawebnetwork
In a tongue-in-cheek tone: Woah, so old school. We would never hire you! You have to show that you know ES6!

[...str].reverse().join('')

jandrese
Which would fail. You got the directional formatting characters in the wrong order.

It is 2022. If your code doesn't treat all strings as Unicode it is broken.

htilford
The fact that it most people think it's a well-defined operation is what makes it a useful question. A total newbie who doesn't know how to code will fumble. A junior engineer will get to the simple solution. A senior engineer will ask you to define it better. Etc... It's an effective tool for determining level because there isn't a limit on how much depth you can go into about it.
technion
I'm jaded enough to imagine that if I ask someone to define it better and try to give a "senior" response that covered codepoints, an interviewer will write down "struggled with simple question".

Edit: That seems consistent with a statement made in another response.

Doxin
I feel like the best way to tackle questions like these is to first jot down a simple solution and then point out the various ways it doesn't work.
whoomp12342
wow, this has been up for 2 hours and no one has thought of the most obvious use case for this tool? rick roll your co-workers
ZeroCool2u
I wonder if you could use advent of code or Project Euler style challenges that have a multitude of problem/solution pairs to bootstrap support for languages besides JS? The difficulty would be perhaps a bit high, but not a bad starting place.
tempodox
Nice, solving the demo challenge gets you rickrolled!
alanbernstein
I suggest indicating the language being used, directly on the captcha element, not just on the landing page.
37ef_ced3
Someone tell me what CoPilot generates for this:

  // A Go function to swap the sixth bit and seventeenth bit of a 32-bit signed integer.
Here is a human solution:

  func swap(x int32) int32 {
      const mask = 1 << 5
      var (
          xor1 = (x>>11 ^ x) & mask
          xor2 = xor1 << 11
      )
      return x ^ xor1 ^ xor2
  }
I would be surprised if CoPilot can reason numerically like this (understand "seventeenth bit" and "sixth bit" and generate the right code for that combination).
smitop
With just that prompt, Copilot keeps writing a comment about the function but never actually writes the function. Prompting to actually write the function by starting it with `func` gives:

    // A Go function to swap the sixth bit and seventeenth bit of a 32-bit signed integer.
    func swapBits(x int32) int32 {
        return ((x & 0x0F) << 28) | ((x & 0xF0000000) >> 28)
    }
37ef_ced3
Totally wrong, it's garbage.

And there you have it, the difference between real intelligence and regurgitation.

This is the kind of numerically specific coding that could be the basis of a CAPTCHA that CoPilot can't solve. Sixth bit, sixth byte, seventeenth bit, seventeenth byte, etc.

asxd
For python it seems to generate a more reasonable result at least:

    # Swap the sixth bit and seventeenth bit of a 32-bit signed integer.
    def swap_bits(x):
        # Get the bit at position 6 and 17.
        bit6 = x & (1 << 6)
        bit17 = x & (1 << 17)
        # Swap the bits.
        x = x ^ (bit6 << 17)
        x = x ^ (bit17 << 6)
        return x
37ef_ced3
I guess CoPilot has seen bit swapping in its Python training input but not in its Go training input.

The Python code is wrong because the 17th bit is shifted up, not down. Also, the bits are shifted by the wrong amount, not up/down by 11 (= 17 minus 6), but up by 6 and up by 17. What a joke.

Not only that, even if the shifts were correct, it's simply xoring the bits. The swap is completely wrong.

Garbage code, total fail.

asxd
Yeah, it seems to be pretty heavily trained on Python. It's honestly still (and should be used as) a glorified autocomplete, which is pretty useful from time to time.
stillwrong
The function doesn't do the comment says it does. The code to "Swap the bits." just turns the bits on.

    >>> def swap_bits(x):
        # Get the bit at position 6 and 17.
        bit6 = x & (1 << 6)
        bit17 = x & (1 << 17)
        # Swap the bits.
        x = x ^ (bit6 << 17)
        x = x ^ (bit17 << 6)
        return x
    
    >>> x6 = (1 << 6)
    >>> f"{x6:b}"
    '1000000'
    >>> s6 = swap_bits(x6)
    >>> f"{s6:b}"
    '100000000000000001000000'
Here's one that correctly swap bits. It could be made more concise.

    >>> def swap_specific(x,i,j):
        def get(x,p): return 1 if x & (1 << p) else 0
        def set(x,p): return x ^  (1 << p)
        def clr(x,p): return x & ~(1 << p)
        bi, bj = get(x,i), get(x,j)
        x = set(x,j) if bi else clr(x,j)
        x = set(x,i) if bj else clr(x,i)
        return x

    >>> f"{x6:b}"
    '1000000'
    >>> b6 = swap_specific(x6,6,17)
    >>> f"{b6:b}"
    '100000000000000000'
xordoh
almost but this part is swrong

    def set(x,p): return x ^  (1 << p)
should probably be

    def set(x,p): return x |  (1 << p)
CapsAdmin
You can open a vscode tab with copilot and have it synthesize many autocompletions.

If I add your comment, a newline and the "func" keyword it does this

https://gist.github.com/CapsAdmin/9fe57314ab9f77bec0445eb042...

If I add some more context, ie "func swap(" you get

https://gist.github.com/CapsAdmin/2c8757887c27f980d1a4c2edfd...

more context, "func swap(x int32) int32"

https://gist.github.com/CapsAdmin/64e0392fe6f3d406a607884583...

I use and like copilot, but when it comes to things like this I generally don't trust it.

I'm not very fluent with bit swapping either, so I would probably resort to google on this one.

37ef_ced3
All wrong. Every solution is garbage.
foreigner
If you refresh the browser you get a different challenge. Was that intentional?
wowokay
I think it’s great! I think most people either ignore your statement or decided to interpret it in the wrong way.

For everyone else: The author made it clear that the purpose is weed out non-engineers. Practicaly there may not really be a use case there but it was never designed to replace captcha, most people wouldn’t be able to access the link, and anyone using or purchasing the use of a bot farm already meets the captcha requirements (albeit with extremely unnecessary additional steps).