HN Explorer

Ask HN: What are other "uncrackable benchmarks"?

1 points by Nail2680 ↗ HN
I have my own benchmark for AI just based off my Doctoral Thesis, which isn't public(university constraints). I am wondering if other people have other "hidden" benchmarks? I have a few I have made, and one wonder if talking about them devalues them, and two can a good test set be made that can't be leaked?

1 comment

[ 0.25 ms ] story [ 8.0 ms ] thread
retrac
Effective natural language processing of sign languages. Current LLMs are almost completely incapable. And it's a steep hill to climb. No text corpus. Must be learned from video. I do not believe current LLM approaches are capable of this with the amount of training material available, regardless of compute. Though I'm eager to be shown wrong.