HN Explorer

lieret

↗ HN profile [ 42.9 ms ] full profile
Karma
0
Created
()
Submissions
0
  1. Show HN: Benchmark: AI doesn't find bugs unless you tell it what's wrong (swesweep.com)

    New benchmark from researchers at Meta, Stanford, Harvard, UW, including the researchers who've worked on SWE-bench, ProgramBench etc. Most benchmarks just test if AI can fix a problem you've already pointed out. But…

    2 points 2
  2. Show HN: New Benchmark from SWE-bench team is 0% solved (programbench.com)
    24 points 3
  3. Show HN: All the LM solutions on SWE-bench are bloated compared to humans (twitter.com)
    1 points 0
  4. Show HN: New eval from SWE-bench team evalutes LMs based on goals not tickets (codeclash.ai)

    Current evals test LMs on tasks: "fix this bug," "write a test" But we code to achieve goals: maximize revenue, cut costs, win users Meet CodeClash: LMs compete via their codebases across multi-round tournaments to…

    5 points 1
  5. Show HN: Randomly switching between LMs at every step boosts SWE-bench score (swebench.com)

    What if your agent uses a different LM at every turn? We let mini-SWE-agent randomly switch between GPT-5 and Sonnet 4 and it scored higher on SWE-bench than with either model separately. GPT-5 by itself gets 65.0%,…

    5 points 1
  6. GPT-5 on SWE-bench: Cost and performance deep-dive (mini-swe-agent.com)
    4 points 3
  7. Show HN: New SWE-bench leaderboard compares LMs without fancy agent scaffolds (swebench.com)

    Hello from the SWE-bench/SWE-agent team at Princeton/Stanford. When we created the SWE-bench benchmark in 2023 from hundreds of real-life GitHub issues/pull requests, the highest score was just a couple of percent. The…

    2 points 0
  8. Show HN: Mini-swe-agent achieves 65% on SWE-bench in 100 lines of python (github.com)
    7 points 2