Show HN: Needle2: 14MB agentic LLM for phones, wearables, smart home and robots (cactuscompute.com)

Query: HN

Result:

{ "function_calls": [ { "name": "lock_door", "arguments": { "door": "front door" } } ], "reasoning": "User wants to lock the door. No specific door mentioned, so use 'front door' as default.", "confidence": 0 }

I'd expect it to at least ignore (call no tools) for the queries that it doesn't understand. And it seems like it does do that, just not consistently.

The demo is super — I'm just having trouble seeing the whole picture for e.g. a screenless device.

ETA: pun not intended

Given its basically an O(1) lookup with disk space being the main constraint, I was curious if you've tried ablating engram layers and sizes across your setup?

Also, why mHC over attention residuals?

With that being said, the web demo is not particularly impressive. It really doesn't like anything I throw at it. I'm fine with accepting that fine-tuning is the solution to this, but I wonder if there's anything to gain from a bigger model? I know it's completely counter to the whole point of this, but a 14MB binary using 28MB of RAM seems unnecessarily small and pretty arbitrary.

Like, what does a 28MB binary get you? Or a 140MB binary? Or a 1.4MB binary? I'm guessing the choice of 14MB came from minimizing the size as much as possible while meeting certain requirements/performance expectations, but even a Pi 5 has plenty more room to spare. Curious if there's a good explanation for this (which I may have missed in my skim of the post).

While most of the industry focuses on the frontier of “intelligence” (function), a release like this represents the frontier of the other end of the spectrum (form).

Both are important if we ever want to see “Opus-level” capability running locally on commodity machines in the future.

Edit: I have a pile of d1 minis, but not much time.

  import needle

  @needle.tool
  def add(a: int, b: int):
      "Add two numbers."
      return a + b

  agent = needle.Needle(tools=[add])
  print(agent.run("calculate 1 + 1?")["reasoning"])

conclusion: completly useless

Do the creators take something like DeepSeek, and then delete most of the neurons to whittle down the size?

  {
    "function_calls": [
      {
        "name": "set_thermostat",
        "arguments": {
          "temperature": 72,
          "mode": "cool",
          "room": "living room"
        }
      }
    ],
    "reasoning": "'as dark as possible' -> set_thermostat to warm; 'dark' implies higher temperature; 'cool' mode for darkness.",
    "confidence": 0
  }

Since it seems limited to matching a few templates and otherwise falling flat on its face, I wonder how 14MB of regexes would fare in its stead. Normally you wouldn't want to parse arbitrary natural language input with regex because of how tedious and brittle it would be, but for the tedium we have LLMs and this alternative isn't exactly robust either.

Please, though, take a pass at humanizing the text on the page. It's Clauded up all over and makes it hard to read.

{ "function_calls": [ { "name": "lock_door", "arguments": { "door": "tv" } } ], "confidence": 0.0158 }

Very interesting, seems confidence is 0 when tool calls are right?

> Make it a little warmer in here.

The reply:

> "name": "set_thermostat", > "arguments": { > "temperature": 65, > "mode": "cool", > ... > "reasoning": "'warmer' implies need for cooling; set_thermostat with temperature 65 (typical warmth) and mode 'cool'.",

Maybe I'm doing it wrong?