Show HN: Code to run Gemini (Nano) locally on desktop/Chrome (github.com)

22 points by notnotrishi 2y ago ↗ HN

Chrome Canary (nightly build for devs) now has Gemini LLM inbuilt. This is just some simple code/demo to enable and use that feature.

5 comments

[ 3.0 ms ] story [ 22.7 ms ] thread

DownrightNifty 2y ago ↗

Instructions didn't work for me. Tried on Windows 10 and macOS 14.3 (x86) with Chrome Canary. The "On Device Model" component doesn't show up.

notnotrishi 2y ago ↗

yea, it's a bit tricky. this came in from google team today:

> To get back on track, make sure you're using Chrome version 128.0.6545.0 or later. If you want the most up-to-date version, try using Chrome Canary or Chrome dev channel.

this document should have the latest changes from Google: https://docs.google.com/document/d/1VG8HIyz361zGduWgNG7R_R8X...

samspenc 2y ago ↗

Huh, looking at the code, it's simpler than I thought, basically these are the 2 lines that enable the request-response functionality:

  const session = await window.ai.createTextSession();
  const response = await session.prompt(prompt);

notnotrishi 2y ago ↗

exactly! also, one can set parameters etc. if they'd like.

also, after this post, i was able to do some tinkering and run it outside chrome, which i think would be more useful. hope they release the weights and make it open

DownrightNifty 2y ago ↗

You should share this with us! If you have a blog, that'd be a really interesting post.