HN Explorer

Ask HN: Has Anyone Successfully Run DeepSeek-R1 on an Nvidia Jetson Nano?

2 points by florianbansac ↗ HN
Hey HN,

I am curious if anyone has successfully run Deepseek-R1 on a Nvidia Jetson Nano, especially for programmatic use.

If so, did it work well?

Any tips on what to do, what to avoid for it to run smoothly?

Thanks

3 comments

[ 2.8 ms ] story [ 14.5 ms ] thread
DrPhish
The Nano only has 4GB VRAM and DS-R1 is 671B FP8 parameters (equivalent to 671GB model size).

You need something with about 800GB to run the full model with context. You'd still need 400GB to even run a half-sized Q4 quant of R1, so there is no reasonable way that it would work.

florianbansac
OK, I understand the flagship model is huge, It seems to be far from local use.

Anyone did it with a smaller/distilled version, and getting good performance?

n0vella
Just curious, what it's the cheapest card do you think that would be needed to run this model or something like llama 3.3-70B?

Only nvidia cards are compatible or AMD ones also could work?