If I want to run LLMs model like Qwen or gpt locally how much would it cost me and I want to connect it to my main website creating an API link, also which models would be best
First, check what LLMs your system can actually handle based on your RAM/VRAM, then choose the model variant accordingly.
If you want to use the model through an API for your website, a good setup would be a dedicated machine/server to host it, since running LLMs locally can consume a lot of memory.
Depending on your hardware, you can look at smaller variants of Gemma or Qwen Coder.
Another option is to use a hosted inference provider. It’ll cost a little, but you’ll usually get much faster inference compared to running locally especially if your system isn’t high-end.
5 comments
[ 3.6 ms ] story [ 18.4 ms ] threadIf you want to use the model through an API for your website, a good setup would be a dedicated machine/server to host it, since running LLMs locally can consume a lot of memory.
Depending on your hardware, you can look at smaller variants of Gemma or Qwen Coder.
Another option is to use a hosted inference provider. It’ll cost a little, but you’ll usually get much faster inference compared to running locally especially if your system isn’t high-end.