HN Explorer

Pay per pre-configured GPU hour or per token for open-source LLM inference?

1 points by yiro13 ↗ HN
Is it worth paying for a pre-configured GPU for open-source LLM inference instead of using a traditional pay-per-token provider? The biggest advantage seems to be having no usage limits

3 comments

[ 3.1 ms ] story [ 16.3 ms ] thread
tensward
Depends on your use case, overall most of the time it's not worth it, regular pay per token are heavily subsidised right now and hard to match, but for non coding use cases some models work great and you can host them yourself, but again it require a but more of a hustle