Curious how the top performers compare to SOTA paid models. Also interested in understanding performance differences between running model on VPS hardware compared to a laptop. Thanks for sharing
> Instead of fixing the rate limiter that was blocking its own tests, it patched the environment detection. That's enterprise development in a nutshell.
That part made me laugh and reflects my experience when I was working on "enterprise development" teams.
I'm curious about having this run on a VPS as opposed to a local VM. What does that provide you? I understand having the VM completely disconnected from your local network etc but is there anything besides that? I ask because mentioning the cost here seemed like one of the important points but that cost wasn't necessary for the experiment itself.
6 comments
[ 1.6 ms ] story [ 16.7 ms ] threadThat part made me laugh and reflects my experience when I was working on "enterprise development" teams.
I'm curious about having this run on a VPS as opposed to a local VM. What does that provide you? I understand having the VM completely disconnected from your local network etc but is there anything besides that? I ask because mentioning the cost here seemed like one of the important points but that cost wasn't necessary for the experiment itself.
What were the models running on? It doesn’t matter a ton but would help paint the whole picture better.