polishgladiator
No user record in our sample, but polishgladiator has activity below (stories or comments). Likely we have partial data — the full bulk-load will fill profiles in.
No user record in our sample, but polishgladiator has activity below (stories or comments). Likely we have partial data — the full bulk-load will fill profiles in.
Something doesn't smell right. Such sloppy errors with measurement and comparison (from people who are supposedly experts?), and cageyness about answering technical questions, reminds me of the era of crypto currency…
Actually no -- that post shows they are not performing measurements and comparisons correctly. These are not serious people.
OK, so this is a case of bad measurement and comparison. If you bothered to look at the llama.cpp output, you would see this line: llama_model_load_internal: offloaded 32/35 layers to GPU The "-ngl 32" means that only…
> [...] llama.cpp is a fantastic framework to run models locally for the single-user case (batch=1) > [...] I don't think it would be particularly fair to compare and show that MKML is better at a given perplexity,…
> If anyone has specific technical questions I'd be happy to answer as best I can. What is the context size for these measurements? Is it the full 4k for llama-2? And just to be clear, when you say memory footprint,…
Based on the integration examples, I don't think they are simply repackaging llama.cpp Rather it looks like they are reimplementing their own quantization scheme, in such a way that it is a little easier to integrate…
I've been doing some hacking with Llama2 on an AMD 7900 XTX this weekend, using llama.cpp and q5_k_s quantization. Compared to MK600 on an RTX 4090 in their data, I am measuring higher throughput and lower perplexity…
In the realm of text editors, Vim has been my unwavering companion for 25 years. It has become an integral part of my daily existence, an inseparable bond that defies the imagination of life without its embrace.…