My take as well. The dual-3090 and dual-spark configs have an outsized voice in this conversation. I'd add pipeline parallelism vs tensor parallelism and in the case of TP setting your lanes and speeds per card.
I might have said the same thing in 2023, but any coding CLI installed on Linux can rip through this work and keep it updated.
Most popular would be Qwen 9b or Qwen 35b with experts offloaded to CPU. Even these aren't great, but they can be run at higher quants on lower hardware. We'll see what Qwen releases version 4 in the coming weeks.
Influencers have latched onto the pitch that everyday people can run frontier models on an 8gb GPU while sticking it to the labs. There's a large audience of people who haven't had the hardware to test larger quants to…
Sure, like in the case of a rear/front plate switch on a stolen car, Flock has every bit of information to identify that this is the case before deploying an officer to perform a pit maneuver on an innocent citizen. But…
The article takes a hand-wavy attitude towards abuse and negligence, with little attempt to counter-balance it with transparent accountability. If we can't address that in good faith, I see no reason to give the other…
lazypi has a sensible default collection of plugins and configs to get started
While I know it's not apples to apples, the target comparison right now is 2x DGX Sparks. Similar price, 256gb. The conversation has focused on memory bandwidth vs. compute in agentic loops, so for most people the raw…
A lot of people just want to play "Smash Bros" on their iPad or get native ulatrawide support on "Majora's Mask". This might be a product vs. eng split, but "whole point of the endeavor" isn't the same for everyone.
Can you share what tools and processes you're using to do this? I've been using Pi to build custom extensions and wrapping workflows in shell processes to make it more deterministic and enforce certain validations, all…
It's exciting, but that's $9k for a model that's in the ballpark of last year's frontier. For many people that's like 10 years of up-front cost in a landscape that's getting better and cheaper every week. but compared…
There's great VST plugins like "Sketch Cassette 2" that will do hiss, flutter, all of that, along with some great presets. That one's less about creating authentic sound profiles, more like creating lo-fi tracks and…
If llama 3 70b were available for $400 today, people would make it work. They would have 4 of them working side by side or 1 of them working with a GPU on another model. To me, it's like imagining if Sonnet 3 was burned…
The barrier to entry for tinkering on Linux is so much lower now for anyone with a $20 Claude/Codex plan. What might have been an hours/days of fruitless head-scratching can now be done in minutes while you grab a…
In Bazzite I've had innumerable issues with waking from sleep and display driver crashes, which means a 50 ft walk to press the reset button. If you're cool with that, this appliance isn't for you.
I bought an irresponsible pile of homelab equipment in 24/25. Hard drives, SSD, memory, GPUs. I feel bad for people locked out right now, since it's become more interesting and important than ever. At the time it seemed…
To go further down this pipe dream - Anthropic / OpenAI would buy them all and still price out the consumer. There's no end-run in this scenario.
I can actually use and enjoy Linux. The "year of the desktop" never came for me, but instead I got the "year of the cli". For 20 years I've used Linux in one form or another, but I've felt like I was kneecapped for the…
It's fine for dense models where you need them in VRAM, less so for MoE where you're offloading layers to ram. But 32/32 is pretty good for both in the popular ~30b range right now.
- You can send any amount of money to anyone in the world very quickly and cheaply, and nobody can stop you. - No government can dilute it or limit its supply. Stuff like that. Maybe that matters to you, maybe not, but…
This is where a lot of the crypto scammers on x have moved, selling ai/bots to do this. It seems like the odds slip significantly after these large bets are filled.
For the huge percentage of devs using vscode, switching to Cursor was essentially adding a new color theme and a chat window. The CLI switch was far more radical.
Thank you for the thoughtful reply. I have some smaller 3080's I'm looking to place and this sounds like a good opportunity.
The memory upgrade is $1k on a Macbook Pro. The laptop is ~$5500.
Expect to pay $4k-10k - Your RTX 6000 is closer to $10k now - Sparks are creeping into the $4-5k range - AMD Strix are ~3.5k - Apple depends on chipset and memory. Sweet spot would be 128gb M3 Ultra, probably $6-8k but…
My take as well. The dual-3090 and dual-spark configs have an outsized voice in this conversation. I'd add pipeline parallelism vs tensor parallelism and in the case of TP setting your lanes and speeds per card.
I might have said the same thing in 2023, but any coding CLI installed on Linux can rip through this work and keep it updated.
Most popular would be Qwen 9b or Qwen 35b with experts offloaded to CPU. Even these aren't great, but they can be run at higher quants on lower hardware. We'll see what Qwen releases version 4 in the coming weeks.
Influencers have latched onto the pitch that everyday people can run frontier models on an 8gb GPU while sticking it to the labs. There's a large audience of people who haven't had the hardware to test larger quants to…
Sure, like in the case of a rear/front plate switch on a stolen car, Flock has every bit of information to identify that this is the case before deploying an officer to perform a pit maneuver on an innocent citizen. But…
The article takes a hand-wavy attitude towards abuse and negligence, with little attempt to counter-balance it with transparent accountability. If we can't address that in good faith, I see no reason to give the other…
lazypi has a sensible default collection of plugins and configs to get started
While I know it's not apples to apples, the target comparison right now is 2x DGX Sparks. Similar price, 256gb. The conversation has focused on memory bandwidth vs. compute in agentic loops, so for most people the raw…
A lot of people just want to play "Smash Bros" on their iPad or get native ulatrawide support on "Majora's Mask". This might be a product vs. eng split, but "whole point of the endeavor" isn't the same for everyone.
Can you share what tools and processes you're using to do this? I've been using Pi to build custom extensions and wrapping workflows in shell processes to make it more deterministic and enforce certain validations, all…
It's exciting, but that's $9k for a model that's in the ballpark of last year's frontier. For many people that's like 10 years of up-front cost in a landscape that's getting better and cheaper every week. but compared…
There's great VST plugins like "Sketch Cassette 2" that will do hiss, flutter, all of that, along with some great presets. That one's less about creating authentic sound profiles, more like creating lo-fi tracks and…
If llama 3 70b were available for $400 today, people would make it work. They would have 4 of them working side by side or 1 of them working with a GPU on another model. To me, it's like imagining if Sonnet 3 was burned…
The barrier to entry for tinkering on Linux is so much lower now for anyone with a $20 Claude/Codex plan. What might have been an hours/days of fruitless head-scratching can now be done in minutes while you grab a…
In Bazzite I've had innumerable issues with waking from sleep and display driver crashes, which means a 50 ft walk to press the reset button. If you're cool with that, this appliance isn't for you.
I bought an irresponsible pile of homelab equipment in 24/25. Hard drives, SSD, memory, GPUs. I feel bad for people locked out right now, since it's become more interesting and important than ever. At the time it seemed…
To go further down this pipe dream - Anthropic / OpenAI would buy them all and still price out the consumer. There's no end-run in this scenario.
I can actually use and enjoy Linux. The "year of the desktop" never came for me, but instead I got the "year of the cli". For 20 years I've used Linux in one form or another, but I've felt like I was kneecapped for the…
It's fine for dense models where you need them in VRAM, less so for MoE where you're offloading layers to ram. But 32/32 is pretty good for both in the popular ~30b range right now.
- You can send any amount of money to anyone in the world very quickly and cheaply, and nobody can stop you. - No government can dilute it or limit its supply. Stuff like that. Maybe that matters to you, maybe not, but…
This is where a lot of the crypto scammers on x have moved, selling ai/bots to do this. It seems like the odds slip significantly after these large bets are filled.
For the huge percentage of devs using vscode, switching to Cursor was essentially adding a new color theme and a chat window. The CLI switch was far more radical.
Thank you for the thoughtful reply. I have some smaller 3080's I'm looking to place and this sounds like a good opportunity.
The memory upgrade is $1k on a Macbook Pro. The laptop is ~$5500.
Expect to pay $4k-10k - Your RTX 6000 is closer to $10k now - Sparks are creeping into the $4-5k range - AMD Strix are ~3.5k - Apple depends on chipset and memory. Sweet spot would be 128gb M3 Ultra, probably $6-8k but…