I'm incredibly lucky (even in the US) that I have symmetric 5gbit connection and can go up to 8gbit if I wanted (but I don't have everything wired for 10gbit so it doesn't make sense right now). I'd love to see more of…
I really haven't seen any 16gb Mac Mini cluster setups running large models at any appreciable speed for the reasons previously provided. Do you happen to have examples that folks are actually using?
Yeah, made me suspicious of how well the Ox Alpha was performing that it wasn't some 'new group' making the model.
I think a lot of it is because the majority of improvements are not seen by those who already comparatively have so much. AIDs used to be a death sentence, and now it's a manageable disease, and there is a lot of work…
Absolutely computer driven. It's hard to say the impact of AI, let's give it the same 25 years of computer advancements.
Do you think monthly AI 'subscriptions' are going to be $100 a month in 5 years? These people using these would probably be on $200/month subscriptions and with that OPs assertion of 'shoving away and paying with…
And you still need at least 5% to do what OP is suggesting.
A Pi isn't a NAS with an old CPU and minimal RAM which is running a bunch of other things as well. I have run Plex on my Synology, and it's fine, but I also have 8gb ram and honestly always have a better experience…
I have had an impossible time getting 120B or better models running on Strix Halo (especially under Windows) with any large context windows. And 30-40 tokens/second is fine, but not the fastest. For the most part lately…
You don't have enough RAM. Not sure why you wouldn't use a cheap PC to run jellyfin on and then just use your NAS as the media pool.
As long as everything is using 10gb, faster transfer speeds when moving data between computers. For downloads, it wont help unless you have 10gb fiber, but for most folks, 2.5gb is quite fast. Hell my spinning media NAS…
I have had a difficult time with running 120b models on my 128gb setup, especially with any larger context size. The 6bit of Qwen 3.5 is already just over 100gb, and when you go down to 4bit it seems a bit lobotomized.
Or just use Privacy.com and use an address in Delaware. Then you can buy it where ever.
How are you liking Omarchy? I saw a video on it recently, and it looks 'pretty' but still looks like it's a lot of memorization of shortcuts and feels like the 40% keyboard of OS's. Like some people it's absolutely…
Big ole pool of very fast ram that can be accessed by the CPU and GPU. Lets you run larger models. AMD does the same thing with Strix Halo. I have a 128gb machine at home, and have had difficulties running 120b models,…
At least in the US it's not even a good method either. Own a house? Boom, your information is publically available and you won't be able to remove it without a court order, which in most states is impossible to get. In…
It's not sad, it's always been the case, and just expanding areas that are implementing it. It seems a lot of people here haven't travelled, or especially haven't travelled to 'restrictive' places in the past.
I want to use this as internal AI guidelines, I am so tired of people in my workplace simply copying and pasting whatever garbage AI gives them and expecting everyone to just go along with it and try to decipher what…
You could say that same exact thing for the entire Industrial Revolution, but that doesn't mean we are going to destroy the looms even though some tried unsuccessfully.
Sure. Realistically you simply regulate them. Espresso martinis are already long a popular 'after dinner' cocktail with the assumption that you'll be buzzed from the caffeine and able to drive better.
I am not sure references are easily faked. Especially in industries where there aren't a whole lot of references that someone is not going to know. "Hey Bob, I have this excellent candidate for the position you are…
Nah, no moral panic here. I don't see the reason to combine stimulants with depressants, you then just mask the underlying issues with both. Hence often the regulation of the combination.
Couple of rack mount batteries and roughly 5kw of solar panels. Feeds into a subpanel so I can flip it when I want a couple rooms of solar on the house, or hook a generator up if needed. Can't power the entire house,…
Strix Halo for me. If I am running something on my laptop, it's a much smaller usually around 12b model, but those are a bit less functional. I mean I think there is a a ROG FLow Z that has the Strix Halo setup, but…
Depends on how you are doing it. LM Studio and tool calling models can use the web, or you could go for something like Perplexica, or if you want to go real crazy, something like Hermes or OpenClaw.
I'm incredibly lucky (even in the US) that I have symmetric 5gbit connection and can go up to 8gbit if I wanted (but I don't have everything wired for 10gbit so it doesn't make sense right now). I'd love to see more of…
I really haven't seen any 16gb Mac Mini cluster setups running large models at any appreciable speed for the reasons previously provided. Do you happen to have examples that folks are actually using?
Yeah, made me suspicious of how well the Ox Alpha was performing that it wasn't some 'new group' making the model.
I think a lot of it is because the majority of improvements are not seen by those who already comparatively have so much. AIDs used to be a death sentence, and now it's a manageable disease, and there is a lot of work…
Absolutely computer driven. It's hard to say the impact of AI, let's give it the same 25 years of computer advancements.
Do you think monthly AI 'subscriptions' are going to be $100 a month in 5 years? These people using these would probably be on $200/month subscriptions and with that OPs assertion of 'shoving away and paying with…
And you still need at least 5% to do what OP is suggesting.
A Pi isn't a NAS with an old CPU and minimal RAM which is running a bunch of other things as well. I have run Plex on my Synology, and it's fine, but I also have 8gb ram and honestly always have a better experience…
I have had an impossible time getting 120B or better models running on Strix Halo (especially under Windows) with any large context windows. And 30-40 tokens/second is fine, but not the fastest. For the most part lately…
You don't have enough RAM. Not sure why you wouldn't use a cheap PC to run jellyfin on and then just use your NAS as the media pool.
As long as everything is using 10gb, faster transfer speeds when moving data between computers. For downloads, it wont help unless you have 10gb fiber, but for most folks, 2.5gb is quite fast. Hell my spinning media NAS…
I have had a difficult time with running 120b models on my 128gb setup, especially with any larger context size. The 6bit of Qwen 3.5 is already just over 100gb, and when you go down to 4bit it seems a bit lobotomized.
Or just use Privacy.com and use an address in Delaware. Then you can buy it where ever.
How are you liking Omarchy? I saw a video on it recently, and it looks 'pretty' but still looks like it's a lot of memorization of shortcuts and feels like the 40% keyboard of OS's. Like some people it's absolutely…
Big ole pool of very fast ram that can be accessed by the CPU and GPU. Lets you run larger models. AMD does the same thing with Strix Halo. I have a 128gb machine at home, and have had difficulties running 120b models,…
At least in the US it's not even a good method either. Own a house? Boom, your information is publically available and you won't be able to remove it without a court order, which in most states is impossible to get. In…
It's not sad, it's always been the case, and just expanding areas that are implementing it. It seems a lot of people here haven't travelled, or especially haven't travelled to 'restrictive' places in the past.
I want to use this as internal AI guidelines, I am so tired of people in my workplace simply copying and pasting whatever garbage AI gives them and expecting everyone to just go along with it and try to decipher what…
You could say that same exact thing for the entire Industrial Revolution, but that doesn't mean we are going to destroy the looms even though some tried unsuccessfully.
Sure. Realistically you simply regulate them. Espresso martinis are already long a popular 'after dinner' cocktail with the assumption that you'll be buzzed from the caffeine and able to drive better.
I am not sure references are easily faked. Especially in industries where there aren't a whole lot of references that someone is not going to know. "Hey Bob, I have this excellent candidate for the position you are…
Nah, no moral panic here. I don't see the reason to combine stimulants with depressants, you then just mask the underlying issues with both. Hence often the regulation of the combination.
Couple of rack mount batteries and roughly 5kw of solar panels. Feeds into a subpanel so I can flip it when I want a couple rooms of solar on the house, or hook a generator up if needed. Can't power the entire house,…
Strix Halo for me. If I am running something on my laptop, it's a much smaller usually around 12b model, but those are a bit less functional. I mean I think there is a a ROG FLow Z that has the Strix Halo setup, but…
Depends on how you are doing it. LM Studio and tool calling models can use the web, or you could go for something like Perplexica, or if you want to go real crazy, something like Hermes or OpenClaw.