I built some BigQuery workflows on 2.0 and 2.5 flash lite that are something like 6x more expensive with 3.1 flash lite. I tried 3 flash for months and it didn’t work using Googles own vertexai integration because it’s…
Lemonade proxies your request to llama.cpp that it installs and manages. Is out of the box all local LLMs, but you can connect it to an API endpoint (supports OpenAI compatible or Anthropic compatible). I think it…
I second this. I used llama swap for a while before leaning into lemonade. The UI has improved a lot, but be careful as the most of the models default to very small 4K context windows by default. They’re doing some nice…
Tmux with vim as a 50% split, I’ll run an agent in a quarter split. I have neovim hooked up to a local llama.cpp (through lemonade) for completion, but I have not found a plugin that I’m happy with. Avante is probably…
Dies the agent have access to is own nix config (and therefore install permissions), or do you have to provide it all the tools externally?
Mistral medium is considerably better at writing than Opus. I’ve also found it very good at pulling info from pdfs. Even a complicated festival with multiple venues and timetables.
AMD ROCm has come a long long long ways since last year, but you'll probably be happier not dealing with AMD's software. I posted another comment with my experience with AMD 395+, I am overall happy and it's usable now,…
I have the Framework Desktop with 395+ 128gb RAM Today I am pretty happy with it. LLMs are finally good enough (fast enough with MTP+MoE, but also just much better in capability) that I can fit local ones into real…
When I started self hosting last year it seemed insane that hosting a service required separately managing the imperative firewall state, vpn, container runner, systemd, and reverse proxy. Nixos is nice because it just…
I don’t know how anyone can look at the innovation going on at DeepSeek and come to the conclusion that China can only copy. Distillation and copying are how they’ve bootstrapped their models, but that feels not so…
The calculus is changing for non US, non Chinese users. Hypothetically if the US continues to restrict their frontier models and adds a ban on Chinese/open models then it would to obliterate services like open router.…
I think 3.5 flash is trying to target agentic work, like Google Search or ADK (agent development kit) use cases. It’s something cheap enough you’d put out in front of your customers, and Opus is expensive enough you…
Thanks for the suggestion, I am looking into it, it looks promising and I like that the pricing model is very upfront
The US is really shooting itself in the foot here. The restrictions on LLM models like Fable has created a viable international LLM market where it was difficult to justify investment two weeks ago. As a non-US citizen…
Related: Masonry Techniques of the Inca’s Master Builders https://www.earthasweknowit.com/pages/inca_construction This article was a fantastic read, and thoroughly debunks a lot of ancient alien style stuff.
It’s so hard to find usable products when everything is “XYZ for the Agentic era” Okay… what does that mean?
That’s on Wikipedia, it’s not PII, it’s also not going to be relevant to any meaningful IRL work. I challenge the assumption you can do meaningful work in this field without blatant disregard for intellectual property.…
This is task dependent. I find devstral (even though it’s weak generally) much better at writing and documentation than Opus. I’m actually now delegating all documentation to devstral and away from Claude, which makes a…
> For years I could ask Google what song was playing and it would identify it. It was one of the most useful features it had. You'd hear a song in a store, on television or drifting over from a neighbour's backyard and…
I feel this is a sort of inverse inspection paradox (the paradox that if you sample waiting time in a process, you’re more likely to sample a larger value). The LLM providers fine tune the models with some kind of…
Well I am definitely not using the models that I'm not able to access. So now the question is whether the capabilities of other models are worth their far cheaper token prices. Plus, are we at all confident Opus or GPT…
As a non-US citizen I guess this is the last money I pay to US companies for AI then. I can't help but wonder if it's now obvious that frontier AI work should not happen in the US. I can understand the KYC aspect of…
I don’t know. I can write a simple query before Claude finishes reading, querying the semantic layer, checking my files, then writes a query that I have to approve, reads the results, hides them (ctrl+o usually works),…
The cynic in me has learned one is measurable and can go on a slide deck, the other is vague and hard to measure.
I think that’s mostly okay, I just worry about the times. It was like 2-3 seconds to search and pull context then generate the answer. I’m writing AI apps these days, and even pulling Gemini 3.5 flash on Google Cloud…
I built some BigQuery workflows on 2.0 and 2.5 flash lite that are something like 6x more expensive with 3.1 flash lite. I tried 3 flash for months and it didn’t work using Googles own vertexai integration because it’s…
Lemonade proxies your request to llama.cpp that it installs and manages. Is out of the box all local LLMs, but you can connect it to an API endpoint (supports OpenAI compatible or Anthropic compatible). I think it…
I second this. I used llama swap for a while before leaning into lemonade. The UI has improved a lot, but be careful as the most of the models default to very small 4K context windows by default. They’re doing some nice…
Tmux with vim as a 50% split, I’ll run an agent in a quarter split. I have neovim hooked up to a local llama.cpp (through lemonade) for completion, but I have not found a plugin that I’m happy with. Avante is probably…
Dies the agent have access to is own nix config (and therefore install permissions), or do you have to provide it all the tools externally?
Mistral medium is considerably better at writing than Opus. I’ve also found it very good at pulling info from pdfs. Even a complicated festival with multiple venues and timetables.
AMD ROCm has come a long long long ways since last year, but you'll probably be happier not dealing with AMD's software. I posted another comment with my experience with AMD 395+, I am overall happy and it's usable now,…
I have the Framework Desktop with 395+ 128gb RAM Today I am pretty happy with it. LLMs are finally good enough (fast enough with MTP+MoE, but also just much better in capability) that I can fit local ones into real…
When I started self hosting last year it seemed insane that hosting a service required separately managing the imperative firewall state, vpn, container runner, systemd, and reverse proxy. Nixos is nice because it just…
I don’t know how anyone can look at the innovation going on at DeepSeek and come to the conclusion that China can only copy. Distillation and copying are how they’ve bootstrapped their models, but that feels not so…
The calculus is changing for non US, non Chinese users. Hypothetically if the US continues to restrict their frontier models and adds a ban on Chinese/open models then it would to obliterate services like open router.…
I think 3.5 flash is trying to target agentic work, like Google Search or ADK (agent development kit) use cases. It’s something cheap enough you’d put out in front of your customers, and Opus is expensive enough you…
Thanks for the suggestion, I am looking into it, it looks promising and I like that the pricing model is very upfront
The US is really shooting itself in the foot here. The restrictions on LLM models like Fable has created a viable international LLM market where it was difficult to justify investment two weeks ago. As a non-US citizen…
Related: Masonry Techniques of the Inca’s Master Builders https://www.earthasweknowit.com/pages/inca_construction This article was a fantastic read, and thoroughly debunks a lot of ancient alien style stuff.
It’s so hard to find usable products when everything is “XYZ for the Agentic era” Okay… what does that mean?
That’s on Wikipedia, it’s not PII, it’s also not going to be relevant to any meaningful IRL work. I challenge the assumption you can do meaningful work in this field without blatant disregard for intellectual property.…
This is task dependent. I find devstral (even though it’s weak generally) much better at writing and documentation than Opus. I’m actually now delegating all documentation to devstral and away from Claude, which makes a…
> For years I could ask Google what song was playing and it would identify it. It was one of the most useful features it had. You'd hear a song in a store, on television or drifting over from a neighbour's backyard and…
I feel this is a sort of inverse inspection paradox (the paradox that if you sample waiting time in a process, you’re more likely to sample a larger value). The LLM providers fine tune the models with some kind of…
Well I am definitely not using the models that I'm not able to access. So now the question is whether the capabilities of other models are worth their far cheaper token prices. Plus, are we at all confident Opus or GPT…
As a non-US citizen I guess this is the last money I pay to US companies for AI then. I can't help but wonder if it's now obvious that frontier AI work should not happen in the US. I can understand the KYC aspect of…
I don’t know. I can write a simple query before Claude finishes reading, querying the semantic layer, checking my files, then writes a query that I have to approve, reads the results, hides them (ctrl+o usually works),…
The cynic in me has learned one is measurable and can go on a slide deck, the other is vague and hard to measure.
I think that’s mostly okay, I just worry about the times. It was like 2-3 seconds to search and pull context then generate the answer. I’m writing AI apps these days, and even pulling Gemini 3.5 flash on Google Cloud…