Well, for a lot of agentic stuff nowadays, having 250k-500K context is where things live - and the benchmarks don't really show that unfortunately - but they could :)
Agreed - there was always a set of things I wanted to do that I knew the magic core for, but wanted a team of implementers for the curft, the 100k of actual testing harnesses, hyperparameter exploration, etc.. . I now…
We are there already pretty much - if I understand your point (“How the models are wielded”) refers to the harness - which is part of model training already. Fable was trained to use Claude code harnesses effectively to…
[dead]
Well - there is a giant push to allow non-qualified investors to invest their 401k (and roth and whatever) into the private equities market - pre-IPO companies and such. I can't shake the feeling of a grand fleecing…
A lot - and over the coming 2 years, even more. Utilization rates are under 50% across the board, and special and cheaper chips are coming out all the time for inference. And a truckload of research - TurboQuant, HC…
Imagine an agent shadowing all your terminals, providing ideas and asking to run commands that will let it verify the hypotheses it comes up with, while at the same time doing research on vendor docs, etc... Quite safe,…
Minor nit re[2]: for agentic workloads that are actually worth money - i.e., claude code and similar, things are either prefill-bound - which this does not help - or more importantly tps/user bound (at 150k+ context…
Kindof yeah - predictivity is a question though for larger layers - when trying to scale this up. But yeah, this is a "95% predictor in latent space is a 7x improvement in speed if done right" approach.
Yeah, forgot about them - 100%.
I kindof agree that it is unattractive - but the regulators are perfectly happy with "EOD also introduces credit risk on the clearing house/bilateral." if it allows them to protect retail and institutional investors.…
Note that _passenger aviation_ is commercially non-competitive. The big 4 US airlines make money on credit cards, not airfare : they lose money on airfare. So, most people who are trying to make money will not use them…
"who do carry liability when things go wrong" -> unless one pierces the corporate veil, it's just money. Not even their money. HIPAA - unless basically stealing data - will not generate personal liability. And even for…
We are multiple orders of magnitude away from Landauer limits - so next big thing in matmul could be photonic multipliers - there’s a bunch of them coming up in the next 3? years. So that’s a 2-4 order of magnitude…
I think the one thing you are not taking into account is that the investors on average fundamentally don’t care. Scale arbitrage means that small companies are fundamentally about velocity - and if they get sued due to…
But it does allow these investors to participate in the markets without losing their shirts - and the lack of such liquidity would impact the market more so than the cost of the risk mitigation - which as you completely…
I mean - I'd say electricity, agriculture, steam power, metallurgy, silicon computing (cmos), atomic power, the scientific method - these are _all_ very impressive - all lead to drastic changes for humanity. Not sure…
The EOD reconciliation (and corresponding inability to settle a position in milliseconds) is a feature - it allows "obvious erroneous trade" roll-back mechanisms, etc. Very few people want the financial system to be a…
Rumor says Hudson River Trading just ordered a bunch. So, the finance AI guys definitely. And they (AI finance - DeShaw, HRT, Citadel, G Research, XTX) have deployed about 15% of total GPU capacity, so not small fries.
I mean, they work? MI355 are quite good for inference.
Note that google cloud has an itar-compatible gemini pro and google drive / docs - so, people do talk to it - and google is of course contractually obligated to not export it, nor to learn from it. This is very…
What if you proxy through bifrost or similar?
The issue is not better - it’s better _AND_ fast enough. An agentic loop is essentially [think,verify] in a loop - i.e. [t1,v1,t2,v2,t3,v3,…] A model that does [t1,t2,t3,t4] in 40 minutes, if verify takes 10 min, will…
1 token ahead or 2? It's interesting - imo we'll soon have draft models specifically post-trained for denser, more complicated models. Wouldn't be surprised if diffusion models made a comeback for this - they can draft…
So, this especially bites if your validation step (let’s say integration tests) take 1hr plus. The harness is just waiting, prefix caching should happily resume things with just a minor new prefill chunk of output from…
Well, for a lot of agentic stuff nowadays, having 250k-500K context is where things live - and the benchmarks don't really show that unfortunately - but they could :)
Agreed - there was always a set of things I wanted to do that I knew the magic core for, but wanted a team of implementers for the curft, the 100k of actual testing harnesses, hyperparameter exploration, etc.. . I now…
We are there already pretty much - if I understand your point (“How the models are wielded”) refers to the harness - which is part of model training already. Fable was trained to use Claude code harnesses effectively to…
[dead]
Well - there is a giant push to allow non-qualified investors to invest their 401k (and roth and whatever) into the private equities market - pre-IPO companies and such. I can't shake the feeling of a grand fleecing…
A lot - and over the coming 2 years, even more. Utilization rates are under 50% across the board, and special and cheaper chips are coming out all the time for inference. And a truckload of research - TurboQuant, HC…
Imagine an agent shadowing all your terminals, providing ideas and asking to run commands that will let it verify the hypotheses it comes up with, while at the same time doing research on vendor docs, etc... Quite safe,…
Minor nit re[2]: for agentic workloads that are actually worth money - i.e., claude code and similar, things are either prefill-bound - which this does not help - or more importantly tps/user bound (at 150k+ context…
Kindof yeah - predictivity is a question though for larger layers - when trying to scale this up. But yeah, this is a "95% predictor in latent space is a 7x improvement in speed if done right" approach.
Yeah, forgot about them - 100%.
I kindof agree that it is unattractive - but the regulators are perfectly happy with "EOD also introduces credit risk on the clearing house/bilateral." if it allows them to protect retail and institutional investors.…
Note that _passenger aviation_ is commercially non-competitive. The big 4 US airlines make money on credit cards, not airfare : they lose money on airfare. So, most people who are trying to make money will not use them…
"who do carry liability when things go wrong" -> unless one pierces the corporate veil, it's just money. Not even their money. HIPAA - unless basically stealing data - will not generate personal liability. And even for…
We are multiple orders of magnitude away from Landauer limits - so next big thing in matmul could be photonic multipliers - there’s a bunch of them coming up in the next 3? years. So that’s a 2-4 order of magnitude…
I think the one thing you are not taking into account is that the investors on average fundamentally don’t care. Scale arbitrage means that small companies are fundamentally about velocity - and if they get sued due to…
But it does allow these investors to participate in the markets without losing their shirts - and the lack of such liquidity would impact the market more so than the cost of the risk mitigation - which as you completely…
I mean - I'd say electricity, agriculture, steam power, metallurgy, silicon computing (cmos), atomic power, the scientific method - these are _all_ very impressive - all lead to drastic changes for humanity. Not sure…
The EOD reconciliation (and corresponding inability to settle a position in milliseconds) is a feature - it allows "obvious erroneous trade" roll-back mechanisms, etc. Very few people want the financial system to be a…
Rumor says Hudson River Trading just ordered a bunch. So, the finance AI guys definitely. And they (AI finance - DeShaw, HRT, Citadel, G Research, XTX) have deployed about 15% of total GPU capacity, so not small fries.
I mean, they work? MI355 are quite good for inference.
Note that google cloud has an itar-compatible gemini pro and google drive / docs - so, people do talk to it - and google is of course contractually obligated to not export it, nor to learn from it. This is very…
What if you proxy through bifrost or similar?
The issue is not better - it’s better _AND_ fast enough. An agentic loop is essentially [think,verify] in a loop - i.e. [t1,v1,t2,v2,t3,v3,…] A model that does [t1,t2,t3,t4] in 40 minutes, if verify takes 10 min, will…
1 token ahead or 2? It's interesting - imo we'll soon have draft models specifically post-trained for denser, more complicated models. Wouldn't be surprised if diffusion models made a comeback for this - they can draft…
So, this especially bites if your validation step (let’s say integration tests) take 1hr plus. The harness is just waiting, prefix caching should happily resume things with just a minor new prefill chunk of output from…