No, this argument doesn't make any sense. With CoT, the model must compress hidden state to actual text and use its scratchpad as a bottlenecked representation of its past thinking. Thus, it is observable and we can…
400k context with 100% on the fiction livebench would make GPT-5 the undisputably best model IMHO. Don't think it will achieve that though, sadly.
Don't count your chickens before they hatch. I believe that the odds of an architecture substantially better than autoregressive causal GPTs coming out of the woodwork within the next year is quite high.
Because of the niceties of Rust, combined with the widespread compatibility and architecture support of gcc / C compilers in general? Rust is a modern language, with package management, streamlined integrated…
No, this argument doesn't make any sense. With CoT, the model must compress hidden state to actual text and use its scratchpad as a bottlenecked representation of its past thinking. Thus, it is observable and we can…
400k context with 100% on the fiction livebench would make GPT-5 the undisputably best model IMHO. Don't think it will achieve that though, sadly.
Don't count your chickens before they hatch. I believe that the odds of an architecture substantially better than autoregressive causal GPTs coming out of the woodwork within the next year is quite high.
Because of the niceties of Rust, combined with the widespread compatibility and architecture support of gcc / C compilers in general? Rust is a modern language, with package management, streamlined integrated…