Nah ! I am not convinced that context engineering is better (in the long trem) than prompt engineering. Context engineering is still complex and needs maintainance. Its much lower level than human level language.
Given that domain expertise of the problem statment, we can apply the same tactics in context engineering on higher level in prompt engineering.
good survey of what people are already implementing, but I’ve convinced we barely understand the possibility space here. There may be much more elaborate structures that we will put context into that haven’t been discovered yet
I read these things and I think : this can never work. This is passing a huge set of parameters to a probabilistic map function.. one token changes and you get a completely useless result.
Is this an argument to upload more specific, detailed info (“context”) to tech companies? Which have lousy track records for protecting privacy? And an insatiable appetite for proprietary data? Why should any person or company trust OpenAI, Meta, Goo, etc? How does this make any sense? Or am I missing some reason to trust this “context” vision?
I'm consistently amazed by how great the first response from o3-pro deep research is, and then consistently disappointed by response number 5 or so if I continue the conversation. Better context management is the most important bottleneck in LLMs, and it seems like a robust solution would involve modifying the transformer architecture itself instead of using context limited LLMs to manage the context for other LLMs.
8 comments
[ 4.3 ms ] story [ 24.4 ms ] threadGiven that domain expertise of the problem statment, we can apply the same tactics in context engineering on higher level in prompt engineering.