These people really mis-understand how people like me use search. I don't want "an experience," I want a list of documents that match the search specifier.
If agents are making value judgments ostensibly on my behalf, why should I trust them to continue to be aligned with my values if in practice they're almost always running on someone else's hardware and being maintained on someone else's budget?
We'd be stupid to ignore the last 15+ years of big tech "democratization"-to-enshittification bait-and-switch.
Unless you're running your own search engine for yourself, search indexes and vector databases already manage what data they want to ingest, they contain rank weights, keyword aliases, and prefiltering for controlling the search result in favor of the service provider's desired outcome. And these all run on someone else's hardware maintained on someone else's budget.
Adding an LLM or agentic data flow and a tuned prompt to the mix does nothing to change who is in charge, it was never you.
Interesting approach. It might be helpful to give the agent more tools though. Some simple aggregations might give it a notion of what's there to query for in the catalog. And a combination of overly broad queries and aggregations (terms, significant terms, etc.) might help it zoom in on interesting results. And of course, relatively large responses are not necessarily as much of a problem for LLMs as they would be for humans.
I agree, my post only scratches the surface. I want to give more knobs to the agent. But not so many that it can't really experiment / reason about them.
The doc string becoming part of the prompt is a nice touch.
It seems plausible and intuitive that simple tools dynamically called by agents would yield better results than complex search pipelines. But do you have any hard data backing this up?
One interesting thing about the lack of click stream feedback is that you can generate it synthetically. If you've got your model evaluating the quality of search responses and adjusting its queries when there's a miss, you get to capture that adjustment and tune your search engine. In user click search you need scale to tune search, now you can generate it. The only problem is you need to trust your agent is doing the right thing as it keeps searching harder.
I think Comet's search is really nice and worth the $20 a month, but not $200 a month that it currently costs although it is a little slow. My experience is similar to this article.
I want my machine to be determinstic and non-magical. I am so tired of search tools that won't let me actually search for what I want because it clearly thinks I meant something else.
17 comments
[ 1.9 ms ] story [ 41.3 ms ] threadCitations needed. Calling recursive bullshit reason does not make it so.
We'd be stupid to ignore the last 15+ years of big tech "democratization"-to-enshittification bait-and-switch.
Adding an LLM or agentic data flow and a tuned prompt to the mix does nothing to change who is in charge, it was never you.
It seems plausible and intuitive that simple tools dynamically called by agents would yield better results than complex search pipelines. But do you have any hard data backing this up?
Turn this into a paragraph-sized prompt
Turn this into a n document length formal proposal,
And then split that into paragraph sized token optimized prompts
I want my machine to be determinstic and non-magical. I am so tired of search tools that won't let me actually search for what I want because it clearly thinks I meant something else.