Tangential, but Librivox's search leaves a lot to be desired. I love that project, and can mostly just use Google, but it would be a great volunteer project for some good Samaritan with a little more experience enhancing search than me.
The money from showing a pancake ad make them do that. It's not a search problem. They know it's not a good experience but they also know it makes them more money than not showing it. The goal is money. Your experience is merely a means to that end.
The profit margins on selling products are abysmal for e-commerce. The profit margins on ads are amazing.
Ideally, manufacturers and sellers would provide more, accurate information about products -- but I generally like how Google solved this within Google Maps via crowdsourcing by prompting users to answer questions about places (e.g. does this place have a wheelchair ramp?). I wish there was a better process to collect and store product information so it could be more easily shared.
And in any sane company, the minute a product person says "oh this egregious error is a problem for some other department" they'd be fired immediately, so it still sounds like a useful exercise.
I'll be the first admit I don't have a ton of respect for the "product owner" role which has morphed largely into Tom Smykowski from Office Space[0] but the places I've seen them be actually effective and net positives for an organization are where they have the reach and authority to demand things like this be fixed, regardless of what department is "in charge." IMO a product owner who doesn't do this is refusing to do one of the few useful product owner roles.
I've worked on products with ALL of these exact issues. The answer is usually that the data isn't annotated well enough, or that there are not enough developer resources to make special case fixes for things 99% of people don't notice or care about (and so are not losing us money, they just annoy us). At the companies I worked for, fixing these problems was not within the remit of the head of product, they were cross-departmental issues and so were in that pernicious category of problems that are highly resistant to change. I assume the reason you think product people are responsible for this is because you aren't a product person, but in my experience those are the people who most desperately want this type of shit to be fixed, because it makes them look like clowns.
The responsibility of a product person is that the product works well. This includes working with design and engineering partners as well as any data annotation departments. If the search experience is shit and the head of product doesn't think it's up to them, they aren't being responsible for the product.
The reason the responsibility for this is on heads of products is not because they're directly responsible for annotating, it's because they're responsible for knowing what can and will get annotated so the feature they asked for actually works.
The work of product person shouldn't stop at the idea phase.
In my experience it's not just about using a product but understanding how a product functions under the hood. Yelling "fix it, fix it, fix it, fix it" in a loop at random teams tends to not actually get things fixed. Instead you'll get a lot of blame shifting, a lot of random "projects" to improve things and in the end no improvement (or often an even worse experience). In other words you'll get corporate politics. Very few executives understand a modern search stack in anything resembling a useful level of detail much less how their teams map to it.
I've never been lucky(?) enough to reach a scale or level of domain messiness where the standard method of indexing things naievely in elasticsearch or even typesense breaks down.
Especially to such an extent as the umbrella example.
What's the failure mode here? Microservices that "own their own data"?
The main failure mode here I find is nuances in more specialized datasets. Elasticsearch is great, powerful and easy to work with once you get the hang of it, but is very open-ended with lots of room to build a sub-optimal solution.
Fuzzy matching and boosting often trip people up or lead to folks shooting themselves in the foot relevancy-wise.
If you want really great results, you need to spend the time crafting your query, dealing with synonyms ("pop" vs. "soda"), stemming, typos, negative boosts ("non-alcoholic", in the example) etc.
It's not necessarily hard, just often forgotten or not included in the initial scope.
Many if not most ecom sites will be using a vendor for search (e.g. Bloomreach) so they're sending a feed of product data to their vendor, just a subset of their product data (which may not have enough metadata for some facets people might reasonably expect). That means there's lag, and also that the integration is stiff and brittle, hard to change without time and money. Even then, whether they're rolling their own search or using a 3rd party, nine times out of ten their search will be ONLY for products, good luck if you wanted to search for a phrase from their privacy policy; for that or any sort of other non-product content you're forced to use Google/Bing/ddg etc.
I spent a couple of minutes thinking of a good example of search. Google? No more by a long shot, Amazon also a cesspool. Finder finds too many irrelevant files (no matter what I type it returns a huge list of node_modules matches), this one might be on me. Search on Windows I haven't bothered in years.
The only search tools that actually give me great value are history shell with fzf on the terminal, and grep (and similar like ag or ripgrep) in combination with a fuzzy search frontend (similar to piping grep to fzf).
I don't know where I'm going with this. I guess searching just sucks now.
My cynical take is that they don’t care what you buy, as long as you buy something. So the more results, the better. Even if they don’t exactly match what you asked for. After all, the absolute worst thing to do would not show any (purchasable) results at all.
If you want to get academic with it, you could say something about revealed preferences vs stated preferences (they asked for Coke but they really wanted Pepsi), but I really think they’re just throwing as many suggestions at the user as possible in the hopes they stay engaged on the site and eventually click buy on something.
> I honestly don't understand how some websites have such awful search
Good website search is hard. Funnily in some respects it's harder than web search, since there are no popularity signals a la pagerank. (There could be such signals, but you have to work hard to create them.)
There is a logic, you type "n" and "news.yc[...].com" comes up, so when you type "ne" then you're presumably (the logic goes) looking for something other than the website that was just presented to you; otherwise you would have chosen that website already rather than continuing to search by adding more letters.
In practice people type what they think will turn up the results "news", say, and don't look at the results until they're finished typing.
I guess that logic isn’t totally absurd. In that case, it would be nice if, under their “current best guess,” a descending list of all the previous “best guesses” was shown.
Another annoying aspect to it is that things enter your history as you browse, so the number of characters to hit that point where it shows what you want might shift around. Which totally destroys muscle memory. This isn’t the browser’s fault of course, but it would be nice if they could design around it.
I think there is some weighting based on how often you go to a certain result after typing a certain number of letters. For example, since I was just doing Advent of Code and going to related sites:
a, ad, adv -> first result is adventofcode.com/2023
adve -> first result is adventofcode.com/2023/day/1 [honestly, this one baffles me, but I guess I went back to day 1 at some point after typing `adve`]
adven, advent, advento -> first result is reddit.com/r/adventofcode
adventof -> back to adventofcode.com/2023
I actually *appreciate* this behavior; it means I can better muscle-memory a certain number of letters and go down to the first result when 2 sites have similar names (very annoying for twitter & twitch btw).
So I don't think the problem is that weights can change as you type more letters; but that an entry can disappear altogether when there are no other suggestions.
I’d much prefer they took VS Code’s fuzzy non contiguous substring searching algorithm. ra would be reddit.com/r/adventifcode, same for redadv, or r/a, etc. and things like a, ad, a3, etc. would be adventofcode.com/2023
And if they included the full URL, sheesh… you could effortlessly query against search params, path segments, fragments, and origins all in one intuitive interface.
Not quite the same, but iOS gives me trouble with its word suggestions. Usually, it’s contractions that make me curse.
‘It’ suggests It, It’s, and Its.
‘Its’ suggests Its and Itself.
Given that the apostrophe requires accessing a different keyboard overlay, and that only two suggestions are presented (leaving one suggestion slot empty), why not just include the contraction “It’s” in the suggestions?
A few versions back, iOS started sometimes not finding my searches in settings at all.
Like I’ll search the full word “passwords”, pausing after each letter, and not get the entry titled “passwords” in my results the whole time. Back out, do it again, this time it’s there by the time I type “pas”.
This used to work perfectly, but has been inconsistently broken for years now. Most recent time I saw it was a week or two ago, so it’s still happening. What’s most baffling to me is that a search over such a relatively static dataset (app settings are in there, but those don’t get added or modified often) can be non-deterministic.
iOS seems to have some weird temporary grammar check issues, where it’ll show the correction while it works out what you are trying to write. I think it only does it for long sentences, though.
But for example, if you start a sentence with “We’re,” it will often take out the apostrophe… until you reach the end of the sentence, and then when you add the period I guess it’ll analyze the sentence and figure out that you actually meant what you wrote, and put it back. It does highlight the corrections, which is really nice. At this point, I just ignore it until I’m done with the sentence and then go back and check.
The worst offender of this practice has to be the windows start menu. Even when you fully type the executable name, it can show up below the fold under unrelated results.
Just try to get to display settings on windows these days through the start menu.
Which is why I have disabled that functionality. I also don’t need M$ to know every app I’m running on my PC (although they probably have that data some other way).
The firefox awesomebar and the history search are atrocious. Sometimes even if I visited the site in the last 5 minutes there will be no autocomplete available.
The one that really hurts my feelings is when the search function on a manufacturer’s website doesn’t recognize their own part numbers. Then I get angry and go to google, which gives me unrelated ad results instead. Ah, familiarity.
Having previously worked for a company with this problem, sometimes you need to start pulling other people’s teeth to get them to stop putting data in random fields.
We just told our customers to google the part number. I don’t know how these places stay in business.
I've never worked at a place where part numbers mattered, but I am very familiar with this mindset of non-technical people that basically boils down to "just put whatever you want in whatever text field then act befuddled when the system doesn't work perfectly."
I've often fantasised about scraping Amazon just to be able to actually search on the site.
eBay too, it's OK (compared to Amazon) but most of the items on a search are always "mousepad [or whatever] suitable for" the laptop you searched for. Why eBay don't clean up their categories and ban sellers I don't know, it makes the site so much worse to use.
I wonder if anyone is _ever_ hoodwinked by the vast majority of sites that sort by Most Expensive by default?
It's all part of enshittification presumably, these sites monitor so many metrics it must surely be a profit generator to make search frustratingly bad.
My wife prefers to use curbside pickup for groceries, and orders via the Harris Teeter website. It’s a disaster.
The site search picks up unrelated items, misses others, and so on. I would complain about the inventory management/substitutions too - but since they also have in store shoppers using the same inventory I guess they can’t always be 100% about what’s on the shelf.
You’d think a chain that size could do better, but I doubt they have much incentive. The competition is minimal and not any better.
we're all looking at these as technical mistakes, but it wouldn't surprise me at all if, during A/B testing, people spend more when search doesn't quite work. For a similar reason as why they put milk in the back of the store. optimizing shoppers' efficiency doesn't optimize their profit.
I used to work on a regional grocery chain's website (in the Southern USA) for online ordering and fulfillment. It's a constant battle. Consider the sheer number of products on store shelves. The average number of products is 30k+. Now, consider that some percentage of those are constantly in flux and potentially unique per store. Holiday items, promotions, marketing from suppliers, overflow, etc etc. Anytime a price is changed or a product brought in/out of stock, that line in the database needs to be updated. Product images and nutrition information can also be in flux as companies adjust their products in response to supply chain or financing particulars.
At the end of the day, gathering and aggregating all of that crap into a stable API is a chore unto itself. Multiple people's jobs. Updating products, associating taxonomies, in-store locations, duplicate locations, optimizing pickup routes, etc etc etc. There is a lot of grunt work here that can't be entirely automated away when dealing such a dynamic business.
Now you need to build a website that takes all of this data and somehow displays it in a format that is comprehensible and accurate for consumers. Showing relevant recommended products, search keywords, cache and image management (for potentially resizing dynamic supplier photos). Then think about the cart and checkout, two areas and Shopify and Amazon have devoted literal billions of dollars to optimize. Your volume is big enough that using an off the shelf payment provider is off the table; not to mention that you don't have the margins anyway. Have you thought about mobile apps and the development effort required there as well?
Grocery stores are hard enough as is but doing them online is a totally different beast. There's a reason that all of the "Kroger" brands use the same exact website with a different logo in the corner. The only players in this space doing it reasonably well are huge conglomerates with money to burn on building the required infrastructure to build the audience. Actually, I just looked up Harris Teeter and they appear to be a Kroger brand as well. TIL. I would say that "Kroger" is one of the better ones. Whole Foods is also doing decently well if you can afford the price premium, but I realize for most people that isn't a luxury they can afford.
The best way to get them to improve is to keep using the service and demanding better (or use a competitor if available). These departments have KPIs. If you get the wrong product, file a complaint at the end of your order process and mark any NPS surveys appropriately. If fulfillment has a lot of bad data from elsewhere in the organization driving their performance scores down, then they are incentivized to get it fixed. I know it sucks, but the only other alternative is to not use it at all and risk is going away forever.
On the other end of the spectrum, I've been impressed by Netflix search. I would search a name and they do not have that movie, but would suggest a mix of similar movies, movies with the same actors or director. It's pretty clever, I just wish they were more explicit about it, as in showing the reasoning behind choosing a result.
That doesn't make a ton of sense in the Netflix case since presumably it's actually more expensive for every additional movie you watch since your subscription price is fixed and each additional movie you watch costs bandwidth and compute?
Netflix wants you to justify the cost in your mind. The bandwidth is free. Netflix is such a high volume of internet traffic that the CDNs actually have edge content caches within the DCs of most major ISPs, so when you stream you’re actually streaming from a box at your ISP (or a cross connect to Netflix) and not costing Netflix any internet-facing bandwidth besides API calls. This setup is cheaper for everyone because the ISP doesn’t have to subscribe as large of an internet pipe because the high bandwidth streaming is siphoned off into private networking and never leaves the DC.
I don’t understand. How is it useful for Netflix to show me movies that they don’t have? And how hard is it to implement search over their catalog when they only seem to have about 100 movies?
Just because they don't have the rights to stream The Office doesn't mean they don't know what shows and movies they have are enjoyed by people who also enjoy The Office. This is literally the entire point of search - show what you're searching for if it exists, and show related things after it.
> This is literally the entire point of search - show what you're searching for if it exists, and show related things after it.
That might make sense in an abstract way, but no, that's not what people expect. If I'm searching for something at Netflix, I don't want the answer to "does it exist?", I want the answer to "do you have available for viewing?".
So you search for The Office on Netflix which is only available on Peacock as far as I know. How is a screen that says "We don't have The Office, sorry!" with no other information at all preferable to this? https://i.imgur.com/W6a1hDk.png
No, it's fine to offer similar products. Good on Netflix for this.
In the more general sense of the search topic,
I don't want a search to return things they don't have.
If it's out of stock,
say so prominently.
Don't make the user click through to the item itself only to see "out of stock".
I had migrated to watching Netflix content on pirate sites for the better experience and stayed for the better experience.
> I've been impressed by Netflix search. I would search a name and they do not have that movie, but would suggest a mix of similar movies, movies with the same actors or director.
An excellent, early pirate feature. Good on Netflix for adopting it.
Searching google's movie service for older movies is annoying because a lot of times you just get a remake. Even if they have the old version! The only way I found to get to the original is to search for some actor in it and scroll...
Their input search works fine, but clicking through the categories seems to surface all the dogshit Netflix-created content first. It's like digging through trash.
My main issue with the input search is the TV keyboard. There was a Netflix hack day that created a circular keyboard like a dial w/ the selector a joystick in the middle, so pressing up to 12 o'clock was A, 12:05 was B, etc. And the joystick would return to center per selection! It was everything I ever wanted in a thing, and now I can't even find the article on it.
Netflix search is possibly the most egregiously bad search in the entire industry, and unfortunately it's infecting other players with very stupid ideas. Netflix search is unfaceted, unsortable, unfilterable and worst of all it very often knows what you're searching for and instead of returning "we don't have this title" it will return a bunch of different, loosely related titles.
What's worse is that they explicitly know that they don't have that title, because they do have that title if you visit from a different region. But they waste your time with showing you irrelevant results instead.
Functional, well-tagged, data-driven parametric search. Rockauto.com sells auto parts, which requires a whole different type of search, but they deliver a pretty impressive product as well.
I suspect that much of this discrepancy is because there's enough of a chance that someone grocery shopping and looking for lentils will think "Oh, I need pancake mix too" and click on the ad. Meanwhile, someone looking for a specific grade of M8x25 socket-head cap screws or a low-power Cortex-M0 microcontroller with a particular set of peripherals is unlikely to abandon their work to add a hose clamp or electrolytic capacitor to their cart instead.
Market forces seem incapable of delivering a product with a better user experience if that hurts the bottom line even a little bit, they're doing gradient descent to local minima and powerless to say "no" to finance and climb out of the awful product that results.
While the overall point holds true, some of the complaints don't make sense. Searching "rice" for example will probably get you parboiled white rice, instant red beans and rice, chicken and rice canned soup, and rice pudding— many of those facets make prefect sense. If you were searching for rice paper for crafting, or even "nuts," that "food" facet would be plenty useful.
You can't look at one narrow usage path to judge an interface. You can only customize the interface so much for specific use cases before it becomes confusing as hell to use because it lacks consistency between tasks.
The complaint about "rice" is that the tagging is incomplete. "vegan" isn't a bad facet, but the count is wrong. When you select "vegan", most vegan products are removed. I don't see how filtering wrong data isn't a valid complaint.
The problem is obviously that the data isn't tagged. But Costco is huge, profitable and doesn't have a huge inventory, so why can't they tag it? And if the tags are so incomplete, maybe it's better not to include them?
Have worked on this and it’s between 60 and 90% OK in my experiments. You run into lots of fun issues because people don’t write titles and descriptions for machines, they write them to appeal to human emotions to close a sale.
> Searching "rice" for example will probably get you parboiled white rice, instant red beans and rice, chicken and rice canned soup, and rice pudding— many of those facets make perfect sense.
Well, no, if I ask for rice, the odds are overwhelming that I'm looking for rice, in which case chicken and rice canned soup is a totally inappropriate response. It would make as much sense as returning alcohol-free wine when asked for "alcohol".
But chicken and rice canned soup contains rice and the word is part of its logical title.
You should be able to unearth the soup with a partial match. What makes the alcohol/alcohol-free problem is that alcohol free is one token that can't be subdivided.
That is not a consistent position. If "alcohol free beer" doesn't contain the word "alcohol", then "chicken and rice soup" doesn't contain the word "rice".
Moving the argument into the realm of things that matter, chicken and rice soup is not a product that can be called "rice", and therefore isn't an appropriate response to a search for "rice". The title is irrelevant; if you're stocking rice with a fancy name and its title doesn't include the word "rice", that product still should show up in a search for "rice".
I don't get this position, chicken and rice soup is not the most relevant result for the query rice but it is a match. It should show up in the results somewhere. If I page through all the results of rice I expect to get all the results that conceivably match the query.
Yes: chicken and rice, rice cakes, rice paper, cauliflower rice, rice cooker, rice krispies, rice university jersey, mochi, risotto.
> I don't get this position, chicken and rice soup is not the most relevant result for the query rice but it is a match.
As I said, that is inconsistent with your simultaneous position that alcohol-free wine is not a match for "alcohol". Either they both are matches to their respective queries or they both aren't; there isn't a difference between the two examples. This is true from the perspective of dumb text searches, but it's equally true from the perspective of correct linguistic parsing.
That rice is part of the listed title for chicken and rice soup is a coincidence; the argument for returning it also tells you that you should return the same soup when asked for "tomatoes", if tomatoes are an ingredient in the soup.
Your "yes" and "no" examples don't seem to work. What rule excludes "licorice" that would include anything containing the raw textual string "rice" without regard to whether any rice is involved? Rice paper, rice cooker, cauliflower rice are all purely textual matches. Mochi is a purely semantic match!
You’re describing the problem from a computer scientist POV. OP is describing the problem from a user’s POV. Nobody cares if there’s a subset of their query in the title. That item is not “rice”. No one is searching for “rice” and wants soup. They would search for “soup” then. Tokens aren’t all of equal value in a title, and this is a big part of why ecom search is horribly bad (I have spent years working on ecom search and partnering with vendors trying to improve it. It’s a hard problem.)
What if I want a rice-based ready meal, but don't know exactly what kind of meal yet (I'll know when I see it), so I'd like to search rice and see what's on offer?
Oh! Chicken and rice soup, I could go for that. Cart.
> What makes the alcohol/alcohol-free problem is that alcohol free is one token that can't be subdivided.
It's not even a problem... it's an unfounded assumption that alcohol-free wine wouldn't ever be relevant to the user at all. If the only place open on a holiday was Walmart that I didn't realize didn't sell booze, and I needed some kind of booze to make a pan sauce or a braise, I wouldn't necessarily think to search for non-alcoholic booze-- but it would be a very useful substitution for me, and not by accident... they're two versions of the same product. I'd much rather see that they had non-alcoholic beer and wine in my search than completely empty search results.
This is why competent companies hire designers to work on interactions like this rather than leaving it up to the implementing developer's gut instinct about user needs. If only the tagging could keep up.
Assuming these sorts of things about your users' intent causes big usability problems. Trying to predict how users will use search usually ends up more frustrating than not for many people. Most people who use these tools don't have a working mental model of how search engines work, so they can't troubleshoot easily if your expectations and theirs don't line up. This is exactly the sort of "smart" feature that developers crow about all the time. Unless you have some way of predicting how individual people will use search boxes then you're much better off with something more general.
To some extent, this is consistent with Google's mobile Maps version when searching for anything. It returns "Relevance" by default, not by "Distance".
I also noticed the same in several other website where "relevance" trumps my sort request (e.g., price, distance).
I hate this with a passion. I can be out somewhere, knowing that there's likely a McDonald's nearby. I zoom in to my location, type in McDonald's, and it zooms out to a 25 mile radius to show me every single McDonald's in the area including the one 0.1 miles away (that I then have to zoom all the way back into to get)
More favorite of mine is then it decides to fire up the first time user experience for some new features for you or shift into driving mode you thought you disabled. The you click what you think is the nearest MCD and it's the one close as the crow flies and on the other side of the freeway so 3 miles away.
Search along route sometimes works better and sometimes also zooms out.
The last time I used Craigslist to apartment hunt, people spam the listings with the name of places 30 minutes away for some reason, as though I would just give up on living in the neighborhood I was searching for if I saw there was something somewhere completely different at a price that wasn't even that good.
These are hilarious and infuriating but also, perhaps, correct?
My gf (who used to work in the sector) says a lot of people like to sit in front of the TV and “watch what’s on” — one of the reasons broadcast TV still exists.
You see this with sites like Netflix becoming decreasingly useful, urging you to simply click on something rather than consult your curated watch list. Some of them now hide your watch list!
She suggests the same applies to shopping: some people just want to push the buy button, which is well known in brick and mortar retail, especially low margin ones like grocery shops: end caps, mid shelves and all sorts of other strategies are used to get you to either buy on impulse or buy the version they’d rather sell of what you have in mind.
So for these sites the same applies: show a few options and the customer will simply pick one. Spam the tags, DWIM the customer’s ill-formed, uninformed, or simply unsatisfiable query, and see if the punter buys.
For nerds like us this is a pessimization. But perhaps for the majority this works better.
I've noticed a lot of streaming services will inconsistently show your watch list and/or show it in different locations on the home screen. I've long wondered if it's a mix of certain services just not being available or taking too long to respond when loading the lolomo (to use Netflix's term) and automated A/B tests trying to maximize some metric other than "starts watching something."
Mentally when I'm at a store and I have 3 options to pick from I seemingly enjoy the process better then if I had 6 or 10. Eventually the picking itself becomes work. If it's a topic I enjoy like motherboards or RAM then all the choices are fine, I can spend hours on that, but if I'm picking tomato sauce for a quick and sloppy dish I don't want to spend the time selecting.
Yep. YouTube search has become smart about this, blending search with browsing. It typically gives you a handful of reasonably close matches, followed by a whole lot of wildly offtopic suggestions based on subscriptions, watch history, or who even knows. And this happens even when there would be more relevant results to show, i.e., there's not even a pretence of trying to give the N most relevant results. The game is to populate search with whatever users will click on.
What you're describing isn't "search", designed to give me information I asked for. It's a recommendation engine, designed to induce a behavior in me, whose input is the "search term" I entered (plus god knows what other tracking data). The distinction is important: the recommendation engine is designed to get me to buy something (not necessarily what I came to buy), whereas my intention was to buy something (or buy nothing, if they don't have it). Calling this recommendation engine "search" is IMO deceptive.
Maybe if I'm on Netflix to watch what's on, that's fine. But e.g. on Amazon, I find "search" so adversarial that I often close the tab and go somewhere else, or give up.
The point of website search (at least on commerce sites) is not to help the users find what they're searching for, but to put other, possibly related, products in front of their eyes. On most sites drilling down through limiters falls apart after about the third or fourth restriction is chosen, often because the sites seem designed to not give three or four clear choices (which is what the user wants to see), but ten or more vaguely related items (is this due to the fear that giving three or four results will imply that the site has a shortage of items to choose from?). Keep marketers & managers away from site design.
I may be wrong, but my guess is the site makes its money off those 'promoted' search terms of most popular/most profitable items and the work needed for the long tail doesn't seem worth it for most teams.
It’s both. What you’re referring to is “searchandizing”, and it’s a component of e-commerce for sure. But if you’re not also retrieving what the customer wants you’ll just piss them off and they’ll shop elsewhere.
I can see that. It feels more like the car salesman who says, "Sure, you can look at the car you actually came here to see, but only after I show you three more we'd prefer to sell."
This has already roots in real stores where they shuffle stuff so it is harder for you to search for just what you need.
It was counter-intuitive for me at first. Because if I can't find in store/site what I need I just leave. But enough people will buy also other things that store put in their way. So broken search is feature, not bug.
Fixing this is the promise of enterprise deployments of LLMs, correct? That you can take the list of initial result texts from a shitty elasticsearch implementation, pass it into an LLM with a prompt "you are a user looking for <query>, please use common sense to comb through these results and find a few good ones" and get better intramural search quality.
The only other enterprise application I am aware of is applying the homework cheating app to customer facing roles that require email/copy generation.
Right but implementing search correctly often means humans combing through the data and applying common sense to tag the products with their actual facets. So really you're just choosing between a square kilometer of office space for cheap labor or GPUs.
If you have >100k items, “implementing search correctly” is an intractable problem if you don’t have a hundred person team of search relevance optimizers and taggers. Search is terrible until you gather tons of metadata about each item. Search is not nearly as simple as text matching within product titles. An LLM by comparison is peanuts. You could run it overnight and just have it build tags and metadata for all your items and it would probably be 75% accurate and a hell of a lot cheaper. Things can then be fine tuned by people later.
* PM to Team: Alright, I need a search feature next sprint. Architect googled for 5 minutes and says we should use elasticsearch!
* Team: Alright, integrating elasticsearch in a sprint should work. Bob, you work on loading the data, Eve, you work on rendering the search results.
* Team (2-3 weeks later): Another successful value delivery! Alright, what's next PM?
* PM: PO says we need a chat bot...
If it's not obvious why this is a problem, getting search to work well requires a lot of tweaking and tuning. You can't just slap a search engine into a product and expect it to work well. It's hilariously incompatible with the assembly line style of work that many software outfits employ these days. It also requires deep understanding of what a search engine does and how its algorithms work.
Intellectually, I knew this was common but I never stopped to consider how fortunate I am to work for a company who understands things like search are difficult.
It helps we have two internal departments whose primary jobs are data management and analysis. (Real statistics, not “AI”.)
It's more about what the company considers core business and what not. Most often they don't see the website as being important enough to the business so they don't invest in it.
We have gotten used to almost flawless experiences from amazon shopping. Google search finds (or used to) results sometimes almost like magic, etc. The thing is, that's the core business. These companies invent new technologies and have huge teams because doing this is so hard.
Now take a random supermarket chain. Their knowledge is about physical stores. Their core business has taught them where to open a store and how to arrange things in it so that they maximize their sales in that environment. It's very hard and it takes a very long time to shift to an online model. You have to find people with the right competencies and the right leadership to convince the company to do this and this is actually very hard to do.
Look at the categories example. That, to me, screams backend database. The company has invested a lot of money into building business intelligence on top of their physical stores. That organization screams "perfectly curated data warehouse" and I imagine suggesting something like "we need to reorganize the way we store data" is going to be met with blank stares if not full on outrage.
I think order by price might be ordering by the lowest available price at time of index. This could be an out of date price if the index isn't updated when other pricing is updated. It almost certainly doesn't include shipping, as that can vary depending on where the item is and where you are so it can't be indexed. Filtering by seller doesn't change the sorting, in my experience, and I think neither does filtering by condition.
So, if someone is selling a used item in poor condition with maximum shipping, that product is going to sort as a low priced item.
At least, that's my reverse engineered understanding.
> almost flawless experiences from amazon shopping
In my opinion its too fuzzy, which is usually the opposite problem I have with website searches (e.g. searching a news site for keywords of an article I know exits, but it's taking me too literally)
If I search for "iPhone 14 Pro case", I don't want to see cases for iPhone 15 __, or non-pro models. I've (to my own fault) bought way too many of the wrong product because I search for a specific model and don't read the title before ordering, only to realize that Amazon didn't give me exactly what I typed in.
what kind of stiff are you buying you call this "flawless"? In my experience the Amazon search is worst there is. Search for "AAA batteries" and it will offer you AA and even N ones. Why on earth would anyone want that?
They even got the basics wrong. The other day I was searching for power bank under $5. Instead, many listings was $10+. How hard is it to get this right?
Because of horrible search quality I actively avoid Amazon when I can.
A big reason why is Search today is an AI Librarian. That AI may be more or less smart, but at the end of the result that's what people mean when they say Search. They certainly do not mean a robust query language that will allow them to iteratively refine down a subset of matching documents in a corpus until the desired document is found.
I think if you remove the cariacature that approach isn't terrible, assuming there's no more suitable technology. Why not have a basic search and iterate, including iterating the technology choice. You can hide it behind a feature flag if you don't think it's ready.
The problem is that the sort of stuff you need to tune a search engine to actually perform well doesn't look like work. It simply can't be broken down into deliveries that provide incremental business value, and throughout the entire process it's going to look like a bunch of fiddling with numbers with nothing to show for it.
Does any product feature that requires care and maintenance easily fit the model of delivering incremental value?
But for a search engine, showing incremental value should be easy if you have a real product reason to do it. If you expect search to help people find content, then you look at statistics on engagement --- in aggregate did people use search, does it look like they found what they wanted, did they return and use search again. In specific, which queries seem popular and provide good results, which queries seem popular and don't provide good results. You provide incremental value by moving queries from the second to the first category, where possible. If you can kind of classify some of the less popular queries, there's value in improving results for those too, but classification is also hard.
Sometimes the numbers are easy... I participated in an application of search where a support request hit the FAQ before submission --- if search works, there should be fewer tickets and especially fewer tickets where the user can easily solve their own problem. Search in a shopping context should lead to more sales. Etc.
I think you may miss how many technology projects fail in companies. A company may have no internal knowledge or skill to make search better, so they get an outside contracting group that produces a shitty product that doesn't improve sales and costs an extraordinary amount for the lack of results.
Quite often the company won't make any more in those long tail sales, but instead better placement of the products they buy in mass.
Almost every hijacking of keyboard and other default behaviors of our peripherals is lazy and arrogant UI. One exception is auto focusing on a search input element where that view is only used for that search box. Even then it's annoying because my browser back/forward via keyboard (cmd-left, cmd-right) gets hijacked by the focus.
All this is unavoidable. But it's also temporary. This phase of search won't last since we're going to replace it with just SKU + inventory plus some fulltext search plus an LLM. An LLM browser assistant could probably solve this problem quite well.
Inventory will never be well-tagged. The process change to ensure it would slow down procurement.
Ideally, they'd expose an inventory XML list and we'd just use our LLM on it.
The hard part is how to incentivize by permitting them control over advertising. An alternative is the browser LLM just searching across the page (Arc Browser can do this).
One thing for sure is that shitty search algorithms in a page showing ads before search results, have the useful side effect of forcing users to refine search, exposing them to more ads before they find what they were searching for.
Tagging ins tedious. Especially when it’s not you who searches but lots of other people, all with their specific ways of describing what they are looking for.
And yes, the results are between amusing, annoying and infuriating.
The “alcohol” example finding mostly comes from alcohol free products advertising this feature in their name - no need to name your regular beer or wine “with alcohol”, at least not in the product name.
tagging doesn't work period because people will tag things to try to get their product in front of your eyes and add any tag they think is popular entirely unrelated to their "product".
Leaving myself a note here for when I'm back behind a computer.
I'm wondering how well something like GPT-4 generates tags with a product description/product image. Be interesting to see its accuracy and comprehensiveness.
On officedepot.com when searching for a produect I have found no way to filter by "in stock at my chosen store" you have to search, then click through each individual item.
The one and only reason I am considering office depot as a place to spend my money is because they're local and I can go get whatever cable or thing I need, right now.
Bad for the store you mean? Because the customer doesn’t convert (buy) something if it says “hurry only 1 left in stock” because everyone knows that store inventory is inaccurate and they ultimately don’t have 1 in stock, they have 0 in stock?
A. [explicit claim; humorous] Analytics confirm that, if you show people whether products are in stock locally, they won't buy those products on your website.
B. [by implication] This is bad, because it "lowers" your sales. So you can't display this information.
C. [by implication] The retailers are making a mistake - the reason people don't buy products on the website when availability information is displayed is that they go to the store and buy the product there.
D. [summing up] If you measure something you don't care about, you will end up doing things that make you look like an idiot.
That's not a good explanation. Either GGP was being sarcastic or is corpo-brained. GP was confused because, unfortunately, both are equally likely these days.
Is it good for conversion when I give up out of frustration at your broken store and decide to just get the thing on Amazon instead?
This isn't just hypothetical: for a while I was using QFC (Kroger) for grocery pickup, but after one too many instances of adding an item to my cart and then only finding out at checkout that a third of my items weren't actually available, I said to hell with it and switched to doing all that at Whole Foods instead, since their inventory counts are actually correct.
I did just check this, and I found it fairly easy to filter by in-stock at my chosen store. Did you set a preferred store then filter by the "store pickup" option? Doing that, I returned results for chair mats with the option to check other local stores.
This drives me crazy. Local stock search is different on each store and most don't work correctly.
Try searching on Walmart to find local stock. You have to click "In-Store", then change the Fulfillment Speed to today and tomorrow to find things that are in stock. Except now they also show results from other stores, like Advance Auto Parts, and there's no way to filter that garbage out.
I tried searching Academy Sports for local stock. The list of items says it's in stock but when you go to the individual item it says it's OOS.
Lowe's and Home Depot local stock does seem to work reasonably well, for now.
Love this article. Worked as a search relevance consultant for years and these are exactly the problems we work on. Most of the time fixing search is a thankless job. People expect it to work and when it does they take it for granted, but when it doesn’t it damages your reputation and brand.
If you want to get started fixing website search relevance I recommend the books “Relevant Search” by Turnbull and Berryman, and “AI Powered Search” by Grainger, Turnbull, and yours truly. Both published by Manning.
Yep I work on Search as well -- both of these books are excellent.
Website search is... hard. A lot of the faceting still needs to be done by hand. I think there's probably some opportunity for LLMs to make some sort of autotagging/categorization easier, but there will likely still need to be a human in the loop to verify.
Great recommendation on Relevant Search. I worked on an ecom site trying to improve search relevance for years, and it’s an incredibly difficult and challenging problem. Especially because it’s difficult to measure success, and execs would constantly come up with “hey why does this query not return exactly the results I expected” which led to a lot of resources spent optimizing a handful of queries based on executive input instead of based on query frequency and query result conversion metrics.
I'm not trying to fix search so what's the tldr on why it's so hard to do with such constrained databases? The Nintendo one seems insanely egregious. Is this not just a symptom of corporate laziness rather than search problem complexity?
The searchable content may be constrained, but the queries aren’t ;)
Search is hard because you need to anticipate and model the language of all potential searchers and the content.
There’s also lots of ambiguity because you’ll only get one or two keywords from the searcher without any other context, and you need to take into account trends and content quality and metadata.
Also, in lots of cases search is bad because the product team either doesn’t know or doesn’t care.
The reality is that yes there is a long tail of searches people might search for, but for popular searches like “laundry detergent” you can check them manually. You get a lot of coverage just by fixing bugs from the top N queries.
Thanks for the recommendations! I also work on a decently large ecom site and search experience is a difficult problem - especially in aftermarket automotive.
Years ago, I cobbled up a simple search engine for some FOSS web forum software, mainly for people who were running private instances behind login walls -- i.e., stuff that public search engines couldn't crawl.
Someone complained that for public instances, Google worked better (this was back before Google started to suck).
My response was that if I could write a better search engine than Google in my spare time, I'd be living a lifestyle considerably more luxurious than the one I actually had. :-)
200 comments
[ 3.3 ms ] story [ 243 ms ] threadThe amount of high level product people that can't demo their own product and cannot find features they hype up in slide decks is depressing.
The profit margins on selling products are abysmal for e-commerce. The profit margins on ads are amazing.
I'll be the first admit I don't have a ton of respect for the "product owner" role which has morphed largely into Tom Smykowski from Office Space[0] but the places I've seen them be actually effective and net positives for an organization are where they have the reach and authority to demand things like this be fixed, regardless of what department is "in charge." IMO a product owner who doesn't do this is refusing to do one of the few useful product owner roles.
[0] https://www.imdb.com/title/tt0151804/characters/nm0726223
If I use my product and hate it, I make it better. Knowing they some other department sucks for years is no good.
I think all this boils down to companies not really giving a flip.
The work of product person shouldn't stop at the idea phase.
Especially to such an extent as the umbrella example.
What's the failure mode here? Microservices that "own their own data"?
Fuzzy matching and boosting often trip people up or lead to folks shooting themselves in the foot relevancy-wise.
If you want really great results, you need to spend the time crafting your query, dealing with synonyms ("pop" vs. "soda"), stemming, typos, negative boosts ("non-alcoholic", in the example) etc.
It's not necessarily hard, just often forgotten or not included in the initial scope.
You can get better results with a 5minute tsvector in Postgres, At least it will show the words you typed in in the results!
And yes I understand it may be a different "scale" of products, but do people not test these things?
The only search tools that actually give me great value are history shell with fzf on the terminal, and grep (and similar like ag or ripgrep) in combination with a fuzzy search frontend (similar to piping grep to fzf).
I don't know where I'm going with this. I guess searching just sucks now.
If you want to get academic with it, you could say something about revealed preferences vs stated preferences (they asked for Coke but they really wanted Pepsi), but I really think they’re just throwing as many suggestions at the user as possible in the hopes they stay engaged on the site and eventually click buy on something.
Good website search is hard. Funnily in some respects it's harder than web search, since there are no popularity signals a la pagerank. (There could be such signals, but you have to work hard to create them.)
You're right though, this is incredibly dumb design.
In practice people type what they think will turn up the results "news", say, and don't look at the results until they're finished typing.
Another annoying aspect to it is that things enter your history as you browse, so the number of characters to hit that point where it shows what you want might shift around. Which totally destroys muscle memory. This isn’t the browser’s fault of course, but it would be nice if they could design around it.
a, ad, adv -> first result is adventofcode.com/2023
adve -> first result is adventofcode.com/2023/day/1 [honestly, this one baffles me, but I guess I went back to day 1 at some point after typing `adve`]
adven, advent, advento -> first result is reddit.com/r/adventofcode
adventof -> back to adventofcode.com/2023
I actually *appreciate* this behavior; it means I can better muscle-memory a certain number of letters and go down to the first result when 2 sites have similar names (very annoying for twitter & twitch btw).
So I don't think the problem is that weights can change as you type more letters; but that an entry can disappear altogether when there are no other suggestions.
And if they included the full URL, sheesh… you could effortlessly query against search params, path segments, fragments, and origins all in one intuitive interface.
‘It’ suggests It, It’s, and Its.
‘Its’ suggests Its and Itself.
Given that the apostrophe requires accessing a different keyboard overlay, and that only two suggestions are presented (leaving one suggestion slot empty), why not just include the contraction “It’s” in the suggestions?
Like I’ll search the full word “passwords”, pausing after each letter, and not get the entry titled “passwords” in my results the whole time. Back out, do it again, this time it’s there by the time I type “pas”.
This used to work perfectly, but has been inconsistently broken for years now. Most recent time I saw it was a week or two ago, so it’s still happening. What’s most baffling to me is that a search over such a relatively static dataset (app settings are in there, but those don’t get added or modified often) can be non-deterministic.
But for example, if you start a sentence with “We’re,” it will often take out the apostrophe… until you reach the end of the sentence, and then when you add the period I guess it’ll analyze the sentence and figure out that you actually meant what you wrote, and put it back. It does highlight the corrections, which is really nice. At this point, I just ignore it until I’m done with the sentence and then go back and check.
Just try to get to display settings on windows these days through the start menu.
I lost hope it and other issues would be fixed and moved to Chromium on Android.
We just told our customers to google the part number. I don’t know how these places stay in business.
eBay too, it's OK (compared to Amazon) but most of the items on a search are always "mousepad [or whatever] suitable for" the laptop you searched for. Why eBay don't clean up their categories and ban sellers I don't know, it makes the site so much worse to use.
I wonder if anyone is _ever_ hoodwinked by the vast majority of sites that sort by Most Expensive by default?
It's all part of enshittification presumably, these sites monitor so many metrics it must surely be a profit generator to make search frustratingly bad.
The site search picks up unrelated items, misses others, and so on. I would complain about the inventory management/substitutions too - but since they also have in store shoppers using the same inventory I guess they can’t always be 100% about what’s on the shelf.
You’d think a chain that size could do better, but I doubt they have much incentive. The competition is minimal and not any better.
At the end of the day, gathering and aggregating all of that crap into a stable API is a chore unto itself. Multiple people's jobs. Updating products, associating taxonomies, in-store locations, duplicate locations, optimizing pickup routes, etc etc etc. There is a lot of grunt work here that can't be entirely automated away when dealing such a dynamic business.
Now you need to build a website that takes all of this data and somehow displays it in a format that is comprehensible and accurate for consumers. Showing relevant recommended products, search keywords, cache and image management (for potentially resizing dynamic supplier photos). Then think about the cart and checkout, two areas and Shopify and Amazon have devoted literal billions of dollars to optimize. Your volume is big enough that using an off the shelf payment provider is off the table; not to mention that you don't have the margins anyway. Have you thought about mobile apps and the development effort required there as well?
Grocery stores are hard enough as is but doing them online is a totally different beast. There's a reason that all of the "Kroger" brands use the same exact website with a different logo in the corner. The only players in this space doing it reasonably well are huge conglomerates with money to burn on building the required infrastructure to build the audience. Actually, I just looked up Harris Teeter and they appear to be a Kroger brand as well. TIL. I would say that "Kroger" is one of the better ones. Whole Foods is also doing decently well if you can afford the price premium, but I realize for most people that isn't a luxury they can afford.
The best way to get them to improve is to keep using the service and demanding better (or use a competitor if available). These departments have KPIs. If you get the wrong product, file a complaint at the end of your order process and mark any NPS surveys appropriately. If fulfillment has a lot of bad data from elsewhere in the organization driving their performance scores down, then they are incentivized to get it fixed. I know it sucks, but the only other alternative is to not use it at all and risk is going away forever.
I had waited so long for rating filters, my kids became adults.
However, I was wondering what Netflix got after 17 years of development at $215k-$700k per dev - and now I know.
That might make sense in an abstract way, but no, that's not what people expect. If I'm searching for something at Netflix, I don't want the answer to "does it exist?", I want the answer to "do you have available for viewing?".
In the more general sense of the search topic, I don't want a search to return things they don't have. If it's out of stock, say so prominently. Don't make the user click through to the item itself only to see "out of stock".
> I've been impressed by Netflix search. I would search a name and they do not have that movie, but would suggest a mix of similar movies, movies with the same actors or director.
An excellent, early pirate feature. Good on Netflix for adopting it.
My main issue with the input search is the TV keyboard. There was a Netflix hack day that created a circular keyboard like a dial w/ the selector a joystick in the middle, so pressing up to 12 o'clock was A, 12:05 was B, etc. And the joystick would return to center per selection! It was everything I ever wanted in a thing, and now I can't even find the article on it.
On the other end of the spectrum, try Digikey:
https://www.digikey.com/en/products/filter/embedded/microcon...
or try McMaster Carr:
https://www.mcmaster.com/products/screws/socket-head-screws~...
Functional, well-tagged, data-driven parametric search. Rockauto.com sells auto parts, which requires a whole different type of search, but they deliver a pretty impressive product as well.
I suspect that much of this discrepancy is because there's enough of a chance that someone grocery shopping and looking for lentils will think "Oh, I need pancake mix too" and click on the ad. Meanwhile, someone looking for a specific grade of M8x25 socket-head cap screws or a low-power Cortex-M0 microcontroller with a particular set of peripherals is unlikely to abandon their work to add a hose clamp or electrolytic capacitor to their cart instead.
Market forces seem incapable of delivering a product with a better user experience if that hurts the bottom line even a little bit, they're doing gradient descent to local minima and powerless to say "no" to finance and climb out of the awful product that results.
You can't look at one narrow usage path to judge an interface. You can only customize the interface so much for specific use cases before it becomes confusing as hell to use because it lacks consistency between tasks.
The problem is obviously that the data isn't tagged. But Costco is huge, profitable and doesn't have a huge inventory, so why can't they tag it? And if the tags are so incomplete, maybe it's better not to include them?
Well, no, if I ask for rice, the odds are overwhelming that I'm looking for rice, in which case chicken and rice canned soup is a totally inappropriate response. It would make as much sense as returning alcohol-free wine when asked for "alcohol".
You should be able to unearth the soup with a partial match. What makes the alcohol/alcohol-free problem is that alcohol free is one token that can't be subdivided.
Moving the argument into the realm of things that matter, chicken and rice soup is not a product that can be called "rice", and therefore isn't an appropriate response to a search for "rice". The title is irrelevant; if you're stocking rice with a fancy name and its title doesn't include the word "rice", that product still should show up in a search for "rice".
Yes: chicken and rice, rice cakes, rice paper, cauliflower rice, rice cooker, rice krispies, rice university jersey, mochi, risotto.
No: licorice, ricetta tiramisu.
As I said, that is inconsistent with your simultaneous position that alcohol-free wine is not a match for "alcohol". Either they both are matches to their respective queries or they both aren't; there isn't a difference between the two examples. This is true from the perspective of dumb text searches, but it's equally true from the perspective of correct linguistic parsing.
That rice is part of the listed title for chicken and rice soup is a coincidence; the argument for returning it also tells you that you should return the same soup when asked for "tomatoes", if tomatoes are an ingredient in the soup.
Your "yes" and "no" examples don't seem to work. What rule excludes "licorice" that would include anything containing the raw textual string "rice" without regard to whether any rice is involved? Rice paper, rice cooker, cauliflower rice are all purely textual matches. Mochi is a purely semantic match!
Oh! Chicken and rice soup, I could go for that. Cart.
It's not even a problem... it's an unfounded assumption that alcohol-free wine wouldn't ever be relevant to the user at all. If the only place open on a holiday was Walmart that I didn't realize didn't sell booze, and I needed some kind of booze to make a pan sauce or a braise, I wouldn't necessarily think to search for non-alcoholic booze-- but it would be a very useful substitution for me, and not by accident... they're two versions of the same product. I'd much rather see that they had non-alcoholic beer and wine in my search than completely empty search results.
This is why competent companies hire designers to work on interactions like this rather than leaving it up to the implementing developer's gut instinct about user needs. If only the tagging could keep up.
However it shows restaurants that paid to be featured at the top of the results.
So the search results will often contain restaurants that are multiple hours away.
I also noticed the same in several other website where "relevance" trumps my sort request (e.g., price, distance).
Search along route sometimes works better and sometimes also zooms out.
My gf (who used to work in the sector) says a lot of people like to sit in front of the TV and “watch what’s on” — one of the reasons broadcast TV still exists.
You see this with sites like Netflix becoming decreasingly useful, urging you to simply click on something rather than consult your curated watch list. Some of them now hide your watch list!
She suggests the same applies to shopping: some people just want to push the buy button, which is well known in brick and mortar retail, especially low margin ones like grocery shops: end caps, mid shelves and all sorts of other strategies are used to get you to either buy on impulse or buy the version they’d rather sell of what you have in mind.
So for these sites the same applies: show a few options and the customer will simply pick one. Spam the tags, DWIM the customer’s ill-formed, uninformed, or simply unsatisfiable query, and see if the punter buys.
For nerds like us this is a pessimization. But perhaps for the majority this works better.
Mentally when I'm at a store and I have 3 options to pick from I seemingly enjoy the process better then if I had 6 or 10. Eventually the picking itself becomes work. If it's a topic I enjoy like motherboards or RAM then all the choices are fine, I can spend hours on that, but if I'm picking tomato sauce for a quick and sloppy dish I don't want to spend the time selecting.
Maybe if I'm on Netflix to watch what's on, that's fine. But e.g. on Amazon, I find "search" so adversarial that I often close the tab and go somewhere else, or give up.
The only other enterprise application I am aware of is applying the homework cheating app to customer facing roles that require email/copy generation.
No, with LLMs you're getting both, because a. LLMs need the human-curated data to ingest and b. after training the model is going to get RLHF.
Also, a square kilometer of office space still uses much less electricity than a square kilometer of GPUs.
* PO: We need search!
* PM to architect: Google how to implement search!
* Architect: googles "best search library 2023 java" Aha! Let's use Elasticsearch!
* PM to Team: Alright, I need a search feature next sprint. Architect googled for 5 minutes and says we should use elasticsearch!
* Team: Alright, integrating elasticsearch in a sprint should work. Bob, you work on loading the data, Eve, you work on rendering the search results.
* Team (2-3 weeks later): Another successful value delivery! Alright, what's next PM?
* PM: PO says we need a chat bot...
If it's not obvious why this is a problem, getting search to work well requires a lot of tweaking and tuning. You can't just slap a search engine into a product and expect it to work well. It's hilariously incompatible with the assembly line style of work that many software outfits employ these days. It also requires deep understanding of what a search engine does and how its algorithms work.
It helps we have two internal departments whose primary jobs are data management and analysis. (Real statistics, not “AI”.)
- Typesense (https://github.com/typesense/typesense)
- Algolia
- Google Programmable Search Engine (https://programmablesearchengine.google.com/about/)
We have gotten used to almost flawless experiences from amazon shopping. Google search finds (or used to) results sometimes almost like magic, etc. The thing is, that's the core business. These companies invent new technologies and have huge teams because doing this is so hard.
Now take a random supermarket chain. Their knowledge is about physical stores. Their core business has taught them where to open a store and how to arrange things in it so that they maximize their sales in that environment. It's very hard and it takes a very long time to shift to an online model. You have to find people with the right competencies and the right leadership to convince the company to do this and this is actually very hard to do.
Look at the categories example. That, to me, screams backend database. The company has invested a lot of money into building business intelligence on top of their physical stores. That organization screams "perfectly curated data warehouse" and I imagine suggesting something like "we need to reorganize the way we store data" is going to be met with blank stares if not full on outrage.
It's pretty good, but there are some longtime flaws that blow my mind, such as the order of results when sorting by price. It never makes sense to me.
So, if someone is selling a used item in poor condition with maximum shipping, that product is going to sort as a low priced item.
At least, that's my reverse engineered understanding.
In my opinion its too fuzzy, which is usually the opposite problem I have with website searches (e.g. searching a news site for keywords of an article I know exits, but it's taking me too literally)
If I search for "iPhone 14 Pro case", I don't want to see cases for iPhone 15 __, or non-pro models. I've (to my own fault) bought way too many of the wrong product because I search for a specific model and don't read the title before ordering, only to realize that Amazon didn't give me exactly what I typed in.
what kind of stiff are you buying you call this "flawless"? In my experience the Amazon search is worst there is. Search for "AAA batteries" and it will offer you AA and even N ones. Why on earth would anyone want that?
They even got the basics wrong. The other day I was searching for power bank under $5. Instead, many listings was $10+. How hard is it to get this right?
Because of horrible search quality I actively avoid Amazon when I can.
But for a search engine, showing incremental value should be easy if you have a real product reason to do it. If you expect search to help people find content, then you look at statistics on engagement --- in aggregate did people use search, does it look like they found what they wanted, did they return and use search again. In specific, which queries seem popular and provide good results, which queries seem popular and don't provide good results. You provide incremental value by moving queries from the second to the first category, where possible. If you can kind of classify some of the less popular queries, there's value in improving results for those too, but classification is also hard.
Sometimes the numbers are easy... I participated in an application of search where a support request hit the FAQ before submission --- if search works, there should be fewer tickets and especially fewer tickets where the user can easily solve their own problem. Search in a shopping context should lead to more sales. Etc.
Quite often the company won't make any more in those long tail sales, but instead better placement of the products they buy in mass.
Inventory will never be well-tagged. The process change to ensure it would slow down procurement.
Ideally, they'd expose an inventory XML list and we'd just use our LLM on it.
The hard part is how to incentivize by permitting them control over advertising. An alternative is the browser LLM just searching across the page (Arc Browser can do this).
And yes, the results are between amusing, annoying and infuriating.
The “alcohol” example finding mostly comes from alcohol free products advertising this feature in their name - no need to name your regular beer or wine “with alcohol”, at least not in the product name.
I'm wondering how well something like GPT-4 generates tags with a product description/product image. Be interesting to see its accuracy and comprehensiveness.
The one and only reason I am considering office depot as a place to spend my money is because they're local and I can go get whatever cable or thing I need, right now.
A. [explicit claim; humorous] Analytics confirm that, if you show people whether products are in stock locally, they won't buy those products on your website.
B. [by implication] This is bad, because it "lowers" your sales. So you can't display this information.
C. [by implication] The retailers are making a mistake - the reason people don't buy products on the website when availability information is displayed is that they go to the store and buy the product there.
D. [summing up] If you measure something you don't care about, you will end up doing things that make you look like an idiot.
This isn't just hypothetical: for a while I was using QFC (Kroger) for grocery pickup, but after one too many instances of adding an item to my cart and then only finding out at checkout that a third of my items weren't actually available, I said to hell with it and switched to doing all that at Whole Foods instead, since their inventory counts are actually correct.
Try searching on Walmart to find local stock. You have to click "In-Store", then change the Fulfillment Speed to today and tomorrow to find things that are in stock. Except now they also show results from other stores, like Advance Auto Parts, and there's no way to filter that garbage out.
I tried searching Academy Sports for local stock. The list of items says it's in stock but when you go to the individual item it says it's OOS.
Lowe's and Home Depot local stock does seem to work reasonably well, for now.
If you want to get started fixing website search relevance I recommend the books “Relevant Search” by Turnbull and Berryman, and “AI Powered Search” by Grainger, Turnbull, and yours truly. Both published by Manning.
Website search is... hard. A lot of the faceting still needs to be done by hand. I think there's probably some opportunity for LLMs to make some sort of autotagging/categorization easier, but there will likely still need to be a human in the loop to verify.
Search is hard because you need to anticipate and model the language of all potential searchers and the content.
There’s also lots of ambiguity because you’ll only get one or two keywords from the searcher without any other context, and you need to take into account trends and content quality and metadata.
Also, in lots of cases search is bad because the product team either doesn’t know or doesn’t care.
That's the definition of a table stakes feature. It's not as if you can not implement it and people won't notice!
Someone complained that for public instances, Google worked better (this was back before Google started to suck).
My response was that if I could write a better search engine than Google in my spare time, I'd be living a lifestyle considerably more luxurious than the one I actually had. :-)