- Baymard's benchmark of more than 170 sites found 56 percent fail to adequately support what people type into a search box, and the failure rate is worse on mobile than on desktop.
- The failures are not spread evenly. Exact product names fail on 12 percent of sites, while queries describing a use case fail on 43 percent and queries about your returns policy fail on 66 percent.
- Keyword search and semantic search fail on opposite query types, which is why serious search stacks run both and merge the two ranked lists rather than choosing one.
- Most of what makes a small shop's search bad is product data, not the search algorithm. A model cannot match a word that appears nowhere in your catalogue.
- Measure the share of searches returning nothing and the share of searchers who leave from the results page. Both are cheap to collect and neither improves by itself.
Somebody lands on your shop, ignores your carefully built navigation and goes straight for the search box. They type "jumper". You sell sweaters. The page returns nothing, offers a paragraph of search tips nobody reads, and they leave. Nothing in your analytics will tell you a sale was lost, because from the outside it looks like a visitor who bounced.
Ecommerce site search is the most under examined surface in a small shop. It gets a fraction of the attention product photography gets, and it is used by the visitors who were closest to buying. Nielsen Norman Group's long running work on the subject found that more than half of users are search dominant, heading for the search field on arrival rather than exploring the way you laid the site out. The people most likely to buy are the ones who never see your category pages.
How broken is site search, really?
Badly enough that it is the norm rather than the exception. Baymard Institute benchmarked more than 170 sites and apps against thousands of usability scores, and reports that 56 percent of sites fail to adequately support users' search needs.
The breakdown by device is the part worth sitting with. Baymard grades 46 percent of desktop sites as mediocre or worse on search, rising to 58 percent on mobile sites and 64 percent in apps. Search gets harder exactly where typing is hardest and patience is shortest.
If you have ever assumed your own search is fine because you can find things with it, that assumption deserves a test. You search using the vocabulary of the person who wrote the product titles. Your customers do not have that advantage.
What are people actually typing?
Not what most catalogues are built to answer. Baymard sorts real search queries into distinct types and reports how often sites fail each one, and the spread between them is the single most useful thing a shop owner can know about their search box.
The table below sets Baymard's query types and failure rates against the retrieval method that handles each type well. That second column is my own mapping rather than Baymard's, drawn from how lexical and semantic retrieval behave, and it is where the practical decision lives.
| Query type | Example | Sites failing it | Handled well by |
|---|---|---|---|
| Exact | A model number copied from elsewhere | 12 percent | Keyword matching |
| Product type | "sandals" | 20 percent | Either, given clean categories |
| Symptom | Describing a problem, not a product | 37 percent | Semantic matching |
| Feature | Colour, material or size inside the phrase | 39 percent | Structured attributes and filters |
| Use case | For an occasion, a place or an activity | 43 percent | Semantic matching |
| Compatibility | An accessory for something already owned | 44 percent | Structured product relationships |
| Abbreviation and symbol | Shortcuts such as mm, or a dash | 54 percent | Keyword matching and normalisation |
| Non product | "return policy" | 66 percent | Neither, unless you index your own pages |
Read down the third column and a pattern appears. The queries sites handle worst are the ones that do not name a product: a symptom, an occasion, an abbreviation, a policy question, and that last category usually belongs in a help center written from real support tickets rather than in the product index at all. The queries sites handle best are the ones where the customer already speaks your catalogue's language, which is to say the customers who needed the least help.
The worst row on that table is also the cheapest to fix. Two thirds of sites fail when somebody types "return policy" into the search box, and they fail for a mundane reason: the search index contains products and nothing else. Your shipping page, your returns page and your size guide are not in it. Adding them is usually a configuration setting rather than a project, and it removes a whole class of support email at the same time, because the customer who cannot find your returns policy writes to you instead. Worth checking today: search your own shop for the word refund and see whether anything comes back.
What does semantic search actually do differently?
It compares meanings rather than words, by turning each product and each query into a list of numbers and measuring the distance between them. Google's documentation describes embeddings as high dimensional numerical vectors representing a piece of text, positioned so that related concepts sit close together and unrelated ones sit far apart.
The practical consequence for a shop is that the word "jumper" can land near the word "sweater" without anyone having written a synonym rule, because the two words are used in similar contexts across the text the model learned from. The same mechanism is what lets "something warm for a cold office" return cardigans.
It is not free of failure modes, and the failures are unfamiliar rather than absent. A keyword search that finds nothing tells you plainly that it found nothing. A semantic search always finds the nearest thing in the catalogue, whether or not the nearest thing is any good. Ask for a product you do not stock and it will confidently return the closest neighbour, which from the customer's side looks like a shop that did not listen.
Should you replace keyword search with AI search?
No, and the people who build search infrastructure for a living do not either. The standard answer is to run both and merge the results, which the industry calls hybrid search.
Elastic's explanation of why the two methods are combined is blunt about the trade. Lexical scoring is precise when the query terms match the document terms and fails when the same thing is expressed differently, missing "shoes" for a search on "athletic footwear". Semantic retrieval finds conceptually similar items regardless of shared words, and introduces noise when the context is ambiguous. Merging the two ranked lists, in Elastic's case with a method that rewards items ranked highly by either approach, keeps the precision and adds the recall.
For a merchant the takeaway is a purchasing question, not an engineering one. If a search vendor tells you their AI search replaces keyword matching, ask what happens when a customer pastes in a part number. If a platform's built in search is pure keyword matching, the gap you are feeling is on the symptom and use case rows of the table above.
Before you evaluate any vendor, collect twenty real queries from your own logs that returned nothing, and twenty that returned results nobody clicked. Test every demo against those forty queries. A demo run on the vendor's sample catalogue tells you nothing about yours.
Why better search often does not help
Because search can only match text that exists. This is the part that gets skipped in every conversation about upgrading a search box, and it is where most of the available gain sits for a shop with a few hundred products.
Consider a customer searching for a waterproof jacket for cycling. If your product description says "technical outerwear, three layer construction, taped seams" and never uses the words waterproof or cycling, then no retrieval method saves you. Semantic matching improves the odds, because taped seams and waterproof do sit near each other, but you are asking the model to infer what you could simply have written.
Catalogue text written for a supplier's spec sheet is the usual culprit, and it is the same underlying problem as the one that makes product feeds get rejected: attributes that live in a human's head rather than in a field. We went through that in detail when looking at what a feed rejection reveals about your product data, and search punishes the same gaps more quietly, because there is no rejection email.
Three fixes usually outrank buying anything. Put the words customers use into the product text, not only the words your supplier uses. Fill the structured attributes, because the feature and compatibility rows of that table are answered by fields rather than by prose. And build a synonym list from your own zero result log, which costs an afternoon.
What does AI search cost to keep running?
More than the subscription line suggests, and in a shape that catches small catalogues out. A keyword index is cheap to keep current because adding a product means adding its words. Vector search has a second step: every product has to be converted into an embedding, and that conversion has to happen again whenever the text changes.
For a catalogue that changes twice a year this is a non issue. For a shop that edits descriptions weekly, adds seasonal lines and runs a supplier feed that rewrites titles overnight, it becomes an operational question. Either the index refreshes on every change, which costs money per product, or it refreshes on a schedule, in which case your search knows about a version of your catalogue that is a day or two old. Vendors are rarely upfront about which one you are buying, and the answer determines whether a product you added this morning is findable this afternoon.
Pricing usually attaches to one of two things: the number of records indexed, or the number of searches served. Those meter very differently. A shop with 300 products and heavy traffic wants record based pricing. A shop with 40,000 supplier listings and modest traffic wants query based pricing. Working out which side you sit on before the demo saves an awkward conversation at renewal, and it is the same arithmetic we walked through for assistant costs in the gap between an estimated AI bill and the real one.
There is a latency budget too. Search runs while a customer waits, so anything that adds a round trip has to earn it. A hybrid setup runs two retrievals and merges them, which is fine at small scale and worth measuring rather than assuming.
What should a no results page do?
Offer a way forward rather than an apology, which about half of sites still fail to do. Baymard found that close to 50 percent of sites handle no results pages in a way that leads to abandonment, and that the common reflex of printing search tips is close to useless because people do not read them and they hand the visitor a reason to leave.
What Baymard found does work is concrete: link to categories that relate to the words in the query, suggest alternative searches with product previews, show recently viewed items, put your contact route in plain sight, and surface best sellers so the page proves the catalogue is not empty. That is five options rather than a dead end, and none of them require an AI product.
There is a behavioural reason this matters more than it sounds. Nielsen Norman Group observed that when a search failed, most people concluded the information was not available rather than trying a different phrasing. Your customers will not reformulate. They will decide you do not sell it.
How do you know if it is working?
Two numbers, both of which you can collect without new software, and neither of which most shops look at. The first is the share of searches that return zero results. The second is the share of sessions that end on a search results page.
Zero result rate is the diagnostic. Every one of those queries is a customer telling you, in their own words, something your catalogue does not answer. Read the actual list rather than the percentage: it will contain product names you do not stock, which is market research, and product names you do stock under a different word, which is free money.
Exit rate from the results page catches the subtler failure, where search returns forty things and none of them is right. That is the signature of semantic matching without precision, or of a relevance ranking that sorts by whatever the platform defaults to. A shop that fixed its zero results and left this one alone has moved the failure rather than removed it.
A third number is worth adding once the first two are under control: how often a search leads to an add to cart, compared with how often browsing does. Search sessions should convert better, because search is an expression of intent, and a conversion gap between searchers and browsers is the clearest signal that the box is broken. If yours do not, the search box is actively costing you the visitors most ready to spend. The same intent logic drives what customers expect from your product pages once they arrive, which we looked at in how product pages read to machine and human shoppers.
Where this leaves a small shop
The order of operations matters more than the technology choice. Fix the catalogue text and the structured attributes first, because they improve every retrieval method at once and they improve your feeds and your recommendations along the way. Then fix the no results page, which is an afternoon of work against a failure affecting roughly half of all shops. Only then does the question of hybrid or semantic search become worth money.
When you do reach that question, the honest framing is that you are buying recall on the query types your catalogue does not literally answer, and paying for it with a system that never admits it found nothing. That is a good trade for most catalogues, and it is a trade rather than an upgrade.
None of this is exotic. A search box that understands the words your customers actually use, backed by product data that says what the product is for, is ordinary shop infrastructure. It is the kind of thing that should come with the storefront rather than being bolted on later, which is how we think about the search and catalogue layer in the stores built with our ecommerce website builder. The technology has been settled for a while. The catalogue text is still the part nobody wants to write, and it is still the part that decides whether any of it works. If your recommendations are also thin, the same data gap is behind both, as we found in making recommendations work with very little data.