Google quietly announced that Programmable Search (ex-Custom Search) won’t allow new engines to “search the entire web” anymore. New engines are capped at searching up to 50 domains, and existing full-web engines have until Jan 1, 2027 to transition.
If you actually need whole-web search, Google now points you to an “interest form” for enterprise solutions (Vertex AI Search etc.), with no public pricing and no guarantee they’ll even reply.
This seems like it effectively ends the era of indie / niche search engines being able to build on Google’s index. Anything that looks like general web search is getting pushed behind enterprise gates.
I haven’t seen much discussion about this yet, but for anyone who built a small search product on Programmable Search, this feels like a pretty big shift.
Curious if others here are affected or already planning alternatives.
UPDATE: I logged into Programmable Search and the message is even more explicit: Full web search via the "Search the entire web" feature will be discontinued within the next year. Please update your search engine to specify specific sites to search. With this link: https://support.google.com/programmable-search/answer/123971...
It's been clear for the last decade that we have to wean ourselves off of centralized search indexes if only to innoculate the Net against censorship/politically motivated black holing.
I can only weep at this point, as the heroes that were the Silent and Greatest generations (in the U.S.), who fought hard to pass on as much institutional knowledge as possible through hardcore organization and distribution via public and University library, have had that legacy shit on by these ad obsessed cretins. The entirety of human published understanding; and we make it nigh impossible for all but the most determined to actually avail themselves of it.
TIL they allowed that before. It sounds a bit crazy. Like Google is inviting people to repackage google search itself and sell it / serve with their own ads.
I know that duckduckgo uses Microsoft Bing Custom search and honestly it is a much more robust system since you don't have to worry about Google axing it. https://www.customsearch.ai
If its Motion for a Partial Stay is denied, or if it loses on appeal, then under this Final Judgement Google will be forced to offer syndicated "full web" search to Qualified Competitors
Thank you for contacting us via the Web Search Products Interest Form. We have received your feedback and are actively reviewing the specific use cases you shared.
We are writing to share important details regarding the transition plan and the available solutions for your search needs.
1. For Unrestricted Web Search: Future Web Search Service
For partners requiring unrestricted "Search the entire web" capabilities, we are developing a new enterprise-grade Web Search Service. As you evaluate your future needs, please be aware of the commercial terms planned for this new service:
Pricing: USD $15 CPM (Cost Per Mille / 1,000 requests).
Minimum Commitment: A minimum monthly fee of USD $30,000 will apply.
We’ll release more information on this service later in 2026. Existing 'Search the entire web' engines remain functional until January 1, 2027.
2. For AI & Advanced Search: Google Vertex AI
We strongly encourage you to explore Google Vertex AI as another option for partners who need enterprise search and AI capabilities across 50 or fewer domains. Vertex AI offers powerful capabilities for:
Grounded Generation: Connecting your AI agents to your own data and/or to Google Search to provide accurate, up-to-date responses.
Custom Data Search: Building enterprise-grade search engines over your own data and specific websites.
This solution is available now and is designed to scale with your specific application needs.
Clarification on Current Service Status:
While you evaluate which path fits your business needs, please remember the timeline for your current implementations:
Existing Projects: If you have an existing Programmable Search Engine configured to "Search the entire web," your service will continue to function until January 1, 2027. You have the full year to plan your migration.
New Projects: As of January 20, 2026, new engines created in the Programmable Search Engine admin console are restricted to "Site Search Only" (specific domains only).
Later in 2026 we’ll provide you with more updates regarding the new Web Search Service and the means to express your desire to use it to power your web search experiences.
"Google will discontinue third-party niche search engine access to full-web search" would be far clearer.
Given that the title supplied is effectively editorialised, and the original article's title is effectively content-free ("Updates to our Web Search Products & Programmable Search Engine Capabilities"), my rewording would be at least as fair.
HN's policy is to try to use text from the article itself where the article title is clickbait, sensational, vague, etc., however. I suspect Google's blog authors are aware of this, and they've carefully avoided any readily-extracted clear statements, though I'll take a stab...
Here's the most direct 'graph from TFA:
Custom Search JSON API: Vertex AI Search is a favorable alternative for up to 50 domains. Alternatively, if your use case necessitates full web search, contact us to express your interest in and get more information about our full web search solution. Your transition to an alternative solution needs to be completed by January 1, 2027.
We can get a clearer, 80-character head that's somewhat faithful to that with:
"Google Search API alternative Vertex AI Search limited to 50 domains" (70 chars).
That's still pretty loosely adherent, though it (mostly) uses words from the original article. I'm suggesting it to mods via email at hn@ycominator.com; others may wish to suggest their own formulations.
The 'Google Graveyard is real' sentiment captures something important: every dependency on a large platform is a loan that can be called in. The 34-million-document indie index project someone mentioned is the right response - own your core infrastructure. Easier said than done for whole-web search, but the same principle applies everywhere.
Are competing search indexes (Bing, Ecosia/Qwant, etc) objectively worse in significant ways, or is Google just so entrenched that people don't want to "risk it" with another provider (and/or preferences and/or inertia).
I suppose I'm asking whether this is actually a _good thing_ in that it will stimulate competition in the space, or if it's just a case that Google's index is now too good for anyone to reasonably catch up at this point.
The beauty about Google Programmable Search across the entire web is that it's free and users can make money by linking it their Adsense account.
Bing charge per query for the average user. Ecosia and Qwant use Bing to power their results, probably under some type of license, which results in them paying much less per query than a normal user.
I can manage fine with other search indexes for English language searches; weather that is because others got better or google got worse i cannot tell, though I suspect the latter.
But for searching in more niche languages google is usually the only decent option and I have little hope that others will ever reach the scale where they could compete.
Bing's index is smaller than Google's, and anecdotally I get fewer relevant results when using it, particularly from sites like Reddit that have exclusive search deals with Google.
Never build a product with core feature depending on a third-party, you will eventually get fucked up for sure. always have a 70:30 rule for revenue where 70% is core independent features.
This will significantly impact (quite possibly kill) Startpage and Ecosia, who are effectively white-label Google, right?
What alternatives are there besides Bing? Is it really so hard that it’s not considered worth doing? Some of the AI companies (Perplexity, Anthropic) seem to have managed to get their own indexing up and running.
Google has consistently ruined its search engine in the
last (almost) 10 years. You can find numerous articles
about this, as well as videos on youtube (which is also
controlled by google).
Not long ago they ruined ublock origin (for chrome; ublock origin
lite is nowhere near as good and effective, from my own experience
here).
Now Google is also committing towards more evil and trying to
ruin things for more - people, competitors, you name it. We
can not allow Google to continue on its wiched path here. It'll
just further erode the quality. There is a reason why "killed
by google" is more than a mere meme - a graveyard of things killed
by google.
We need alternatives, viable ones, for ALL Google services. Let's
all work to make this world better - a place without Google.
I keep seeing posts about how ~"the volume of AI scrapers is making hosting untenable."
There must a ton of new full-web datasets out there, right?
What are the major hurdles that prevent the owners of these datasets from providing them to third parties via API? Is it the quality of SERP, or staleness? Otherwise, this seems like a potentially lucrative pivot/side hustle?
> There must a ton of new full-web datasets out there, right?
Sadly, no. There's CommonCrawl (https://commoncrawl.org/) which still, sadly, far removed from "full-web dataset."
So everyone runs their own search instead, hammering the sites, going into gray areas (you either ignore robots.txt or your results suck), etc. It's a tragedy of the commons that keeps Google entrenched: https://senkorasic.com/articles/ai-scraper-tragedy-commons
Not directly covered by this blog, but for low cost and good performance the combination of gemini-3-flash with search grounding is hard to beat, at least for the many small experiments I use it for.
One thing touched upon in comments here: I never understood how it was proper for 3rd parties to scrape Google search results and reuse/resell them.
Really off topic, sorry, but I am surprised that more companies don’t build local search indices for just the few hundred web domains that are important to their businesses. I have tried this in combination with local (small and fast) LLMs and I think this is unappreciated tech: fast, cheap, and local.
I built many products on Google PSE (Custom Search). Results were nowhere near as good as regular Google, but still useful. I usually needed to use another library to get the DOM content anyway. But it still was solid for grounding/checking data.
45 comments
[ 3.6 ms ] story [ 73.3 ms ] threadIf you actually need whole-web search, Google now points you to an “interest form” for enterprise solutions (Vertex AI Search etc.), with no public pricing and no guarantee they’ll even reply.
This seems like it effectively ends the era of indie / niche search engines being able to build on Google’s index. Anything that looks like general web search is getting pushed behind enterprise gates.
I haven’t seen much discussion about this yet, but for anyone who built a small search product on Programmable Search, this feels like a pretty big shift.
Curious if others here are affected or already planning alternatives.
UPDATE: I logged into Programmable Search and the message is even more explicit: Full web search via the "Search the entire web" feature will be discontinued within the next year. Please update your search engine to specify specific sites to search. With this link: https://support.google.com/programmable-search/answer/123971...
I can only weep at this point, as the heroes that were the Silent and Greatest generations (in the U.S.), who fought hard to pass on as much institutional knowledge as possible through hardcore organization and distribution via public and University library, have had that legacy shit on by these ad obsessed cretins. The entirety of human published understanding; and we make it nigh impossible for all but the most determined to actually avail themselves of it.
TIL they allowed that before. It sounds a bit crazy. Like Google is inviting people to repackage google search itself and sell it / serve with their own ads.
https://dn710204.ca.archive.org/0/items/gov.uscourts.dcd.223...
Dear Programmable Search Engine user,
Thank you for contacting us via the Web Search Products Interest Form. We have received your feedback and are actively reviewing the specific use cases you shared.
We are writing to share important details regarding the transition plan and the available solutions for your search needs.
1. For Unrestricted Web Search: Future Web Search Service
For partners requiring unrestricted "Search the entire web" capabilities, we are developing a new enterprise-grade Web Search Service. As you evaluate your future needs, please be aware of the commercial terms planned for this new service:
Pricing: USD $15 CPM (Cost Per Mille / 1,000 requests).
Minimum Commitment: A minimum monthly fee of USD $30,000 will apply.
We’ll release more information on this service later in 2026. Existing 'Search the entire web' engines remain functional until January 1, 2027.
2. For AI & Advanced Search: Google Vertex AI
We strongly encourage you to explore Google Vertex AI as another option for partners who need enterprise search and AI capabilities across 50 or fewer domains. Vertex AI offers powerful capabilities for:
Grounded Generation: Connecting your AI agents to your own data and/or to Google Search to provide accurate, up-to-date responses.
Custom Data Search: Building enterprise-grade search engines over your own data and specific websites.
This solution is available now and is designed to scale with your specific application needs.
Clarification on Current Service Status:
While you evaluate which path fits your business needs, please remember the timeline for your current implementations:
Existing Projects: If you have an existing Programmable Search Engine configured to "Search the entire web," your service will continue to function until January 1, 2027. You have the full year to plan your migration.
New Projects: As of January 20, 2026, new engines created in the Programmable Search Engine admin console are restricted to "Site Search Only" (specific domains only).
Later in 2026 we’ll provide you with more updates regarding the new Web Search Service and the means to express your desire to use it to power your web search experiences.
Sincerely,
Programmable Search Engine Team
The correct parsing is: "Google is ending (full-web search for niche search engines)"
Given that the title supplied is effectively editorialised, and the original article's title is effectively content-free ("Updates to our Web Search Products & Programmable Search Engine Capabilities"), my rewording would be at least as fair.
HN's policy is to try to use text from the article itself where the article title is clickbait, sensational, vague, etc., however. I suspect Google's blog authors are aware of this, and they've carefully avoided any readily-extracted clear statements, though I'll take a stab...
Here's the most direct 'graph from TFA:
Custom Search JSON API: Vertex AI Search is a favorable alternative for up to 50 domains. Alternatively, if your use case necessitates full web search, contact us to express your interest in and get more information about our full web search solution. Your transition to an alternative solution needs to be completed by January 1, 2027.
We can get a clearer, 80-character head that's somewhat faithful to that with:
"Google Search API alternative Vertex AI Search limited to 50 domains" (70 chars).
That's still pretty loosely adherent, though it (mostly) uses words from the original article. I'm suggesting it to mods via email at hn@ycominator.com; others may wish to suggest their own formulations.
I suppose I'm asking whether this is actually a _good thing_ in that it will stimulate competition in the space, or if it's just a case that Google's index is now too good for anyone to reasonably catch up at this point.
Bing charge per query for the average user. Ecosia and Qwant use Bing to power their results, probably under some type of license, which results in them paying much less per query than a normal user.
If you want programmatic access to search results there aren't really many options left.
But for searching in more niche languages google is usually the only decent option and I have little hope that others will ever reach the scale where they could compete.
What alternatives are there besides Bing? Is it really so hard that it’s not considered worth doing? Some of the AI companies (Perplexity, Anthropic) seem to have managed to get their own indexing up and running.
Just dissolve them in acid.
Not long ago they ruined ublock origin (for chrome; ublock origin lite is nowhere near as good and effective, from my own experience here).
Now Google is also committing towards more evil and trying to ruin things for more - people, competitors, you name it. We can not allow Google to continue on its wiched path here. It'll just further erode the quality. There is a reason why "killed by google" is more than a mere meme - a graveyard of things killed by google.
We need alternatives, viable ones, for ALL Google services. Let's all work to make this world better - a place without Google.
I keep seeing posts about how ~"the volume of AI scrapers is making hosting untenable."
There must a ton of new full-web datasets out there, right?
What are the major hurdles that prevent the owners of these datasets from providing them to third parties via API? Is it the quality of SERP, or staleness? Otherwise, this seems like a potentially lucrative pivot/side hustle?
Sadly, no. There's CommonCrawl (https://commoncrawl.org/) which still, sadly, far removed from "full-web dataset."
So everyone runs their own search instead, hammering the sites, going into gray areas (you either ignore robots.txt or your results suck), etc. It's a tragedy of the commons that keeps Google entrenched: https://senkorasic.com/articles/ai-scraper-tragedy-commons
Although, it needs some more work and peers to be usable as a general-purpose search engine.
One thing touched upon in comments here: I never understood how it was proper for 3rd parties to scrape Google search results and reuse/resell them.
Really off topic, sorry, but I am surprised that more companies don’t build local search indices for just the few hundred web domains that are important to their businesses. I have tried this in combination with local (small and fast) LLMs and I think this is unappreciated tech: fast, cheap, and local.
RIP, another one to the Google Graveyard.