Edward Zitron has been reading all of google’s internal emails that have been released as evidence in the DOJ’s antitrust case against google.
This is the story of how Google Search died, and the people responsible for killing it.
The story begins on February 5th 2019, when Ben Gomes, Google’s head of search, had a problem. Jerry Dischler, then the VP and General Manager of Ads at Google, and Shiv Venkataraman, then the VP of Engineering, Search and Ads on Google properties, had called a “code yellow” for search revenue due to, and I quote, “steady weakness in the daily numbers” and a likeliness that it would end the quarter significantly behind.
HackerNews thread: https://news.ycombinator.com/item?id=40133976
MetaFilter thread: https://www.metafilter.com/203456/The-core-query-softness-continues-without-mitigation
You say that because it’s clear you have no fucking clue how difficult a problem this is. This isn’t something you can do overnight, and I’m not even sure a self-hosted solution is possible.
You say that, but it’s clear you have no fucking clue how easy a solution is.
https://yacy.net/
Commercial options:
https://solr.apache.org/
https://www.meilisearch.com/
No, you just haven’t thought through the implications more than a single step.
The real trick is SEO. These systems will be gamed. Google used to handle this by using its monopoly on search to enforce rules. It wasn’t perfect, but it kept the worst spam from being in the top five results for the most part. Doing this self-hosted would mean a million users having to agree to do the same thing to punish spam results, and that does not work.
And then there’s the problem of crawling and storing the entire web. Doing this for specific topics is doable. The entire web is not. Not for a home user with limited budget. YaCy’s P2P mode might be a way around that, but it’s also not really “self-hosted” anymore.
Microsoft dumped tons of money into making the second best search engine, and it’s a bit of a joke. This is not an easy problem.