Internet search engines
Tips, tricks, confidentiality
2021-08-02 — 2026-07-24
Wherein the Decline of Commercial Search Engines Is Chronicled, and Remedy Is Sought Among Privacy-Minded Alternatives, Metasearch Proxies, and AI-augmented Upstarts, With Kagi and Marginalia Noted as Curious Survivors.
Finding things on the internet! At one point this felt like a solved problem, but it seems to have become unsolved.
Famously, Google seems not to be particularly good at search any longer. Speculative reasons for this include losing the battle to the SEO, or that human-friendly content is being squeezed aside in general, that Google is spending down its credibility in order to bring in advertising revenue, or that some other more complicated mechanisms and incentives are just making things terrible or boring.
For some quantifiable data on the theme, see webis-de/ecir24-seo-spam-in-search-engines (Bevendorff et al. 2024).
Regardless of the details or reasons, it does seem to be true for me that search results are bad right now.
In addition, I am uncomfortable with the surveillance and tracking involved in search engines. Insofar as they are the way I access the world, they can potentially know too much about me.
I am interested in solving these problems: the badness of search results, the skeeziness of search providers, and more general knowledge discovery and synthesis problems. Let us see if I got anywhere by reading on.
1 Better commercial search providers
Here are some links to search engines which may reduce the degree of user surveillance, or at least diffuse the surveillance across a few different players if I use several of them. Hell they might even provide added value over the classic searches.
Many of these make strong claims to protect user privacy, although few offer substantive guarantees beyond inspecting tracking headers. Some of them repackage other searches; some run their own indices. Most of them have very unclear business models, which makes me uneasy.
1.1 Kagi
An exception to the opaque-business-model rule is Kagi. Their value proposition is, they claim, to be credibly user-centric:
Kagi has no ads and is fully supported only by its users. We worked very hard to provide high quality, fast and tracking-free results at a minimum cost to ensure sustainability of our operation.
By choosing a paid Kagi plan, you are also helping accelerate our mission of humanising the web.
There is no free plan any more, just a 100-search trial, after which it is USD $5/month for 300 searches or $10/month for unlimited. Here is what we get for the money:
No ads
Ability to block/boost domains
Bangs allow you to quickly jump to all popular sites on the web.
zero telemetry, zero tracking
See how fast is a website or how many ads/trackers it has before clicking the result.
They have been criticised for being chaos pants. These criticisms to me seem reasonable but not fatal.
Obviously, if I become a subscriber, they can in principle track me, so the privacy angle hinges on some trust.
1.2 Marginalia
File under quirky/quixotic/small web, Marginalia Search (which moved off search.marginalia.nu in 2025; the old interface lingers at old-search.marginalia.nu):
This is an independent DIY search engine that focuses on non-commercial content, and attempts to show you sites you perhaps weren’t aware of in favour of the sort of sites you probably already knew existed.
The software for this search engine is all custom-built, and all crawling and indexing is done in-house. The project is open source. Feel free to poke about in the source code or contribute to the development!
Still actively developed, which is more than most of the entries below can say.
1.3 Startpage
Startpage claims to repackage Google search results anonymously, AFAIK, although I cannot see much information about why I should believe them on this. Dutch company. To use them as a search bar in Firefox I needed to add a browser extension, for some tedious reason.
In 2019, when the adtech firm System1 took a controlling investment and Privacy Guides delisted them for it (briefly).
1.4 DuckDuckGo
Perennial favourite, duckduckgo is a search engine run by strident privacy advocates, which is laudable I s’pose. The search is… OK. Usually not as good as Google. Every now and again it is serendipitously wonderful, but not reliably.
1.5 Brave
Brave Search recently launched, backed by the creators of the Brave browser. TBC.
1.6 Mojeek
Mojeek/Mojeek Focus (Bookmark) Search Engine
Mojeek was created to provide a globally competitive and genuine alternative search engine based in the UK, and from the outset one that didn’t track its users nor simply retrieve its results from another engine (i.e. to provide real alternative results).
Mojeek’s technology has been developed entirely from scratch by Marc Smith, mostly using the C programming language, and uses no pre-existing search or web crawler technology. All technology and IP is fully owned by Mojeek Limited.
1.7 Qwant
Qwant promises to forget user data rapidly. French company. More interestingly, with Ecosia they now run the European Search Perspective joint venture and its Staan index, which is a serious attempt at a European crawl rather than another Bing reseller — about half of French Qwant queries are served from it.
1.8 Dead ends
runnaroo.com promised to aggregate many other search engines. Taken down in 2021, eaten by scrapers.
searchencrypt.com claimed additional privacy via Perfect Forward Secrecy. The domain is now an SEO content farm shilling solar panels and dental implants.
1.9 Suppressing spam in search results
2 DIY search proxies
A.k.a. meta-searching. I suspect these imply maintenance overhead as the search companies attempt to circumvent this circumvention of their business model. Effectively, we would be participating in an arms race.
2.1 SearXNG
The SearXNG family is a network of metasearch engine portals with the aim of protecting the privacy of users. It does not share users’ IP addresses or search history with the search engines from which it gathers results. Tracking cookies served by the search engines are blocked, etc. There are many user-operated instances, or we can run our own DIY search anonymizer.
searx, the original, is no longer maintained; SearXNG is the fork that kept going. Adam Tauber’s stated reason for walking away is an argument against this whole section: a metasearch engine is only as good as what upstream engines expose, its privacy guarantees are partial because the requests still leave the machine, and it has no memory of what we already read.
Tauber’s answer to that is Hister, which inverts the problem: index the pages we already read instead of proxying queries at engines that indexed everything. Same bet as my local qmd index — a small curated corpus beats a large uncurated one.
2.2 mysearch
mysearch was a local search portal designed to anonymize search requests. Dead.
3 AI-augmented search
The new hotness. For theory on how it works see AI search. Everything in this section is aimed at humans; the equivalents sold to agents are over at web search for machines.
3.1 Perplexity
Perplexity seemed to be the AI search frontrunner for the internet at large.
Perplexity is an alternative to traditional search engines, where you can directly pose your questions and receive concise, accurate answers backed up by a curated set of sources. It has a conversational interface, contextual awareness and personalisation to learn your interests and preferences over time.
Perplexity’s mission is to make searching for information online feel like you have a knowledgeable assistant guiding you, it is a powerful productivity and knowledge tool that can help you save time and energy with mundane tasks for a multitude of use cases.
I was a big fan of Perplexity at first, and I got good mileage out of it. Wow, when they first hosted DeepSeek-R1 in the search results, that was well cool.
However, their search product has languished behind Perplexity Computer (cloud-hosted agentic orchestration) and their Comet browser (AI-instrumented browser). That previously-core search product is starting to feel outdated. Increasingly, I find myself needing to manually remind it to check that its information is up to date, whereas my Claude Desktop doesn’t need reminding when it’s in agentic search mode — this has begun to feel tedious.
3.2 You
You.com was an AI-heavy search thing with a strong private-mode pitch. It is now a web search API vendor, selling the plumbing to other people’s agents rather than water to me directly. Common trajectory.
3.3 Yep
Yep is Ahrefs’ search engine, running on its own index off a crawler that has been going since 2010, i.e. one of the few independent indices left. It has made the same move as You.com but kept the consumer search box, pitching itself at “AI agents and humans” (in that order?).
3.4 Free/ FOSS -ish:
nilsherzig/LLocalSearch— a locally-running LLM-agent search aggregator, archived in March 2026.nashsu/FreeAskInternet— multi-engine search plus local LLM answer generation, no GPU needed. No commits since April 2024.
3.5 Others
komo.aiwas an AI search engine; it is now an “AI Revenue Engine” that writes cold-outreach emails for sales teams.waldo.fyipromised to make its users experts in seconds; it now sells brand-intelligence dashboards to ad agencies.
4 Decentralised search
What does the decentralised web do?
5 Discovering a website’s search
If we want our cool hand-rolled search to magically appear as a search option, we are looking for OpenSearch.
Worked example: Add Google Scholar to your browser.
Detailed documentation: opensearch/mediawiki/Specifications/OpenSearch.
6 Incoming
Vicki Boykis, How I search in 2024
We are now in a very weird liminal space in information retrieval for consumers, particularly those attuned to trends in search and working on the bleeding edge of LLMs.
[…]we have the fall of old companies. Broadcast-based centralised social media, which steadily served as a newsfeed and realtime search for a small, vocal minority, is basically dead, or on its last legs. Search, namely Google, is basically a useless pile of ads and SEO gamification at this point and a stopping point for Reddit results. Everyone has written about it and covered this extensively. […]
… on the heels of the large companies of the last 15 years declining, we have a new indie search engine scene emerging, hungry, armed with AI tooling, and ready to take back quality on the web.
She picks Kagi, Marginalia, and Perplexity.
Introducing Simple Search — The Markup’s extension for overlaying the “ten blue links” on a Google or Bing results page.
The extensionis delisted from both the Chrome and Firefox stores; the article is still a good read on how much of the page is not results.Gwern’s Internet Search Tips
Google Is the Only Search Engine That Works on Reddit Now Thanks to AI Deal
How to Turn Off AI Overview in Google and Set “Web” as Default