ai search

enough-is-enough—i-dumped-google’s-worsening-search-for-kagi

Enough is enough—I dumped Google’s worsening search for Kagi


I like how the search engine is the product instead of me.

Artist's depiction of the article author heaving a large multicolored

“Won’t be needing this anymore!” Credit: Aurich “The King” Lawson

“Won’t be needing this anymore!” Credit: Aurich “The King” Lawson

Mandatory AI summaries have come to Google, and they gleefully showcase hallucinations while confidently insisting on their truth. I feel about them the same way I felt about mandatory G+ logins when all I wanted to do was access my damn YouTube account: I hate them. Intensely.

But unlike those mandatory G+ logins—on which Google eventually relented before shutting down the G+ service—our reading of the tea leaves suggests that, this time, the search giant is extremely pleased with how things are going.

Fabricated AI dreck polluting your search? It’s the new normal. Miss your little results page with its 10 little blue links? Too bad. They’re gone now, and you can’t get them back, no matter what ephemeral workarounds or temporarily functional flags or undocumented, could-fail-at-any-time URL tricks you use.

And the galling thing is that Google expects you to be a good consumer and just take it. The subtext of the company’s (probably AI-generated) robo-MBA-speak non-responses to criticism and complaining is clear: “LOL, what are you going to do, use a different search engine? Now, shut up and have some more AI!”

But like the old sailor used to say: “That’s all I can stands, and I can’t stands no more.” So I did start using a different search engine—one that doesn’t constantly shower me with half-baked, anti-consumer AI offerings.

Out with Google, in with Kagi.

What the hell is a Kagi?

Kagi was founded in 2018, but its search product has only been publicly available since June 2022. It purports to be an independent search engine that pulls results from around the web (including from its own index) and is aimed at returning search to a user-friendly, user-focused experience. The company’s stated purpose is to deliver useful search results, full stop. The goal is not to blast you with AI garbage or bury you in “Knowledge Graph” summaries hacked together from posts in a 12-year-old Reddit thread between two guys named /u/WeedBoner420 and /u/14HitlerWasRight88.

Kagi’s offerings (it has a web browser, too, though I’ve not used it) are based on a simple idea. There’s an (oversimplified) axiom that if a good or service (like Google search, for example, or good ol’ Facebook) is free for you to use, it’s because you’re the product, not the customer. With Google, you pay with your attention, your behavioral metrics, and the intimate personal details of your wants and hopes and dreams (and the contents of your emails and other electronic communications—Google’s got most of that, too).

With Kagi, you pay for the product using money. That’s it! You give them some money, and you get some service—great service, really, which I’m overall quite happy with and which I’ll get to shortly. You don’t have to look at any ads. You don’t have to look at AI droppings. You don’t have to give perpetual ownership of your mind-palace to a pile of optioned-out tech bros in sleeveless Patagonia vests while you are endlessly subjected to amateur AI Rorschach tests every time you search for “pierogis near me.”

How much money are we talking?

I dunno, about a hundred bucks a year? That’s what I’m spending as an individual for unlimited searches. I’m using Kagi’s “Professional” plan, but there are others, including a free offering so that you can poke around and see if the service is worth your time.

image of kagi billing panel

This is my account’s billing page, showing what I’ve paid for Kagi in the past year. (By the time this article runs, I’ll have renewed my subscription!)

Credit: Lee Hutchinson

This is my account’s billing page, showing what I’ve paid for Kagi in the past year. (By the time this article runs, I’ll have renewed my subscription!) Credit: Lee Hutchinson

I’d previously bounced off two trial runs with Kagi in 2023 and 2024 because the idea of paying for search just felt so alien. But that was before Google’s AI enshittification rolled out in full force. Now, sitting in the middle of 2025 with the world burning down around me, a hundred bucks to kick Google to the curb and get better search results feels totally worth it. Your mileage may vary, of course.

The other thing that made me nervous about paying for search was the idea that my money was going to enrich some scumbag VC fund, but fortunately, there’s good news on that front. According to the company’s “About” page, Kagi has not taken any money from venture capitalist firms. Instead, it has been funded by a combination of self-investment by the founder, selling equity to some Kagi users in two rounds, and subscription revenue:

Kagi was bootstrapped from 2018 to 2023 with ~$3M initial funding from the founder. In 2023, Kagi raised $670K from Kagi users in its first external fundraise, followed by $1.88M raised in 2024, again from our users, bringing the number of users-investors to 93… In early 2024, Kagi became a Public Benefit Corporation (PBC).

What about DuckDuckGo? Or Bing? Or Brave?

Sure, those can be perfectly cromulent alternatives to Google, but honestly, I don’t think they go far enough. DuckDuckGo is fine, but it largely utilizes Bing’s index; and while DuckDuckGo exercises considerable control over its search results, the company is tied to the vicissitudes of Microsoft by that index. It’s a bit like sitting in a boat tied to a submarine. Sure, everything’s fine now, but at some point, that sub will do what subs do—and your boat is gonna follow it down.

And as for Bing itself, perhaps I’m nitpicky [Ed. note: He is!], but using Bing feels like interacting with 2000-era MSN’s slightly perkier grandkid. It’s younger and fresher, yes, but it still radiates that same old stanky feeling of taste-free, designed-by-committee artlessness. I’d rather just use Google—which is saying something. At least Google’s search home page remains uncluttered.

Brave Search is another fascinating option I haven’t spent a tremendous amount of time with, largely because Brave’s cryptocurrency ties still feel incredibly low-rent and skeevy. I’m slowly warming up to the Brave Browser as a replacement for Chrome (see the screenshots in this article!), but I’m just not comfortable with Brave yet—and likely won’t be unless the company divorces itself from cryptocurrencies entirely.

More anonymity, if you want it

The feature that convinced me to start paying for Kagi was its Privacy Pass option. Based on a clean-sheet Rust implementation of the Privacy Pass standard (IETF RFCs 9576, 9577, and 9578) by Raphael Robert, this is a technology that uses cryptographic token-based auth to send an “I’m a paying user, please give me results” signal to Kagi, without Kagi knowing which user made the request. (There’s a much longer Kagi blog post with actual technical details for the curious.)

To search using the tool, you install the Privacy Pass extension (linked in the docs above) in your browser, log in to Kagi, and enable the extension. This causes the plugin to request a bundle of tokens from the search service. After that, you can log out and/or use private windows, and those tokens are utilized whenever you do a Kagi search.

image of a kagi search with privacy pass enabled

Privacy pass is enabled, allowing me to explore the delicious mystery of pierogis with some semblance of privacy.

Credit: Lee Hutchinson

Privacy pass is enabled, allowing me to explore the delicious mystery of pierogis with some semblance of privacy. Credit: Lee Hutchinson

The obvious flaw here is that Kagi still records source IP addresses along with Privacy Pass searches, potentially de-anonymizing them, but there’s a path around that: Privacy Pass functions with Tor, and Kagi maintains a Tor onion address for searches.

So why do I keep using Privacy Pass without Tor, in spite of the opsec flaw? Maybe it’s the placebo effect in action, but I feel better about putting at least a tiny bit of friction in the way of someone with root attempting to casually browse my search history. Like, I want there to be at least a SQL JOIN or two between my IP address and my searches for “best Mass Effect alien sex choices” or “cleaning tips for Garrus body pillow.” I mean, you know, assuming I were ever to search for such things.

What’s it like to use?

Moving on with embarrassed rapidity, let’s look at Kagi a bit and see how using it feels.

My anecdotal observation is that Kagi doesn’t favor Reddit-based results nearly as much as Google does, but sometimes it still has them near or at the top. And here is where Kagi curb-stomps Google with quality-of-life features: Kagi lets you prioritize or de-prioritize a website’s prominence in your search results. You can even pin that site to the top of the screen or block it completely.

This is a feature I’ve wanted Google to get for about 25 damn years but that the company has consistently refused to properly implement (likely because allowing users to exclude sites from search results notionally reduces engagement and therefore reduces the potential revenue that Google can extract from search). Well, screw you, Google, because Kagi lets me prioritize or exclude sites from my results, and it works great—I’m extraordinarily pleased to never again have to worry about Quora or Pinterest links showing up in my search results.

Further, Kagi lets me adjust these settings both for the current set of search results (if you don’t want Reddit results for this search but you don’t want to drop Reddit altogether) and also globally (for all future searches):

image of kagi search personalization options

Goodbye forever, useless crap sites.

Credit: Lee Hutchinson

Goodbye forever, useless crap sites. Credit: Lee Hutchinson

Another tremendous quality-of-life improvement comes via Kagi’s image search, which does a bunch of stuff that Google should and/or used to do—like giving you direct right-click access to save images without having to fight the search engine with workarounds, plugins, or Tampermonkey-esque userscripts.

The Kagi experience is also vastly more customizable than Google’s (or at least, how Google’s has become). The widgets that appear in your results can be turned off, and the “lenses” through which Kagi sees the web can be adjusted to influence what kinds of things do and do not appear in your results.

If that doesn’t do it for you, how about the ability to inject custom CSS into your search and landing pages? Or to automatically rewrite search result URLs to taste, doing things like redirecting reddit.com to old.reddit.com? Or breaking free of AMP pages and always viewing originals instead?

Image of kagi custom css field

Imagine all the things Ars readers will put here.

Credit: Lee Hutchinson

Imagine all the things Ars readers will put here. Credit: Lee Hutchinson

Is that all there is?

Those are really all the features I care about, but there are loads of other Kagi bits to discover—like a Kagi Maps tool (it’s pretty good, though I’m not ready to take it up full time yet) and a Kagi video search tool. There are also tons of classic old-Google-style inline search customizations, including verbatim mode, where instead of trying to infer context about your search terms, Kagi searches for exactly what you put in the box. You can also add custom search operators that do whatever you program them to do, and you get API-based access for doing programmatic things with search.

A quick run-through of a few additional options pages. This is the general customization page. Lee Hutchinson

I haven’t spent any time with Kagi’s Orion browser, but it’s there as an option for folks who want a WebKit-based browser with baked-in support for Privacy Pass and other Kagi functionality. For now, Firefox continues to serve me well, with Brave as a fallback for working with Google Docs and other tools I can’t avoid and that treat non-Chromium browsers like second-class citizens. However, Orion is probably on the horizon for me if things in Mozilla-land continue to sour.

Cool, but is it any good?

Rather than fill space with a ton of comparative screenshots between Kagi and Google or Kagi and Bing, I want to talk about my subjective experience using the product. (You can do all the comparison searches you want—just go and start searching—and your comparisons will be a lot more relevant to your personal use cases than any examples I can dream up!)

My time with Kagi so far has included about seven months of casual opportunistic use, where I’d occasionally throw a query at it to see how it did, and about five months of committed daily use. In the five months of daily usage, I can count on one hand the times I’ve done a supplementary Google search because Kagi didn’t have what I was looking for on the first page of results. I’ve done searches for all the kinds of things I usually look for in a given day—article fact-checking queries, searches for details about the parts of speech, hunts for duck facts (we have some feral Muscovy ducks nesting in our front yard), obscure technical details about Project Apollo, who the hell played Dupont in Equilibrium (Angus Macfadyen, who also played Robert the Bruce in Braveheart), and many, many other queries.

Image of Firefox history window showing kagi searches for july 22

A typical afternoon of Kagi searches, from my Firefox history window.

Credit: Lee Hutchinson

A typical afternoon of Kagi searches, from my Firefox history window. Credit: Lee Hutchinson

For all of these things, Kagi has responded quickly and correctly. The time to service a query feels more or less like Google’s service times; according to the timer at the top of the page, my Kagi searches complete in between 0.2 and 0.8 seconds. Kagi handles misspellings in search terms with the grace expected of a modern search engine and has had no problem figuring out my typos.

Holistically, taking search customizations into account on top of the actual search performance, my subjective assessment is that Kagi gets me accurate, high-quality results on more or less any given query, and it does so without festooning the results pages with features I find detractive and irrelevant.

I know that’s not a data-driven assessment, and it doesn’t fall back on charts or graphs or figures, but it’s how I feel after using the product every single day for most of 2025 so far. For me, Kagi’s search performance is firmly in the “good enough” category, and that’s what I need.

Kagi and AI

Unfortunately, the thing that’s stopping me from being completely effusive in my praise is that Kagi is exhibiting a disappointing amount of “keeping-up-with-the-Joneses” by rolling out a big ‘ol pile of (optional, so far) AI-enabled search features.

A blog post from founder Vladimir Prelovac talks about the company’s use of AI, and it says all the right things, but at this point, I trust written statements from tech company founders about as far as I can throw their corporate office buildings. (And, dear reader, that ain’t very far).

image of kagi ai features

No thanks. But I would like to exclude AI images from my search results, please.

Credit: Lee Hutchinson

No thanks. But I would like to exclude AI images from my search results, please. Credit: Lee Hutchinson

The short version is that, like Google, Kagi has some AI features: There’s an AI search results summarizer, an AI page summarizer, and an “ask questions about your results” chatbot-style function where you can interactively interrogate an LLM about your search topic and results. So far, all of these things can be disabled or ignored. I don’t know how good any of the features are because I have disabled or ignored them.

If the existence of AI in a product is a bright red line you won’t cross, you’ll have to turn back now and find another search engine alternative that doesn’t use AI and also doesn’t suck. When/if you do, let me know, because the pickings are slim.

Is Kagi for you?

Kagi might be for you—especially if you’ve recently typed a simple question into Google and gotten back a pile of fabricated gibberish in place of those 10 blue links that used to serve so well. Are you annoyed that Google’s search sucks vastly more now than it did 10 years ago? Are you unhappy with how difficult it is to get Google search to do what you want? Are you fed up? Are you pissed off?

If your answer to those questions is the same full-throated “Hell yes, I am!” that mine was, then perhaps it’s time to try an alternative. And Kagi’s a pretty decent one—if you’re not averse to paying for it.

It’s a fantastic feeling to type in a search query and once again get useful, relevant, non-AI results (that I can customize!). It’s a bit of sanity returning to my Internet experience, and I’m grateful. Until Kagi is bought by a value-destroying vampire VC fund or implodes into its own AI-driven enshittification cycle, I’ll probably keep paying for it.

After that, who knows? Maybe I’ll throw away my computers and live in a cave. At least until the cave’s robot exclusion protocol fails and the Googlebot comes for me.

Photo of Lee Hutchinson

Lee is the Senior Technology Editor, and oversees story development for the gadget, culture, IT, and video sections of Ars Technica. A long-time member of the Ars OpenForum with an extensive background in enterprise storage and security, he lives in Houston.

Enough is enough—I dumped Google’s worsening search for Kagi Read More »

new-grok-ai-model-surprises-experts-by-checking-elon-musk’s-views-before-answering

New Grok AI model surprises experts by checking Elon Musk’s views before answering

Seeking the system prompt

Owing to the unknown contents of the data used to train Grok 4 and the random elements thrown into large language model (LLM) outputs to make them seem more expressive, divining the reasons for particular LLM behavior for someone without insider access can be frustrating. But we can use what we know about how LLMs work to guide a better answer. xAI did not respond to a request for comment before publication.

To generate text, every AI chatbot processes an input called a “prompt” and produces a plausible output based on that prompt. This is the core function of every LLM. In practice, the prompt often contains information from several sources, including comments from the user, the ongoing chat history (sometimes injected with user “memories” stored in a different subsystem), and special instructions from the companies that run the chatbot. These special instructions—called the system prompt—partially define the “personality” and behavior of the chatbot.

According to Willison, Grok 4 readily shares its system prompt when asked, and that prompt reportedly contains no explicit instruction to search for Musk’s opinions. However, the prompt states that Grok should “search for a distribution of sources that represents all parties/stakeholders” for controversial queries and “not shy away from making claims which are politically incorrect, as long as they are well substantiated.”

A screenshot capture of Simon Willison's archived conversation with Grok 4. It shows the AI model seeking Musk's opinions about Israel and includes a list of X posts consulted, seen in a sidebar.

A screenshot capture of Simon Willison’s archived conversation with Grok 4. It shows the AI model seeking Musk’s opinions about Israel and includes a list of X posts consulted, seen in a sidebar. Credit: Benj Edwards

Ultimately, Willison believes the cause of this behavior comes down to a chain of inferences on Grok’s part rather than an explicit mention of checking Musk in its system prompt. “My best guess is that Grok ‘knows’ that it is ‘Grok 4 built by xAI,’ and it knows that Elon Musk owns xAI, so in circumstances where it’s asked for an opinion, the reasoning process often decides to see what Elon thinks,” he said.

Without official word from xAI, we’re left with a best guess. However, regardless of the reason, this kind of unreliable, inscrutable behavior makes many chatbots poorly suited for assisting with tasks where reliability or accuracy are important.

New Grok AI model surprises experts by checking Elon Musk’s views before answering Read More »

google-begins-rolling-out-ai-search-in-youtube

Google begins rolling out AI search in YouTube

Over the past year, Google has transformed its web search experience with AI, driving toward a zero-click experience. Now, the same AI focus is coming to YouTube, and Premium subscribers can get a preview of the new search regime. Select searches on the video platform will now produce an AI-generated results carousel with a collection of relevant videos. Even if you don’t pay for YouTube, AI is still coming for you with an expansion of Google’s video chatbot.

Google says the new AI search feature, which appears at the top of the results page, will include multiple videos, along with an AI summary of each. You can tap the video thumbnails to begin playing them right from the carousel. The summary is intended to extract the information most relevant to your search query, so you may not even have to watch the videos.

The AI results carousel is only a test right now, and it’s limited to YouTube Premium subscribers. If you’re paying for Premium, you can enable the feature on YouTube’s experimental page. While the feature is entirely opt-in, that probably won’t last long. Like AI Overviews in search, this feature will take precedence over organic search results and get people interacting with Google’s AI, and that’s the driving force behind most of the company’s decisions lately.

It’s not hard to see where this feature could lead because we’ve seen the same thing play out in general web search. By putting AI-generated content at the top of search results, Google will reduce the number of videos people click to watch. The carousel gives you the relevant parts of the video along with a summary, but the video page is another tap away. Rather than opening videos, commenting, subscribing, and otherwise interacting with creators, some users will just peruse the AI carousel. That could make it harder for channels to grow and earn revenue from their content—the same content Google will feed into Gemini to generate the AI carousel.

Google begins rolling out AI search in YouTube Read More »

the-resume-is-dying,-and-ai-is-holding-the-smoking-gun

The résumé is dying, and AI is holding the smoking gun

Beyond volume, fraud poses an increasing threat. In January, the Justice Department announced indictments in a scheme to place North Korean nationals in remote IT roles at US companies. Research firm Gartner says that fake identity cases are growing rapidly, with the company estimating that by 2028, about 1 in 4 job applicants could be fraudulent. And as we have previously reported, security researchers have also discovered that AI systems can hide invisible text in applications, potentially allowing candidates to game screening systems using prompt injections in ways human reviewers can’t detect.

Illustration of a robot generating endless text, controlled by a scientist.

And that’s not all. Even when AI screening tools work as intended, they exhibit similar biases to human recruiters, preferring white male names on résumés—raising legal concerns about discrimination. The European Union’s AI Act already classifies hiring under its high-risk category with stringent restrictions. Although no US federal law specifically addresses AI use in hiring, general anti-discrimination laws still apply.

So perhaps résumés as a meaningful signal of candidate interest and qualification are becoming obsolete. And maybe that’s OK. When anyone can generate hundreds of tailored applications with a few prompts, the document that once demonstrated effort and genuine interest in a position has devolved into noise.

Instead, the future of hiring may require abandoning the résumé altogether in favor of methods that AI can’t easily replicate—live problem-solving sessions, portfolio reviews, or trial work periods, just to name a few ideas people sometimes consider (whether they are good ideas or not is beyond the scope of this piece). For now, employers and job seekers remain locked in an escalating technological arms race where machines screen the output of other machines, while the humans they’re meant to serve struggle to make authentic connections in an increasingly inauthentic world.

Perhaps the endgame is robots interviewing other robots for jobs performed by robots, while humans sit on the beach drinking daiquiris and playing vintage video games. Well, one can dream.

The résumé is dying, and AI is holding the smoking gun Read More »

ai-overviews-hallucinates-that-airbus,-not-boeing,-involved-in-fatal-air-india-crash

AI Overviews hallucinates that Airbus, not Boeing, involved in fatal Air India crash

When major events occur, most people rush to Google to find information. Increasingly, the first thing they see is an AI Overview, a feature that already has a reputation for making glaring mistakes. In the wake of a tragic plane crash in India, Google’s AI search results are spreading misinformation claiming the incident involved an Airbus plane—it was actually a Boeing 787.

Travelers are more attuned to the airliner models these days after a spate of crashes involving Boeing’s 737 lineup several years ago. Searches for airline disasters are sure to skyrocket in the coming days, with reports that more than 200 passengers and crew lost their lives in the Air India Flight 171 crash. The way generative AI operates means some people searching for details may get the wrong impression from Google’s results page.

Not all searches get AI answers, but Google has been steadily expanding this feature since it debuted last year. One searcher on Reddit spotted a troubling confabulation when searching for crashes involving Airbus planes. AI Overviews, apparently overwhelmed with results reporting on the Air India crash, stated confidently (and incorrectly) that it was an Airbus A330 that fell out of the sky shortly after takeoff. We’ve run a few similar searches—some of the AI results say Boeing, some say Airbus, and some include a strange mashup of both Airbus and Boeing. It’s a mess.

In this search, Google’s AI says the crash involved an Airbus A330 instead of a Boeing 787.

Credit: /u/stuckintrraffic

In this search, Google’s AI says the crash involved an Airbus A330 instead of a Boeing 787. Credit: /u/stuckintrraffic

But why is Google bringing up the Air India crash at all in the context of Airbus? Unfortunately, it’s impossible to predict if you’ll get an AI Overview that blames Boeing or Airbus—generative AI is non-deterministic, meaning the output is different every time, even for identical inputs. Our best guess for the underlying cause is that numerous articles on the Air India crash mention Airbus as Boeing’s main competitor. AI Overviews is essentially summarizing these results, and the AI goes down the wrong path because it lacks the ability to understand what is true.

AI Overviews hallucinates that Airbus, not Boeing, involved in fatal Air India crash Read More »

anthropic’s-new-ai-search-feature-digs-through-the-web-for-answers

Anthropic’s new AI search feature digs through the web for answers

Caution over citations and sources

Claude users should be warned that large language models (LLMs) like those that power Claude are notorious for sneaking in plausible-sounding confabulated sources. A recent survey of citation accuracy by LLM-based web search assistants showed a 60 percent error rate. That particular study did not include Anthropic’s new search feature because it took place before this current release.

When using web search, Claude provides citations for information it includes from online sources, ostensibly helping users verify facts. From our informal and unscientific testing, Claude’s search results appeared fairly accurate and detailed at a glance, but that is no guarantee of overall accuracy. Anthropic did not release any search accuracy benchmarks, so independent researchers will likely examine that over time.

A screenshot example of what Anthropic Claude's web search citations look like, captured March 21, 2025.

A screenshot example of what Anthropic Claude’s web search citations look like, captured March 21, 2025. Credit: Benj Edwards

Even if Claude search were, say, 99 percent accurate (a number we are making up as an illustration), the 1 percent chance it is wrong may come back to haunt you later if you trust it blindly. Before accepting any source of information delivered by Claude (or any AI assistant) for any meaningful purpose, vet it very carefully using multiple independent non-AI sources.

A partnership with Brave under the hood

Behind the scenes, it looks like Anthropic partnered with Brave Search to power the search feature, from a company, Brave Software, perhaps best known for its web browser app. Brave Search markets itself as a “private search engine,” which feels in line with how Anthropic likes to market itself as an ethical alternative to Big Tech products.

Simon Willison discovered the connection between Anthropic and Brave through Anthropic’s subprocessor list (a list of third-party services that Anthropic uses for data processing), which added Brave Search on March 19.

He further demonstrated the connection on his blog by asking Claude to search for pelican facts. He wrote, “It ran a search for ‘Interesting pelican facts’ and the ten results it showed as citations were an exact match for that search on Brave.” He also found evidence in Claude’s own outputs, which referenced “BraveSearchParams” properties.

The Brave engine under the hood has implications for individuals, organizations, or companies that might want to block Claude from accessing their sites since, presumably, Brave’s web crawler is doing the web indexing. Anthropic did not mention how sites or companies could opt out of the feature. We have reached out to Anthropic for clarification.

Anthropic’s new AI search feature digs through the web for answers Read More »

ai-search-engines-cite-incorrect-sources-at-an-alarming-60%-rate,-study-says

AI search engines cite incorrect sources at an alarming 60% rate, study says

A new study from Columbia Journalism Review’s Tow Center for Digital Journalism finds serious accuracy issues with generative AI models used for news searches. The research tested eight AI-driven search tools equipped with live search functionality and discovered that the AI models incorrectly answered more than 60 percent of queries about news sources.

Researchers Klaudia Jaźwińska and Aisvarya Chandrasekar noted in their report that roughly 1 in 4 Americans now use AI models as alternatives to traditional search engines. This raises serious concerns about reliability, given the substantial error rate uncovered in the study.

Error rates varied notably among the tested platforms. Perplexity provided incorrect information in 37 percent of the queries tested, whereas ChatGPT Search incorrectly identified 67 percent (134 out of 200) of articles queried. Grok 3 demonstrated the highest error rate, at 94 percent.

A graph from CJR shows

A graph from CJR shows “confidently wrong” search results. Credit: CJR

For the tests, researchers fed direct excerpts from actual news articles to the AI models, then asked each model to identify the article’s headline, original publisher, publication date, and URL. They ran 1,600 queries across the eight different generative search tools.

The study highlighted a common trend among these AI models: rather than declining to respond when they lacked reliable information, the models frequently provided confabulations—plausible-sounding incorrect or speculative answers. The researchers emphasized that this behavior was consistent across all tested models, not limited to just one tool.

Surprisingly, premium paid versions of these AI search tools fared even worse in certain respects. Perplexity Pro ($20/month) and Grok 3’s premium service ($40/month) confidently delivered incorrect responses more often than their free counterparts. Though these premium models correctly answered a higher number of prompts, their reluctance to decline uncertain responses drove higher overall error rates.

Issues with citations and publisher control

The CJR researchers also uncovered evidence suggesting some AI tools ignored Robot Exclusion Protocol settings, which publishers use to prevent unauthorized access. For example, Perplexity’s free version correctly identified all 10 excerpts from paywalled National Geographic content, despite National Geographic explicitly disallowing Perplexity’s web crawlers.

AI search engines cite incorrect sources at an alarming 60% rate, study says Read More »

perplexity-wants-to-reinvent-the-web-browser-with-ai—but-there’s-fierce-competition

Perplexity wants to reinvent the web browser with AI—but there’s fierce competition

It has recently been expanding its offerings—for example, it recently launched a deep research tool competing with similar ones provided by OpenAI and Google, as well as Sonar, an API for generative AI-powered search.

It will face fierce competition in the browser market, though. Google’s Chrome accounts for the majority of web browser use around the world, and despite its position at the forefront of AI search, Perplexity isn’t the first to introduce a browser with heavy use of generative AI features. For example, The Browser Company showed off its Dia browser in December.

Dia will allow users to type natural language commands into the search bar, like finding a document or webpage or creating a calendar event. It’s possible that Comet will do similar things, but again, we don’t know.

So far, most consumer-facing AI tools have come in one of three forms. There are general-purpose chatbots (like OpenAI’s ChatGPT and Anthropic’s Claude); features that use trained deep learning models subtly baked into existing software (as in Adobe Photoshop or Apple’s iOS); and, less commonly, standalone software meant to remake existing application categories using AI features (like the Cursor IDE).

There haven’t been a ton of AI-specific applications in existing categories like this before, but expect to see more coming over the next couple of years.

Perplexity wants to reinvent the web browser with AI—but there’s fierce competition Read More »

google-will-apparently-offer-“ai-mode”-right-on-its-main-search-page

Google will apparently offer “AI Mode” right on its main search page

Google will soon take more steps to make AI a part of search, exposing more users to its Gemini agent, according to recent reports and app teardowns.

“AI Mode,” shown at the top left of the web results page and inside the Google app, will provide an interface similar to a Gemini AI chat, according to The Information.

This tracks with a finding from Android Authority earlier this month, which noted a dedicated “AI mode” button inside an early beta of the Google app. This shortcut also appeared on Google’s Android search widget, and a conversation history button was added to the Google app. Going even deeper into the app, 9to5Google found references to “aim” (AI mode) and “ai_mode” which suggest a dedicated tab in the Google app, with buttons for speaking to an AI or sending it pictures.

Google already promotes Gemini with links below its search homepage. (“5 ways Gemini can help during the Holidays” is currently showing for me.) Search results on Google can also contain an “AI Overview,” which launched with some “use glue for pizza sauce” notoriety. People averse to AI answers can avoid them with URL parameters and proxy sites (or sticking to the “web” tab). Gemini has also been prominently added to other Google products, like Pixel phones, Gmail, and Drive/Workspace. And the search giant has also been testing the ability to attach files to a web search for analysis.

Google will apparently offer “AI Mode” right on its main search page Read More »

google’s-plan-to-keep-ai-out-of-search-trial-remedies-isn’t-going-very-well

Google’s plan to keep AI out of search trial remedies isn’t going very well


DOJ: AI is not its own market

Judge: AI will likely play “larger role” in Google search remedies as market shifts.

Google got some disappointing news at a status conference Tuesday, where US District Judge Amit Mehta suggested that Google’s AI products may be restricted as an appropriate remedy following the government’s win in the search monopoly trial.

According to Law360, Mehta said that “the recent emergence of AI products that are intended to mimic the functionality of search engines” is rapidly shifting the search market. Because the judge is now weighing preventive measures to combat Google’s anticompetitive behavior, the judge wants to hear much more about how each side views AI’s role in Google’s search empire during the remedies stage of litigation than he did during the search trial.

“AI and the integration of AI is only going to play a much larger role, it seems to me, in the remedy phase than it did in the liability phase,” Mehta said. “Is that because of the remedies being requested? Perhaps. But is it also potentially because the market that we have all been discussing has shifted?”

To fight the DOJ’s proposed remedies, Google is seemingly dragging its major AI rivals into the trial. Trying to prove that remedies would harm Google’s ability to compete, the tech company is currently trying to pry into Microsoft’s AI deals, including its $13 billion investment in OpenAI, Law360 reported. At least preliminarily, Mehta has agreed that information Google is seeking from rivals has “core relevance” to the remedies litigation, Law360 reported.

The DOJ has asked for a wide range of remedies to stop Google from potentially using AI to entrench its market dominance in search and search text advertising. They include a ban on exclusive agreements with publishers to train on content, which the DOJ fears might allow Google to block AI rivals from licensing data, potentially posing a barrier to entry in both markets. Under the proposed remedies, Google would also face restrictions on investments in or acquisitions of AI products, as well as mergers with AI companies.

Additionally, the DOJ wants Mehta to stop Google from any potential self-preferencing, such as making an AI product mandatory on Android devices Google controls or preventing a rival from distribution on Android devices.

The government seems very concerned that Google may use its ownership of Android to play games in the emerging AI sector. They’ve further recommended an order preventing Google from discouraging partners from working with rivals, degrading the quality of rivals’ AI products on Android devices, or otherwise “coercing” manufacturers or other Android partners into giving Google’s AI products “better treatment.”

Importantly, if the court orders AI remedies linked to Google’s control of Android, Google could risk a forced sale of Android if Mehta grants the DOJ’s request for “contingent structural relief” requiring divestiture of Android if behavioral remedies don’t destroy the current monopolies.

Finally, the government wants Google to be required to allow publishers to opt out of AI training without impacting their search rankings. (Currently, opting out of AI scraping automatically opts sites out of Google search indexing.)

All of this, the DOJ alleged, is necessary to clear the way for a thriving search market as AI stands to shake up the competitive landscape.

“The promise of new technologies, including advances in artificial intelligence (AI), may present an opportunity for fresh competition,” the DOJ said in a court filing. “But only a comprehensive set of remedies can thaw the ecosystem and finally reverse years of anticompetitive effects.”

At the status conference Tuesday, DOJ attorney David Dahlquist reiterated to Mehta that these remedies are needed so that Google’s illegal conduct in search doesn’t extend to this “new frontier” of search, Law360 reported. Dahlquist also clarified that the DOJ views these kinds of AI products “as new access points for search, rather than a whole new market.”

“We’re very concerned about Google’s conduct being a barrier to entry,” Dahlquist said.

Google could not immediately be reached for comment. But the search giant has maintained that AI is beyond the scope of the search trial.

During the status conference, Google attorney John E. Schmidtlein disputed that AI remedies are relevant. While he agreed that “AI is key to the future of search,” he warned that “extraordinary” proposed remedies would “hobble” Google’s AI innovation, Law360 reported.

Microsoft shields confidential AI deals

Microsoft is predictably protective of its AI deals, arguing in a court filing that its “highly confidential agreements with OpenAI, Perplexity AI, Inflection, and G42 are not relevant to the issues being litigated” in the Google trial.

According to Microsoft, Google is arguing that it needs this information to “shed light” on things like “the extent to which the OpenAI partnership has driven new traffic to Bing and otherwise affected Microsoft’s competitive standing” or what’s required by “terms upon which Bing powers functionality incorporated into Perplexity’s search service.”

These insights, Google seemingly hopes, will convince Mehta that Google’s AI deals and investments are the norm in the AI search sector. But Microsoft is currently blocking access, arguing that “Google has done nothing to explain why” it “needs access to the terms of Microsoft’s highly confidential agreements with other third parties” when Microsoft has already offered to share documents “regarding the distribution and competitive position” of its AI products.

Microsoft also opposes Google’s attempts to review how search click-and-query data is used to train OpenAI’s models. Those requests would be better directed at OpenAI, Microsoft said.

If Microsoft gets its way, Google’s discovery requests will be limited to just Microsoft’s content licensing agreements for Copilot. Microsoft alleged those are the only deals “related to the general search or the general search text advertising markets” at issue in the trial.

On Tuesday, Microsoft attorney Julia Chapman told Mehta that Microsoft had “agreed to provide documents about the data used to train its own AI model and also raised concerns about the competitive sensitivity of Microsoft’s agreements with AI companies,” Law360 reported.

It remains unclear at this time if OpenAI will be forced to give Google the click-and-query data Google seeks. At the status hearing, Mehta ordered OpenAI to share “financial statements, information about the training data for ChatGPT, and assessments of the company’s competitive position,” Law360 reported.

But the DOJ may also be interested in seeing that data. In their proposed final judgment, the government forecasted that “query-based AI solutions” will “provide the most likely long-term path for a new generation of search competitors.”

Because of that prediction, any remedy “must prevent Google from frustrating or circumventing” court-ordered changes “by manipulating the development and deployment of new technologies like query-based AI solutions.” Emerging rivals “will depend on the absence of anticompetitive constraints to evolve into full-fledged competitors and competitive threats,” the DOJ alleged.

Mehta seemingly wants to see the evidence supporting the DOJ’s predictions, which could end up exposing carefully guarded secrets of both Google’s and its biggest rivals’ AI deals.

On Tuesday, the judge noted that integration of AI into search engines had already evolved what search results pages look like. And from his “very layperson’s perspective,” it seems like AI’s integration into search engines will continue moving “very quickly,” as both parties seem to agree.

Whether he buys into the DOJ’s theory that Google could use its existing advantage as the world’s greatest gatherer of search query data to block rivals from keeping pace is still up in the air, but the judge seems moved by the DOJ’s claim that “AI has the ability to affect market dynamics in these industries today as well as tomorrow.”

Photo of Ashley Belanger

Ashley is a senior policy reporter for Ars Technica, dedicated to tracking social impacts of emerging policies and new technologies. She is a Chicago-based journalist with 20 years of experience.

Google’s plan to keep AI out of search trial remedies isn’t going very well Read More »