AI confabulations

education-report-calling-for-ethical-ai-use-contains-over-15-fake-sources

Education report calling for ethical AI use contains over 15 fake sources

AI language models like the kind that power ChatGPT, Gemini, and Claude excel at producing exactly this kind of believable fiction when they lack actual information on a topic because they first and foremost produce plausible outputs, not accurate ones. If there are no patterns in the dataset that match what the user is seeking they will create the best approximation based on statistical patterns learned during training. Even AI models that can search the web for real sources can potentially fabricate citations, choose the wrong ones, or mischaracterize them.

“Errors happen. Made-up citations are a totally different thing where you essentially demolish the trustworthiness of the material,” Josh Lepawsky, the former president of the Memorial University Faculty Association who resigned from the report’s advisory board in January, told CBC, citing a “deeply flawed process.”

The irony runs deep

The presence of potentially AI-generated fake citations becomes especially awkward given that one of the report’s 110 recommendations specifically states the provincial government should “provide learners and educators with essential AI knowledge, including ethics, data privacy, and responsible technology use.”

Sarah Martin, a Memorial political science professor who spent days reviewing the document, discovered multiple fabricated citations. “Around the references I cannot find, I can’t imagine another explanation,” she told CBC. “You’re like, ‘This has to be right, this can’t not be.’ This is a citation in a very important document for educational policy.”

When contacted by CBC, co-chair Karen Goodnough declined an interview request, writing in an email: “We are investigating and checking references, so I cannot respond to this at the moment.”

The Department of Education and Early Childhood Development acknowledged awareness of “a small number of potential errors in citations” in a statement to CBC from spokesperson Lynn Robinson. “We understand that these issues are being addressed, and that the online report will be updated in the coming days to rectify any errors.”

Education report calling for ethical AI use contains over 15 fake sources Read More »

ai-search-engines-cite-incorrect-sources-at-an-alarming-60%-rate,-study-says

AI search engines cite incorrect sources at an alarming 60% rate, study says

A new study from Columbia Journalism Review’s Tow Center for Digital Journalism finds serious accuracy issues with generative AI models used for news searches. The research tested eight AI-driven search tools equipped with live search functionality and discovered that the AI models incorrectly answered more than 60 percent of queries about news sources.

Researchers Klaudia Jaźwińska and Aisvarya Chandrasekar noted in their report that roughly 1 in 4 Americans now use AI models as alternatives to traditional search engines. This raises serious concerns about reliability, given the substantial error rate uncovered in the study.

Error rates varied notably among the tested platforms. Perplexity provided incorrect information in 37 percent of the queries tested, whereas ChatGPT Search incorrectly identified 67 percent (134 out of 200) of articles queried. Grok 3 demonstrated the highest error rate, at 94 percent.

A graph from CJR shows

A graph from CJR shows “confidently wrong” search results. Credit: CJR

For the tests, researchers fed direct excerpts from actual news articles to the AI models, then asked each model to identify the article’s headline, original publisher, publication date, and URL. They ran 1,600 queries across the eight different generative search tools.

The study highlighted a common trend among these AI models: rather than declining to respond when they lacked reliable information, the models frequently provided confabulations—plausible-sounding incorrect or speculative answers. The researchers emphasized that this behavior was consistent across all tested models, not limited to just one tool.

Surprisingly, premium paid versions of these AI search tools fared even worse in certain respects. Perplexity Pro ($20/month) and Grok 3’s premium service ($40/month) confidently delivered incorrect responses more often than their free counterparts. Though these premium models correctly answered a higher number of prompts, their reluctance to decline uncertain responses drove higher overall error rates.

Issues with citations and publisher control

The CJR researchers also uncovered evidence suggesting some AI tools ignored Robot Exclusion Protocol settings, which publishers use to prevent unauthorized access. For example, Perplexity’s free version correctly identified all 10 excerpts from paywalled National Geographic content, despite National Geographic explicitly disallowing Perplexity’s web crawlers.

AI search engines cite incorrect sources at an alarming 60% rate, study says Read More »