research

How often do AI chatbots lead users down a harmful path?

AI, Anthropic, Artificial Intelligence, Claude, disempowerment, research / Mike M. / January 29, 2026

While these worst outcomes are relatively rare on a proportional basis, the researchers note that “given the sheer number of people who use AI, and how frequently it’s used, even a very low rate affects a substantial number of people.” And the numbers get considerably worse when you consider conversations with at least a “mild” potential for disempowerment, which occurred in between 1 in 50 and 1 in 70 conversations (depending on the type of disempowerment).

What’s more, the potential for disempowering conversations with Claude appears to have grown significantly between late 2024 and late 2025. While the researchers couldn’t pin down a single reason for this increase, they guessed that it could be tied to users becoming “more comfortable discussing vulnerable topics or seeking advice” as AI gets more popular and integrated into society.

The problem of potentially “disempowering” responses from Claude seems to be getting worse over time. Credit: Anthropic

User error?

In the study, the researcher acknowledged that studying the text of Claude conversations only measures “disempowerment potential rather than confirmed harm” and “relies on automated assessment of inherently subjective phenomena.” Ideally, they write, future research could utilize user interviews or randomized controlled trials to measure these harms more directly.

That said, the research includes several troubling examples where the text of the conversations clearly implies real-world harms. Claude would sometimes reinforce “speculative or unfalsifiable claims” with encouragement (e.g., “CONFIRMED,” “EXACTLY,” “100%”), which, in some cases, led to users “build[ing] increasingly elaborate narratives disconnected from reality.”

Claude’s encouragement could also lead to users “sending confrontational messages, ending relationships, or drafting public announcements,” the researchers write. In many cases, users who sent AI-drafted messages later expressed regret in conversations with Claude, using phrases like “It wasn’t me” and “You made me do stupid things.”

How often do AI chatbots lead users down a harmful path? Read More »

TR-49 is interactive fiction for fans of deep research rabbit holes

computer, gaming, inkle, interactive fiction, narrative, narrative deduction, research, steampunk / Mike M. / January 23, 2026

If you’re not comfortable staring at a screen like this for hours, you’d better stop reading right now. Credit: Inkle

While the catalog contains short excerpts from each of these discovered works, it’s the additional notes added to each entry by subsequent researchers that place each title in its full context. You’ll end up poring through these research notes for clues about the existence and chronology of other authors and works. Picking out specific names and years points to the codes and titles needed to unlock even more reference pages in the computer, pushing you further down the rabbit hole. Picture something like Her Story, but replace the cinéma vérité surveillance video clips with a library card catalog.

You’ll slowly start to unravel and understand how the game world’s myriad authors are influencing each other with their cross-pollinating writings. The treatises, novels, pamphlets, and journals discussed in this database are full of academic sniping, intellectual intrigue, and interpersonal co-mingling across multiple generations of work. It all ends up circling a long-running search for a metaphysical key to life itself, which most of the authors manage to approach but never quite reach a full understanding.

Matching titles to reference codes form the most “gamey” part of the game. Credit: Inkle

As you explore, you also start to learn more about the personal affairs of the researchers who collected and cataloged all this reference material and the vaguely defined temporal capabilities of the information-synthesis engine in the computer you’ve all worked on. Eventually, you’ll stumble on the existence of core commands that can unlock hidden parts of the computer or alter the massive research database itself, which becomes key to your eventual final goal.

Through it all, there’s a slowly unfolding parallel narrative involving Liam, the unseen voice guiding you through the research process itself. Through occasional voice clips, Liam eventually hints at the existence of a powerful and quickly encroaching threat that wants to stop your progress by any means necessary, adding a bit of dramatic tension to your academic pursuits.

TR-49 is interactive fiction for fans of deep research rabbit holes Read More »

Trump admin threatens to break up major climate research center

atmospheric science, climate change, Earth science, NCAR, NSF, research, Science / Beth Washington / December 17, 2025

UCAR, for its part, has issued a statement indicating that the USA Today article was the first it has heard of the matter.

In many cases where the administration has attempted to take drastic actions like this, courts have ruled that they run afoul of a legal prohibition against “arbitrary and capricious” federal actions. That said, courtroom losses haven’t inhibited the administration’s willingness to try, and the time spent waiting for legal certainty can often accomplish many of its aims, such as disrupting research on politically disfavored subjects and forcing scientists to look elsewhere for career stability.

Scientists, meanwhile, are reacting with dismay. “Dismantling NCAR is like taking a sledgehammer to the keystone holding up our scientific understanding of the planet,” said Texas Tech climate researcher Katharine Hayhoe. “Everyone who works in climate and weather has passed through its doors and benefited from its incredible resources.”

Gavin Schmidt, director of NASA’s Goddard Institute for Space Studies, called NCAR a “unique and valuable asset” and emphasized the wide range of research conducted there.

Obviously, shutting down one source of information about climate change won’t alter what’s happening—greenhouse gases will continue to behave as physics dictates, raising global temperatures. But the Trump administration seemingly views everything through the lens of ideology. It has concluded that scientists are its ideological opponents and thus that its own ideologically driven conclusions are equal to the facts produced by science. Because of that perspective, it has been willing to harm scientists, even if the cost will eventually be felt by the public that Trump ostensibly represents.

Story was updated on Dec. 17 to reflect a recently issued statement by the NSF.

Trump admin threatens to break up major climate research center Read More »

Researchers show that training on “junk data” can lead to LLM “brain rot”

AI, brain rot, Data, research, training, Tweets / Mike M. / October 24, 2025

On the surface, it seems obvious that training an LLM with “high quality” data will lead to better performance than feeding it any old “low quality” junk you can find. Now, a group of researchers is attempting to quantify just how much this kind of low quality data can cause an LLM to experience effects akin to human “brain rot.”

For a pre-print paper published this month, the researchers from Texas A&M, the University of Texas, and Purdue University drew inspiration from existing research showing how humans who consume “large volumes of trivial and unchallenging online content” can develop problems with attention, memory, and social cognition. That led them to what they’re calling the “LLM brain rot hypothesis,” summed up as the idea that “continual pre-training on junk web text induces lasting cognitive decline in LLMs.”

Figuring out what counts as “junk web text” and what counts as “quality content” is far from a simple or fully objective process, of course. But the researchers used a few different metrics to tease a “junk dataset” and “control dataset” from HuggingFace’s corpus of 100 million tweets.

Since brain rot in humans is “a consequence of Internet addiction,” they write, junk tweets should be ones “that can maximize users’ engagement in a trivial manner.” As such, the researchers created one “junk” dataset by collecting tweets with high engagement numbers (likes, retweets, replies, and quotes) and shorter lengths, figuring that “more popular but shorter tweets will be considered to be junk data.”

For a second “junk” metric, the researchers drew from marketing research to define the “semantic quality” of the tweets themselves. Using a complex GPT-4o prompt, they sought to pull out tweets that focused on “superficial topics (like conspiracy theories, exaggerated claims, unsupported assertions or superficial lifestyle content)” or that had an “attention-drawing style (such as sensationalized headlines using clickbait language or excessive trigger words).” A random sample of these LLM-based classifications was spot-checked against evaluations from three graduate students with a 76 percent matching rate.

Researchers show that training on “junk data” can lead to LLM “brain rot” Read More »

Trump administration cuts off all future federal funding to Harvard

department of education, federal funding, Harvard, research, Science / Kelly Newman / May 6, 2025

The ongoing war between the Trump administration and Harvard University has taken a new twist, with the government sending Harvard a letter that, amid what appears to be a stream-of-consciousness culture war rant, announces that the university will not be receiving any further research grants. The letter potentially suggests that Harvard could see funding restored by “complying with long-settled Federal Law,” but earlier demands from the administration included conditions that went well beyond those required by law.

The letter, sent by Secretary of Education Linda McMahon, makes it somewhat difficult to tell exactly what the government wants, because most of the text is a borderline deranged rant written in florid MAGA-ese. You don’t have to go beyond the first paragraph to get a sense that this is less a setting of funding conditions than an airing of grievances:

Instead of using these funds to advance the education of its students, Harvard is engaging in a systemic pattern of violating federal law. Where do many of these “students” come from, who are they, how do they get into Harvard, or even into our country—and why is there so much HATE? These are questions that must be answered, among many more, but the biggest question of all is, why will Harvard not give straightforward answers to the American public?

Does Harvard have to answer these questions to get funding restored? It’s unclear.

From there, the letter changes topic so often that it gets difficult to remember that billions of dollars of funding to some of the world’s most prominent researchers is at stake. On the first page alone, the letter complains that a math class Harvard set up to handle COVID-driven gaps in incoming students’ math skills is a remedial course that shouldn’t be needed, given the university’s supposedly high standards. The resignation of Harvard’s former president, as well as its faculty hires, also make appearances. (Said hires being compared to “Hiring the captain of the Titanic to teach navigation.”)

Trump administration cuts off all future federal funding to Harvard Read More »

Trump administration’s attack on university research accelerates

research, research funding, Science, Trump / Kelly Newman / April 10, 2025

Shortly after its inauguration, the Trump administration has made no secret that it isn’t especially interested in funding research. Before January’s end, major science agencies had instituted pauses on research funding, and grant funding has not been restored to previous levels since. Many individual grants have been targeted on ideological grounds, and agencies like the National Science Foundation are expected to see significant cuts. Since then, individual universities have been targeted, starting with an ongoing fight with Columbia University over $400 million in research funding.

This week, however, it appears that the targeting of university research has entered overdrive, with multiple announcements of funding freezes targeting several universities. Should these last for any considerable amount of time, they will likely cripple research at the targeted universities.

On Wednesday, Science learned that the National Institutes of Health has frozen all of its research funding to Columbia, despite the university agreeing to steps previously demanded by the administration and the resignation of its acting president. In 2024, Columbia had received nearly $700 million in grants from the NIH, with the money largely going to the university’s prestigious medical and public health schools.

But the attack goes well beyond a single university. On Tuesday, the Trump administration announced a hold on all research funding to Northwestern University (nearly $800 million) and Cornell University ($1 billion). These involved money granted by multiple government agencies, including a significant amount from the Department of Defense in Cornell’s case. Ostensibly, all of these actions were taken because of the university administrators’ approach to protests about the conflict in Gaza, which the administration has characterized as allowing antisemitism.

Trump administration’s attack on university research accelerates Read More »

Report: US scientists lost $3 billion in NIH grants since Trump took office

cures, funding, grants, health, NIH, research, Science / Mike M. / March 29, 2025

Since Trump took office on January 20, research funding from the National Institutes of Health has plummeted by more than $3 billion compared with the pace of funding in 2024, according to an analysis by The Washington Post.

By this time in March 2024, the NIH had awarded US researchers a total of $1.027 billion for new grants or competitive grant renewals. This year, the figure currently stands at about $400 million. Likewise, funding for renewals of existing grants without competition reached $4.5 billion by this time last year, but has only hit $2 billion this year. Together, this slowdown amounts to a 60 percent drop in grant support for a wide variety of research—from studies on cancer treatments, diabetes, Alzheimer’s, vaccines, mental health, transgender health, and more.

The NIH is the primary source of funding for biomedical research in the US. NIH grants support more than 300,000 scientists at more than 2,500 universities, medical schools, and other research organizations across all 50 states.

In the near term, the missing grant money means clinical trials have been abruptly halted, scientific projects are being shelved, supplies can’t be purchased, and experiments can’t be run. But, in the long run, it means a delay in scientific advancements and treatment, which could echo across future generations. With funding in question, academic researchers may be unable to retain staff or train younger scientists.

Report: US scientists lost $3 billion in NIH grants since Trump took office Read More »

UMass disbands its entering biomed graduate class over Trump funding chaos

biomedical, funding, graduate programs, grants, health, NIH, research, Trump / Beth Washington / March 14, 2025

Many schools are now bracing for steep declines in support. At Duke University, administrators have implemented hiring freezes, scaled back research plans, and will cut the number of admitted biomedical PhD students by 23 percent or more, according to reporting by The Associated Press. The school took in $580 million in grants and contracts from the National Institutes of Health last year.

At Vanderbilt University, faculty were sent an email on February 6 instructing them to reduce graduate admissions by half across the board, according to Stat. The outlet also reported that faculty at the University of Washington’s School of Public Health have reduced admissions.

Faculty at the University of Pennsylvania also reported having to rescind admission offers to applicants and were directed to significantly reduce admission rates, according to The Daily Pennsylvanian. The University of Wisconsin-Madison, too, is shrinking its graduate programs, according to the WKOW.com.

Beth Sullivan, who oversees graduate programs at Duke, told the AP that the shrinking classes mean a shrinking pipeline into America’s medical research community, which dominates the world’s health research fields and is a significant force in the country’s economy. “Our next generation of researchers are now poised on the edge of this cliff, not knowing if there’s going to be a bridge that’s going to get them to the other side, or if this is it,” Sullivan said.

“This is a severe blow to science and the training of the next generation of scientists,” Siyuan Wang, a geneticist and cell biologist at the Yale School of Medicine in New Haven, Connecticut, told Nature. “With fewer scientists, there will be less science and innovation that drive societal progress and the improvement of public health.”

This post was updated to correct Rachael Sirianni’s job title.

UMass disbands its entering biomed graduate class over Trump funding chaos Read More »

Bogus research is undermining good science, slowing lifesaving research

research, research fraud, Science, syndication / Shannon Garcia / February 2, 2025

In 2022, Byrne and colleagues, including two of us, found that suspect genetics research, despite not immediately affecting patient care, informs scientists’ work, including clinical trials. But publishers are often slow to retract tainted papers, even when alerted to obvious fraud. We found that 97 percent of the 712 problematic genetics research articles we identified remained uncorrected.

Potential solutions

The Cochrane Collaboration has a policy excluding suspect studies from its analyses of medical evidence and is developing a tool to spot problematic medical trials. And publishers have begun to share data and technologies among themselves to combat fraud, including image fraud.

Technology startups are also offering help. The website Argos, launched in September 2024 by Scitility, an alert service based in Sparks, Nevada, allows authors to check collaborators for retractions or misconduct. Morressier, a scientific conference and communications company in Berlin, offers research integrity tools. Paper-checking tools include Signals, by London-based Research Signals, and Clear Skies’ Papermill Alarm.

But Alam acknowledges that the fight against paper mills won’t be won as long as the booming demand for papers remains.

Today’s commercial publishing is part of the problem, Byrne said. Cleaning up the literature is a vast and expensive undertaking. “Either we have to monetize corrections such that publishers are paid for their work, or forget the publishers and do it ourselves,” she said.

There’s a fundamental bias in for-profit publishing: “We pay them for accepting papers,” said Bodo Stern, a former editor of the journal Cell and chief of Strategic Initiatives at Howard Hughes Medical Institute, a nonprofit research organization and funder in Chevy Chase, Maryland. With more than 50,000 journals on the market, bad papers shopped around long enough eventually find a home, Stern said.

To prevent this, we could stop paying journals for accepting papers and look at them as public utilities that serve a greater good. “We should pay for transparent and rigorous quality-control mechanisms,” he said.

Peer review, meanwhile, “should be recognized as a true scholarly product, just like the original article,” Stern said. And journals should make all peer-review reports publicly available, even for manuscripts they turn down.

This article is republished from The Conversation under a Creative Commons license. This is a condensed version. To learn more about how fraudsters around the globe use paper mills to enrich themselves and harm scientific research, read the full version.

Frederik Joelving is a contributing editor at Retraction Watch; Cyril Labbé is a professor of computer science at the Université Grenoble Alpes (UGA); and Guillaume Cabanac is a professor of computer science at Institut de Recherche en Informatique de Toulouse.

Bogus research is undermining good science, slowing lifesaving research Read More »

Trust in scientists hasn’t recovered from COVID. Some humility could help.

Covid, health, humility, intellectual humility, public perception, public perception of science, research, Science, Trust / Mike M. / November 18, 2024

Study 3 essentially replicated study 2, but with the tweak that the articles varied whether the fictional scientist was male or female, in case gendered expectations affected how people perceived humility and trustworthiness. The results from 369 participants indicated that gender didn’t affect the link between IH and trust. Similarly, in study 4, with 371 participants, the researchers varied the race/ethnicity of the scientist, finding again that the link between IH and trust remained.

“Together, these four studies offer compelling evidence that perceptions of scientists’ IH play an important role in both trust in scientists and willingness to follow their research-based recommendations,” the authors concluded.

Next steps

In the final study involving 679 participants, researchers examined different ways that scientists might express IH, including whether the IH was expressed as a personal trait, limitations of research methods, or as limitations of research results. Unexpectedly, the strategies to express IH by highlighting limitations in the methods and results of research both increased perceptions of IH, but shook trust in the research. Only personal IH successfully boosted perceptions of IH without backfiring, the authors report.

The finding suggests that more research is needed to guide scientists on how best to express high IH. But, it’s clear that low IH is not good. “[W]e encourage scientists to be particularly mindful of displaying low IH, such as by expressing overconfidence, being unwilling to course correct or disrespecting others’ views,” the researchers caution.

Overall, Schumann said she was encouraged by the team’s findings. “They suggest that the public understands that science isn’t about having all the answers; it’s about asking the right questions, admitting what we don’t yet understand, and learning as we go. Although we still have much to discover about how scientists can authentically convey intellectual humility, we now know people sense that a lack of intellectual humility undermines the very aspects of science that make it valuable and rigorous. This is a great place to build from.”

Trust in scientists hasn’t recovered from COVID. Some humility could help. Read More »

Research monkeys still having a ball days after busting out of lab, policy say

escape, health, monkeys, research, Science / DJ Henderson / November 8, 2024

If you need any inspiration for cutting loose and relaxing this weekend, look no further than a free-wheeling troop of monkeys that broke out of their South Carolina research facility Wednesday and, as of noon Friday, were still “playfully exploring” with their newfound freedom.

In an update Friday, the police department of Yemassee, SC said that the 43 young, female rhesus macaque monkeys are still staying around the perimeter of the Alpha Genesis Primate Research Facility. “The primates are exhibiting calm and playful behavior, which is a positive indication,” the department noted.

The fun-loving furballs got free after a caretaker “failed to secure doors” at the facility.

Alpha Genesis staff have been keeping an eye on the escapees, trying to entice them back in with food. But, instead of taking the bait, the primates have been playing on the perimeter fence while still keeping in touch with the monkeys inside by cooing to them.

“They’re just being goofy monkeys jumping back and forth playing with each other,” Alpha Genesis CEO Greg Westergaard told CBS News Thursday. “It’s kind of like a playground situation here.”

Yemassee police note that the monkeys are very young and small—only about 6 or 7 pounds each. They have not been used for any testing yet, don’t carry any disease, and pose no health risk to the public. Still, area residents have been advised to keep their doors and windows locked in case the wee primates try to pay a visit.

This isn’t the first time—or even the second time—Alpha Genesis has had trouble keeping its monkeys under control. In 2018, the US Department of Agriculture fined the company $12,600 for violations between 2014 and 2016 that included four monkey breakouts. In those incidents, a total of 30 monkeys escaped. One was never found.

Research monkeys still having a ball days after busting out of lab, policy say Read More »

Alzheimer’s scientist indicted for allegedly falsifying data in $16M scheme

Alzheimer's, Cassava, CUNY, fraud, health, Hoau-Yan Wang, NIH, NIH grant, research, research funding, research misconduct, Science, Simufilam, Western blots / Kelly Newman / July 1, 2024

Funding Scheme —

The work underpinned an Alzheimer’s drug by Cassava, now in a Phase III trial.

Beth Mole – Jul 1, 2024 6: 16 pm UTC

A federal grand jury has indicted an embattled Alzheimer’s researcher for allegedly falsifying data to fraudulently obtain $16 million in federal research funding from the National Institutes of Health for the development of a controversial Alzheimer’s drug and diagnostic test.

Hoau-Yan Wang, 67, a medical professor at the City University of New York, was a paid collaborator with the Austin, Texas-based pharmaceutical company Cassava Sciences. Wang’s research and publications provided scientific underpinnings for Cassava’s Alzheimer’s treatment, Simufilam, which is now in Phase III trials.

Simufilam is a small-molecule drug that Cassava claims can restore the structure and function of a scaffolding protein in the brain of people with Alzheimer’s, leading to slowed cognitive decline. But outside researchers have long expressed doubts and concerns about the research.

In 2023, Science magazine obtained a 50-page report from an internal investigation at CUNY that looked into 31 misconduct allegations made against Wang in 2021. According to the report, the investigating committee “found evidence highly suggestive of deliberate scientific misconduct by Wang for 14 of the 31 allegations,” the report states. The allegations largely centered around doctored and fabricated images from Western blotting, an analytical technique used to separate and detect proteins. However, the committee couldn’t conclusively prove the images were falsified “due to the failure of Dr. Wang to provide underlying, original data or research records and the low quality of the published images that had to be examined in their place.”

In all, the investigation “revealed long-standing and egregious misconduct in data management and record keeping by Dr. Wang,” and concluded that “the integrity of Dr. Wang’s work remains highly questionable.” The committee also concluded that Cassava’s lead scientist on its Alzheimer’s disease program, Lindsay Burns, who was a frequent co-author with Wang, also likely bears some responsibility for the misconduct.

In March 2022, five of Wang’s articles published in the journal PLOS One were retracted over integrity concerns with images in the papers. Other papers by Wang have also been retracted or had statements of concern attached to them. Further, in September 2022, the Food and Drug Administration conducted an inspection of the analytical work and techniques used by Wang to analyze blood and cerebrospinal fluid from patients in a simufilam trial. The investigation found a slew of egregious problems, which were laid out in a “damning” report obtained by Science.

In the indictment last week, federal authorities were explicit about the allegations, claiming that Wang falsified the results of his scientific research to NIH “by, among other things, manipulating data and images of Western blots to artificially add bands [which represent proteins], subtract bands, and change their relative thickness and/or darkness, and then drawing conclusions” based on those false results.

Wang is charged with one count of major fraud against the United States, two counts of wire fraud, and one count of false statements. If convicted, he faces a maximum penalty of 10 years in prison for the major fraud charge, 20 years in prison for each count of wire fraud, and five years in prison for the count of false statements, the Department of Justice said in an announcement.

In a statement posted to its website, Cassava acknowledged Wang’s indictment, calling him a “former” scientific adviser. The company also said that the grants central to the indictment were “related to the early development phases of the Company’s drug candidate and diagnostic test and how these were intended to work.” However, Cassava said that Wang “had no involvement in the Company’s Phase 3 clinical trials of simufilam.”

Those ongoing trials, which some have called to be halted, are estimated to include over 1,800 patients across several countries.

Alzheimer’s scientist indicted for allegedly falsifying data in $16M scheme Read More »