Author name: Beth Washington

motorola-razr-and-razr-ultra-(2025)-review:-cool-as-hell,-but-too-much-ai

Motorola Razr and Razr Ultra (2025) review: Cool as hell, but too much AI


The new Razrs are sleek, capable, and overflowing with AI features.

Razr Ultra and Razr (2025)

Motorola’s 2025 Razr refresh includes its first Ultra model. Credit: Ryan Whitwam

Motorola’s 2025 Razr refresh includes its first Ultra model. Credit: Ryan Whitwam

For phone nerds who’ve been around the block a few times, the original Motorola Razr is undeniably iconic. The era of foldables has allowed Motorola to resurrect the Razr in an appropriately flexible form, and after a few generations of refinement, the 2025 Razrs are spectacular pieces of hardware. They look great, they’re fun to use, and they just about disappear in your pocket.

The new Razrs also have enormous foldable OLEDs, along with external displays that are just large enough to be useful. Moto has upped its design game, offering various Pantone shades with interesting materials and textures to make the phones more distinctive, but Motorola’s take on mobile AI could use some work, as could its long-term support policy. Still, these might be the coolest phones you can get right now.

An elegant tactile experience

Many phone buyers couldn’t care less about how a phone’s body looks or feels—they’ll just slap it in a case and never look at it again. Foldables tend not to fit as well in cases, so the physical design of the Razrs is important. The good news is that Motorola has refined the foldable formula with an updated hinge and some very interesting material choices.

Razr Ultra back

The Razr Ultra is available with a classy wood back.

Credit: Ryan Whitwam

The Razr Ultra is available with a classy wood back. Credit: Ryan Whitwam

The 2025 Razrs come in various colors, all of which have interesting material choices for the back panel. There are neat textured plastics, wood, vegan leather, and synthetic fabrics. We’ve got wood (Razr Ultra) and textured plastic (Razr) phones to test—they look and feel great. The Razr is very grippy, and the wooden Ultra looks ultra-stylish, though not quite as secure in the hand. The aluminum frames are also colored to match the back with a smooth matte finish. Motorola has gone to great lengths to make these phones feel unique without losing the premium vibe. It’s nice to see a phone maker do that without resorting to a standard glass sandwich body.

The buttons are firm and tactile, but we’re detecting just a bit of rattle in the power button. That’s also where you’ll find the fingerprint sensor. It’s reasonably quick and accurate, whether the phone is open or closed. The Razr Ultra also has an extra AI button on the opposite side, which is unnecessary, for reasons we’ll get to later. And no, you can’t remap it to something else.

Motorola Razr 2025

The Razrs have a variety of neat material options.

Credit: Ryan Whitwam

The Razrs have a variety of neat material options. Credit: Ryan Whitwam

The front of the flip on these phones features a big sheet of Gorilla Glass Ceramic, which is supposedly similar to Apple’s Ceramic Shield glass. That should help ward off scratches. The main camera sensors poke through this front OLED, which offers some interesting photographic options we’ll get to later. The Razr Ultra has a larger external display, clocking in at 4 inches. The cheaper Razr gets a smaller 3.6-inch front screen, but that’s still plenty of real estate, even with the camera lenses at the bottom.

Specs at a glance: 2025 Motorola Razrs
Motorola Razr ($699.99) Motorola Razr+ ($999.99) Motorola Razr Ultra ($1,299.99)
SoC MediaTek Dimensity 7400X Snapdragon 8s Gen 3 Snapdragon 8 Elite
Memory 8GB 12GB 16GB
Storage 256GB 256GB 512GB, 1TB
Display 6.9″ foldable OLED (120 Hz, 2640 x 1080), 3.6″ external (90 Hz) 6.9″ foldable OLED (165 Hz, 2640 x 1080), 4″ external (120 Hz, 1272 x 1080) 7″ foldable OLED (165 Hz, 2992 x 1224), 4″ external (165 Hz)
Cameras 50 MP f/1.7 OIS primary; 13 MP f/2.2  ultrawide, 32 MP selfie 50 MP f/1.7 OIS primary; 50 MP 2x telephoto f/2.0, 32 MP selfie 50 MP f/1.8 OIS primary, 50 MP ultrawide + macro, f/2.0, 50 MP selfie
Software Android 15 Android 15 Android 15
Battery 4,500 mAh, 30 W wired charging, 15 W wireless charging 4,000 mAh, 45 W wired charging, 15 W wireless charging 4,700 mAh, 68 W wired charging, 15 W wireless charging
Connectivity Wi-Fi 6e, NFC, Bluetooth 5.4, sub-6 GHz 5G, USB-C 2.0 Wi-Fi 7, NFC, Bluetooth 5.4, sub-6 GHz 5G, USB-C 2.0 Wi-Fi 7, NFC, Bluetooth 5.4, sub-6 GHz 5G, USB-C 2.0
Measurements Open: 73.99 x 171.30 x 7.25 mm;

Closed: 73.99 x 88.08 x 15.85 mm; 188 g
Open: 73.99 x 171.42 x 7.09 mm;

Closed: 73.99 x 88.09x 15.32 mm; 189 g
Open: 73.99 x 171.48 x 7.19 mm;

Closed: 73.99 x 88.12 x 15.69 mm; 199 g

Motorola says the updated foldable hinge has been reinforced with titanium. This is the most likely point of failure for a flip phone, but the company’s last few Razrs already felt pretty robust. It’s good that Moto is still thinking about durability, though. The hinge is smooth, allowing you to leave the phone partially open, but there are magnets holding the two halves together with no gap when closed. The magnets also allow for a solid snap when you shut it. Hanging up on someone is so, so satisfying when you’re using a Razr flip phone.

Flip these phones open, and you get to the main event. The Razr has a 6.9-inch, 2640×1080 foldable OLED, and the Ultra steps up to 7 inches at an impressive 2992×1224. These phones have almost exactly the same dimensions, so the additional bit of Ultra screen comes from thinner bezels. Both phones are extremely tall when open, but they’re narrow enough to be usable in one hand. Just don’t count on reaching the top of the screen easily. While Motorola has not fully eliminated the display crease, it’s much smoother and less noticeable than it is on Samsung’s or Google’s foldables.

Motorola Razr Ultra

The Razr Ultra has a 7-inch foldable OLED.

Credit: Ryan Whitwam

The Razr Ultra has a 7-inch foldable OLED. Credit: Ryan Whitwam

The Razr can hit 3,000 nits of brightness, and the $1,300 Razr Ultra tops out at 4,500 nits. Both are bright enough to be usable outdoors, though the Ultra is noticeably brighter. However, both suffer from the standard foldable drawbacks of having a plastic screen. The top layer of the foldable screen is a non-removable plastic protector, which has very high reflectivity that makes it harder to see the display. That plastic layer also means you have to be careful not to poke or scratch the inner screen. It’s softer than your fingernails, so it’s not difficult to permanently damage the top layer.

Too much AI

Motorola’s big AI innovation for last year’s Razr was putting Gemini on the phone, making it one of the first to ship with Google’s generative AI system. This time around, it has AI features based on Gemini, Meta Llama, Perplexity, and Microsoft Copilot. It’s hard to say exactly how much AI is worth having on a phone with the rapid pace of change, but Motorola has settled on the wrong amount. To be blunt, there’s too much AI. What is “too much” in this context? This animation should get the point across.

Moto AI

Motorola’s AI implementation is… a lot.

Credit: Ryan Whitwam

Motorola’s AI implementation is… a lot. Credit: Ryan Whitwam

The Ask and Search bar appears throughout the UI, including as a floating Moto AI icon. It’s also in the app drawer and is integrated with the AI button on the Razr Ultra. You can use it to find settings and apps, but it’s also a full LLM (based on Copilot) for some reason. Gemini is a better experience if you’re looking for a chatbot, though.

Moto AI also includes a raft of other features, like Pay Attention, which can record and summarize conversations similar to the Google recorder app. However, unlike that app, the summarizing happens in the cloud instead of locally. That’s a possible privacy concern. You also get Perplexity integration, allowing you to instantly search based on your screen contents. In addition, the Perplexity app is preloaded with a free trial of the premium AI search service.

There’s so much AI baked into the experience that it can be difficult to keep all the capabilities straight, and there are some more concerning privacy pitfalls. Motorola’s Catch Me Up feature is a notification summarizer similar to a feature of Apple Intelligence. On the Ultra, this feature works locally with a Llama 3 model, but the less powerful Razr can’t do that. It sends your notifications to a remote server for processing when you use Catch Me Up. Motorola says data is “anonymous and secure” and it does not retain any user data, but you have to put a lot of trust in a faceless corporation to send it all your chat notifications.

Razr Ultra and Razr (2025)

The Razrs have additional functionality if you prop them up in “tent” or “stand” mode.

Credit: Ryan Whitwam

The Razrs have additional functionality if you prop them up in “tent” or “stand” mode. Credit: Ryan Whitwam

If you can look past Motorola’s frenetic take on mobile AI, the version of Android 15 on the Razrs is generally good. There are a few too many pre-loaded apps and experiences, but it’s relatively simple to debloat these phones. It’s quick, doesn’t diverge too much from the standard Android experience, and avoids duplicative apps.

We appreciate the plethora of settings and features for the external display. It’s a much richer experience than you get with Samsung’s flip phones. For example, we like how easy it is to type out a reply in a messaging app without even opening the phone. In fact, you can run any app on the phone without opening it, even though many of them won’t work quite right on a smaller square display. Still, it can be useful for chat apps, email, and other text-based stuff. We also found it handy for using smart home devices like cameras and lights. There are also customizable panels for weather, calendar, and Google “Gamesnack” games.

Razr Ultra and Razr (2025)

The Razr Ultra (left) has a larger screen than the Razr (right).

Credit: Ryan Whitwam

The Razr Ultra (left) has a larger screen than the Razr (right). Credit: Ryan Whitwam

Motorola promises three years of full OS updates and an additional year of security patches. This falls far short of the seven-year update commitment from Samsung and Google. For a cheaper phone like the Razr, four years of support might be fine, but it’s harder to justify that when the Razr Ultra costs as much as a Galaxy S25 Ultra.

One fast foldable, one not so much

Motorola is fond of saying the Razr Ultra is the fastest flip phone in the world, which is technically true. It has the Snapdragon 8 Elite chip with 16GB of RAM, but we expect to see the Elite in Samsung’s 2025 foldables later this year. For now, though, the Razr Ultra stands alone. The $700 Razr runs a Mediatek Dimensity 7400X, which is a distinctly midrange processor with just 8GB of RAM.

Razr geekbench

The Razr Ultra gets close to the S25.

Credit: Ryan Whitwam

The Razr Ultra gets close to the S25. Credit: Ryan Whitwam

In daily use, neither phone feels slow. Side by side, you can see the Razr is slower to open apps and unlock, and the scrolling exhibits occasional jank. However, it’s not what we’d call a slow phone. It’s fine for general smartphone tasks like messaging, browsing, and watching videos. You may have trouble with gaming, though. Simple games run well enough, but heavy 3D titles like Diablo Immortal are rough with the Dimensity 7400X.

The Razr Ultra is one of the fastest Android phones we’ve tested, thanks to the Snapdragon chip. You can play complex games and multitask to your heart’s content without fear of lag. It does run a little behind the Galaxy S25 series in benchmarks, but it thankfully doesn’t get as toasty as Samsung’s phones.

We never expect groundbreaking battery life from foldables. The hinge takes up space, which limits battery capacity. That said, Motorola did fairly well cramming a 4,700 mAh battery in the Razr Ultra and a 4,500 mAh cell in the Razr.

Based on our testing, both of these phones should last you all day. The large external displays can help by giving you just enough information that you don’t have to use the larger, more power-hungry foldable OLED. If you’re playing games or using the main display exclusively, you may find the Razrs just barely make it to bedtime. However, no matter what you do, these are not multi-day phones. The base model Razr will probably eke out a few more hours, even with its smaller battery, due to the lower-power MediaTek processor. The Snapdragon 8 Elite in the Razr Ultra really eats into the battery when you take advantage of its power.

Motorola Razr Ultra

The Razrs are extremely pocketable.

Credit: Ryan Whitwam

The Razrs are extremely pocketable. Credit: Ryan Whitwam

While the battery life is just this side of acceptable, the Razr Ultra’s charging speed makes this less of a concern. This phone hits an impressive 68 W, which is faster than the flagship phones from Google, Samsung, and Apple. Just a few minutes plugged into a compatible USB-C charger and you’ve got enough power that you can head out the door without worry. Of course, the phone doesn’t come with a charger, but we’ve tested a few recent models, and they all hit the max wattage.

OK cameras with super selfies

Camera quality is another area where foldable phones tend to compromise. The $1,300 Razr Ultra has just two sensors—a 50 MP primary sensor and a 50 MP ultrawide lens. The $700 Razr has a slightly different (and less capable) 50 MP primary camera and a 13 MP ultrawide. There are also selfie cameras peeking through the main foldable OLED panels—50 MP for the Ultra and 32 MP for the base model.

Motorola Razr 2025 in hand

The cheaper Razr has a smaller external display, but it’s still large enough to be usable.

Credit: Ryan Whitwam

The cheaper Razr has a smaller external display, but it’s still large enough to be usable. Credit: Ryan Whitwam

Motorola’s Razrs tend toward longer exposures compared to Pixels—they’re about on par with Samsung phones. That means capturing fast movement indoors is difficult, and you may miss your subject outside due to a perceptible increase in shutter lag compared to Google’s phones. Images from the base model Razr’s primary camera also tend to look a bit more overprocessed than they do on the Ultra, which leads to fuzzy details and halos in bright light.

Razr Ultra outdoors. Ryan Whitwam

That said, Motorola’s partnership with Pantone is doing some good. The colors in our photos are bright and accurate, capturing the vibe of the scene quite well. You can get some great photos of stationary or slowly moving subjects.

Razr 2025 indoor medium light. Ryan Whitwam

The 50 MP ultrawide camera on the Razr Ultra has a very wide field of view, but there’s little to no distortion at the edges. The colors are also consistent between the two sensors, but that’s not always the case for the budget Razr. Its ultrawide camera also lacks detail compared to the Ultra, which isn’t surprising considering the much lower resolution.

You should really only use the dedicated front-facing cameras for video chat. For selfies, you’ll get much better results by taking advantage of the Razr’s distinctive form factor. When closed, the Razrs let you take selfies with the main camera sensors, using the external display as the viewfinder. These are some of the best selfies you’ll get with a smartphone, and having the ultrawide sensor makes group shots excellent as well.

Flip phones are still fun

While we like these phones for what they are, they are objectively not the best value. Whether you’re looking at the Razr or the Razr Ultra, you can get more phone for the same money from other companies—more cameras, more battery, more updates—but those phones don’t fold in half. There’s definitely a cool-factor here. Flip phones are stylish, and they’re conveniently pocket-friendly in a world where giant phones barely fit in your pants. We also like the convenience and functionality of the external displays.

Motorola Razr Ultra

The Razr Ultra is all screen from the front.

Credit: Ryan Whitwam

The Razr Ultra is all screen from the front. Credit: Ryan Whitwam

The Razr Ultra makes the usual foldable compromises, but it’s as capable a flip phone as you’ll find right now. It’s blazing fast, it has two big displays, and the materials are top-notch. However, $1,300 is a big ask.

Is the Ultra worth $500 more than the regular Razr? Probably not. Most of what makes the foldable Razrs worth using is present on the cheaper model. You still get the solid construction, cool materials, great selfies, and a useful (though slightly smaller) outer display. Yes, it’s a little slower, but it’s more than fast enough as long as you’re not a heavy gamer. Just be aware of the potential for Moto AI to beam your data to the cloud.

There is also the Razr+, which slots in between the models we have tested at $1,000. It’s faster than the base model and has the same large external display as the Ultra. This model could be the sweet spot if neither the base model nor the flagship does it for you.

The good

  • Sleek design with distinctive materials
  • Great performance from Razr Ultra
  • Useful external display
  • Big displays in a pocket-friendly package

The bad

  • Too much AI
  • Razr Ultra is very expensive
  • Only three years of OS updates, four years of security patches
  • Cameras trail the competition

Photo of Ryan Whitwam

Ryan Whitwam is a senior technology reporter at Ars Technica, covering the ways Google, AI, and mobile technology continue to change the world. Over his 20-year career, he’s written for Android Police, ExtremeTech, Wirecutter, NY Times, and more. He has reviewed more phones than most people will ever own. You can follow him on Bluesky, where you will see photos of his dozens of mechanical keyboards.

Motorola Razr and Razr Ultra (2025) review: Cool as hell, but too much AI Read More »

incorporated-in-us:-$8.4b-money-launderer-for-chinese-speaking-crypto-scammers

Incorporated in US: $8.4B money launderer for Chinese-speaking crypto scammers


Before crackdown, this was one of the ‘Net’s biggest markets for Chinese-speaking scammers.

As the underground industry of crypto investment scams has grown into one of the world’s most lucrative forms of cybercrime, the secondary market of money launderers for those scammers has grown to match it. Amid that black market, one such Chinese-language service on the messaging platform Telegram blossomed into an all-purpose underground bazaar: It has offered not only cash-out services to scammers but also money laundering for North Korean hackers, stolen data, targeted harassment-for-hire, and even what appears to be sex trafficking. And somehow, it’s all overseen by a company legally registered in the United States.

According to new research released today by crypto-tracing firm Elliptic, a company called Xinbi Guarantee has since 2022 facilitated no less than $8.4 billion in transactions via its Telegram-based marketplace prior to Telegram’s actions in recent days to remove its accounts from the platform. Money stolen from scam victims likely represents the “vast majority” of that sum, according to Elliptic’s cofounder Tom Robinson. Yet even as the market serves Chinese-speaking scammers, it also boasts on the top of its website—in Mandarin—that it’s registered in Colorado.

“Xinbi Guarantee has served as a giant, purportedly US-incorporated illicit online marketplace for online scams that primarily offers money laundering services,” says Robinson. He adds, though, that Elliptic has also found a remarkable variety of other criminal offerings on the market: child-bearing surrogacy and egg donors, harassment services that offer to threaten or throw feces at any chosen victim, and even sex workers in their teens who are likely trafficking victims.

Xinbi Guarantee is the second such crime-friendly Chinese-language market that Robinson and his team of researchers have uncovered over the past year. Last July, they published a report on Huione Guarantee, a similar Cambodia-based service that Elliptic said in January had facilitated $24 billion in transactions—largely from crypto scammers—making it the biggest illicit online marketplace in history by Elliptic’s accounting. That market’s parent company, Huione Group, was added to a list of known money laundering operations by the US Treasury’s Financial Crimes Enforcement Network earlier this month in an attempt to limit its access to US financial institutions.

Telegram bans

After WIRED reached out to Telegram last week about the illicit activity taking place on Xinbi Guarantee’s and Huione Guarantee’s channels on its messaging platform, Telegram appears to have responded Monday by banning many of the central channels and administrator accounts used by both Xinbi Guarantee and Huione Guarantee. “Criminal activities like scamming or money laundering are forbidden by Telegram’s terms of service and are always removed whenever discovered,” Telegram spokesperson Remi Vaughn wrote to WIRED in a statement. “Communities previously reported to us by WIRED or included in reports published by Elliptic have all been taken down.”

Telegram had banned several of Huione Guarantee’s channels in February following an earlier Elliptic report on the marketplace, but Huione Guarantee quickly re-created them, and it’s not clear whether the new removals will prevent the two companies from rebuilding their presence on Telegram again, perhaps with new accounts or even new branding. “These are very lucrative businesses, and they’ll attempt to rebuild in some way,” Robinson said of the two marketplaces following Telegram’s latest purge.

Elliptic’s accounting of the total lifetime revenue of the biggest online black markets.Courtesy of Elliptic

Xinbi Guarantee didn’t respond to multiple requests for comment on Elliptic’s findings that WIRED sent to the market’s administrators on Telegram.

Like Huione Guarantee, Xinbi Guarantee has offered a similar “guarantee” model of enabling third-party vendors to offer services by requiring a deposit from them to prevent fraud. Yet it’s flown under the radar, even as it grew into one of the biggest hubs for crypto crime on the Internet. In terms of scale of transactions prior to Telegram’s crackdown, it was second only to Huione’s market, according to Elliptic.

Both services “offer a window into the China-based underground banking network,” Robinson says. “It’s another example of these huge Chinese-language ‘guaranteed’ marketplaces that have thrived for years.”

On Xinbi Guarantee, Elliptic found numerous posts from vendors offering to accept funds related to “quick kills,” “slow kills,” and “pig butchering” transactions, all different terms for crypto investment scams and other forms of fraud. In some cases, Robinson explains, these Xinbi Guarantee vendors offer bank accounts in the same country as the victim so that they can receive whatever payment they’re tricked into making, then pay the scammer in the cryptocurrency Tether. In other cases, the Xinbi Guarantee merchants offer to receive cryptocurrency payments and cash them out in the scammer’s local currency, such as Chinese renminbi.

Not just money laundering

Aside from Xinbi Guarantee’s central use as a cash-out point for crypto scammers, Elliptic also found that the market’s vendors offered other wares for scammers such as stolen data that could be used for finding victims, as well as services for registering SIM cards and Starlink Internet subscriptions through proxies.

North Korean state-sponsored cybercriminals also appear to have used the platform for money laundering. Elliptic found through blockchain analysis, for instance, that about $220,000 stolen from the Indian cryptocurrency exchange WazirX—the victim of a $235 million theft in July 2024, widely attributed to North Korean hackers—had flowed into Xinbi Guarantee in a series of transactions in November.

Those money-laundering and scam-enabling services, however, are far from the only shady offerings found on Xinbi Guarantee’s market. Elliptic also found listings for surrogate mothers and egg donors, with one post showing faceless pictures of the donor’s body. Other accounts have offered services that will, for a payment in Tether, place a funeral wreath at a target’s door, deface their home with graffiti, post damaging statements around their home, have someone verbally threaten them, throw feces at them, or even, most bizarrely, surround their home with AIDS patients. One posting suggested these AIDS patients would carry “case reports and needles for intimidation.”

Other listings have offered sex workers as young as 18 years old, noting the specific sex acts that are allowed and forbidden. Elliptic says that one of its researchers was even offered a 14-year-old by a Xinbi Guarantee merchant. (The account holder noted, however, that no transaction for sex with someone below the age of 18 would be guaranteed by Xinbi. The legal age of consent in China is 14.)

Exactly why Xinbi Guarantee is legally registered in the US remains a mystery. Its incorporation record on the Colorado Secretary of State’s website shows an address at an office park in the city of Aurora that has no external Xinbi branding. The company appears to have been registered there in August of 2022 by someone named “Mohd Shahrulnizam Bin Abd Manap.” (WIRED connected that name with several people in Malaysia but couldn’t determine which one might be Xinbi Guarantee’s registrant.) The listing is currently marked as “delinquent,” perhaps due to failure to file more recent paperwork to renew it.

For fledgling Chinese companies—legitimate and illegitimate—incorporating in the US is an increasingly common tactic for “projecting legitimacy,” says Jacob Sims, a visiting fellow at Harvard’s Asia Center who focuses on transnational Chinese crime. “If you have a US presence, you can also open US bank accounts,” Sims says. “You could potentially hire staff in the US. You could in theory have more formalized connections to US entities.” But he notes that the registration’s delinquent status may mean Xinbi Guarantee tried to make some sort of inroads in the US in the past but gave up.

While Telegram has served as the chief means of communication for the two markets, the stablecoin cryptocurrency Tether has served as their primary means of payment, Elliptic found. And despite Telegram’s new round of removals of their channels and accounts, Xinbi Guarantee and Huione Guarantee are far from the only companies to use Tether and Telegram to create essentially a new, largely Chinese-language darknet: Elliptic is tracking close to 30 similar marketplaces, Robinson says, though he declined to name others in the midst of the company’s investigations.

Just as Telegram shows new signs of cracking down on that sprawling black market, Tether, too, has the ability to disrupt criminal use of its services. Unlike other more decentralized cryptocurrencies such as Bitcoin, Tether can freeze payments when it identifies bad actors. Yet it’s not clear to what degree Tether has taken measures to stop Chinese-language crypto scammers and others on Xinbi Guarantee and Huione Guarantee from using its currency.

When WIRED wrote to Tether to ask about its role in those black markets, the company responded in a statement that it encourages “firms like Elliptic and other blockchain intelligence providers to share critical data with law enforcement so we can act swiftly and in coordination.”

“We are not passive observers—we are active players in the global fight against financial crime,” the Tether statement continued. “If you’re considering using Tether for illicit purposes, think again: it is the most traceable asset in existence. We will identify you, and we will work to ensure you are brought to justice.”

Despite that promise—and Telegram’s new effort to remove Huione Guarantee and Xinbi Guarantee from its platform—both tools have already been used to facilitate tens of billions of dollars in theft and other black market deals, much of it occurring in plain sight. The two largely illegal and very public markets have been “remarkable for both the scale at which they’re operating and also the brazenness,” says Harvard’s Jacob Sims.

Given that brazenness and the massive criminal fortunes at stake, expect both markets to attempt a revival in some form—and plenty of competitors to try to take their place atop the Chinese-language crypto crime economy.

This story originally appeared on wired.com.

Photo of WIRED

Wired.com is your essential daily guide to what’s next, delivering the most original and complete take you’ll find anywhere on innovation’s impact on technology, science, business and culture.

Incorporated in US: $8.4B money launderer for Chinese-speaking crypto scammers Read More »

welcome-to-the-age-of-paranoia-as-deepfakes-and-scams-abound

Welcome to the age of paranoia as deepfakes and scams abound


AI-driven fraud is leading people to verify every online interaction they have.

These days, when Nicole Yelland receives a meeting request from someone she doesn’t already know, she conducts a multistep background check before deciding whether to accept. Yelland, who works in public relations for a Detroit-based nonprofit, says she’ll run the person’s information through Spokeo, a personal data aggregator that she pays a monthly subscription fee to use. If the contact claims to speak Spanish, Yelland says, she will casually test their ability to understand and translate trickier phrases. If something doesn’t quite seem right, she’ll ask the person to join a Microsoft Teams call—with their camera on.

If Yelland sounds paranoid, that’s because she is. In January, before she started her current nonprofit role, Yelland says, she got roped into an elaborate scam targeting job seekers. “Now, I do the whole verification rigamarole any time someone reaches out to me,” she tells WIRED.

Digital imposter scams aren’t new; messaging platforms, social media sites, and dating apps have long been rife with fakery. In a time when remote work and distributed teams have become commonplace, professional communications channels are no longer safe, either. The same artificial intelligence tools that tech companies promise will boost worker productivity are also making it easier for criminals and fraudsters to construct fake personas in seconds.

On LinkedIn, it can be hard to distinguish a slightly touched-up headshot of a real person from a too-polished, AI-generated facsimile. Deepfake videos are getting so good that longtime email scammers are pivoting to impersonating people on live video calls. According to the US Federal Trade Commission, reports of job and employment related scams nearly tripled from 2020 to 2024, and actual losses from those scams have increased from $90 million to $500 million.

Yelland says the scammers that approached her back in January were impersonating a real company, one with a legitimate product. The “hiring manager” she corresponded with over email also seemed legit, even sharing a slide deck outlining the responsibilities of the role they were advertising. But during the first video interview, Yelland says, the scammers refused to turn their cameras on during a Microsoft Teams meeting and made unusual requests for detailed personal information, including her driver’s license number. Realizing she’d been duped, Yelland slammed her laptop shut.

These kinds of schemes have become so widespread that AI startups have emerged promising to detect other AI-enabled deepfakes, including GetReal Labs and Reality Defender. OpenAI CEO Sam Altman also runs an identity-verification startup called Tools for Humanity, which makes eye-scanning devices that capture a person’s biometric data, create a unique identifier for their identity, and store that information on the blockchain. The whole idea behind it is proving “personhood,” or that someone is a real human. (Lots of people working on blockchain technology say that blockchain is the solution for identity verification.)

But some corporate professionals are turning instead to old-fashioned social engineering techniques to verify every fishy-seeming interaction they have. Welcome to the Age of Paranoia, when someone might ask you to send them an email while you’re mid-conversation on the phone, slide into your Instagram DMs to ensure the LinkedIn message you sent was really from you, or request you text a selfie with a time stamp, proving you are who you claim to be. Some colleagues say they even share code words with each other, so they have a way to ensure they’re not being misled if an encounter feels off.

“What’s funny is, the lo-fi approach works,” says Daniel Goldman, a blockchain software engineer and former startup founder. Goldman says he began changing his own behavior after he heard a prominent figure in the crypto world had been convincingly deepfaked on a video call. “It put the fear of god in me,” he says. Afterward, he warned his family and friends that even if they hear what they believe is his voice or see him on a video call asking for something concrete—like money or an Internet password—they should hang up and email him first before doing anything.

Ken Schumacher, founder of the recruitment verification service Ropes, says he’s worked with hiring managers who ask job candidates rapid-fire questions about the city where they claim to live on their résumé, such as their favorite coffee shops and places to hang out. If the applicant is actually based in that geographic region, Schumacher says, they should be able to respond quickly with accurate details.

Another verification tactic some people use, Schumacher says, is what he calls the “phone camera trick.” If someone suspects the person they’re talking to over video chat is being deceitful, they can ask them to hold up their phone camera to show their laptop. The idea is to verify whether the individual may be running deepfake technology on their computer, obscuring their true identity or surroundings. But it’s safe to say this approach can also be off-putting: Honest job candidates may be hesitant to show off the inside of their homes or offices, or worry a hiring manager is trying to learn details about their personal lives.

“Everyone is on edge and wary of each other now,” Schumacher says.

While turning yourself into a human captcha may be a fairly effective approach to operational security, even the most paranoid admit these checks create an atmosphere of distrust before two parties have even had the chance to really connect. They can also be a huge time suck. “I feel like something’s gotta give,” Yelland says. “I’m wasting so much time at work just trying to figure out if people are real.”

Jessica Eise, an assistant professor studying climate change and social behavior at Indiana University Bloomington, says her research team has been forced to essentially become digital forensics experts due to the amount of fraudsters who respond to ads for paid virtual surveys. (Scammers aren’t as interested in the unpaid surveys, unsurprisingly.) For one of her research projects, which is federally funded, all of the online participants have to be over the age of 18 and living in the US.

“My team would check time stamps for when participants answered emails, and if the timing was suspicious, we could guess they might be in a different time zone,” Eise says. “Then we’d look for other clues we came to recognize, like certain formats of email address or incoherent demographic data.”

Eise says the amount of time her team spent screening people was “exorbitant” and that they’ve now shrunk the size of the cohort for each study and have turned to “snowball sampling,” or recruiting people they know personally to join their studies. The researchers are also handing out more physical flyers to solicit participants in person. “We care a lot about making sure that our data has integrity, that we’re studying who we say we’re trying to study,” she says. “I don’t think there’s an easy solution to this.”

Barring any widespread technical solution, a little common sense can go a long way in spotting bad actors. Yelland shared with me the slide deck that she received as part of the fake job pitch. At first glance, it seemed legit, but when she looked at it again, a few details stood out. The job promised to pay substantially more than the average salary for a similar role in her location and offered unlimited vacation time, generous paid parental leave, and fully covered health care benefits. In today’s job environment, that might have been the biggest tipoff of all that it was a scam.

This story originally appeared on wired.com.

Photo of WIRED

Wired.com is your essential daily guide to what’s next, delivering the most original and complete take you’ll find anywhere on innovation’s impact on technology, science, business and culture.

Welcome to the age of paranoia as deepfakes and scams abound Read More »

new-lego-building-ai-creates-models-that-actually-stand-up-in-real-life

New Lego-building AI creates models that actually stand up in real life

The LegoGPT system works in three parts, shown in this diagram.

The LegoGPT system works in three parts, shown in this diagram. Credit: Pun et al.

The researchers also expanded the system’s abilities by adding texture and color options. For example, using an appearance prompt like “Electric guitar in metallic purple,” LegoGPT can generate a guitar model, with bricks assigned a purple color.

Testing with robots and humans

To prove their designs worked in real life, the researchers had robots assemble the AI-created Lego models. They used a dual-robot arm system with force sensors to pick up and place bricks according to the AI-generated instructions.

Human testers also built some of the designs by hand, showing that the AI creates genuinely buildable models. “Our experiments show that LegoGPT produces stable, diverse, and aesthetically pleasing Lego designs that align closely with the input text prompts,” the team noted in its paper.

When tested against other AI systems for 3D creation, LegoGPT stands out through its focus on structural integrity. The team tested against several alternatives, including LLaMA-Mesh and other 3D generation models, and found its approach produced the highest percentage of stable structures.

A video of two robot arms building a LegoGPT creation, provided by the researchers.

Still, there are some limitations. The current version of LegoGPT only works within a 20×20×20 building space and uses a mere eight standard brick types. “Our method currently supports a fixed set of commonly used Lego bricks,” the team acknowledged. “In future work, we plan to expand the brick library to include a broader range of dimensions and brick types, such as slopes and tiles.”

The researchers also hope to scale up their training dataset to include more objects than the 21 categories currently available. Meanwhile, others can literally build on their work—the researchers released their dataset, code, and models on their project website and GitHub.

New Lego-building AI creates models that actually stand up in real life Read More »

linux-kernel-is-leaving-486-cpus-behind,-only-18-years-after-the-last-one-made

Linux kernel is leaving 486 CPUs behind, only 18 years after the last one made

It’s not the first time Torvalds has suggested dropping support for 32-bit processors and relieving kernel developers from implementing archaic emulation and work-around solutions. “We got rid of i386 support back in 2012. Maybe it’s time to get rid of i486 support in 2022,” Torvalds wrote in October 2022. Failing major changes to the 6.15 kernel, which will likely arrive late this month, i486 support will be dropped.

Where does that leave people running a 486 system for whatever reason? They can run older versions of the Linux kernel and Linux distributions. They might find recommendations for teensy distros like MenuetOS, KolibriOS, and Visopsys, but all three of those require at least a Pentium. They can run FreeDOS. They might get away with the OS/2 descendant ArcaOS. There are some who have modified Windows XP to run on 486 processors, and hopefully, they will not connect those devices to the Internet.

Really, though, if you’re dedicated enough to running a 486 system in 2025, you’re probably resourceful enough to find copies of the software meant for that system. One thing about computers—you never stop learning.

This post was updated at 3: 30 p.m. to fix a date error.

Linux kernel is leaving 486 CPUs behind, only 18 years after the last one made Read More »

fidji-simo-joins-openai-as-new-ceo-of-applications

Fidji Simo joins OpenAI as new CEO of Applications

In the message, Altman described Simo as bringing “a rare blend of leadership, product and operational expertise” and expressed that her addition to the team makes him “even more optimistic about our future as we continue advancing toward becoming the superintelligence company.”

Simo becomes the newest high-profile female executive at OpenAI following the departure of Chief Technology Officer Mira Murati in September. Murati, who had been with the company since 2018 and helped launch ChatGPT, left alongside two other senior leaders and founded Thinking Machines Lab in February.

OpenAI’s evolving structure

The leadership addition comes as OpenAI continues to evolve beyond its origins as a research lab. In his announcement, Altman described how the company now operates in three distinct areas: as a research lab focused on artificial general intelligence (AGI), as a “global product company serving hundreds of millions of users,” and as an “infrastructure company” building systems that advance research and deliver AI tools “at unprecedented scale.”

Altman mentioned that as CEO of OpenAI, he will “continue to directly oversee success across all pillars,” including Research, Compute, and Applications, while staying “closely involved with key company decisions.”

The announcement follows recent news that OpenAI abandoned its original plan to cede control of its nonprofit branch to a for-profit entity. The company began as a nonprofit research lab in 2015 before creating a for-profit subsidiary in 2019, maintaining its original mission “to ensure artificial general intelligence benefits everyone.”

Fidji Simo joins OpenAI as new CEO of Applications Read More »

cheaters-gonna-cheat-cheat-cheat-cheat-cheat

Cheaters Gonna Cheat Cheat Cheat Cheat Cheat

Cheaters. Kids these days, everyone says, are all a bunch of blatant cheaters via AI.

Then again, look at the game we are forcing them to play, and how we grade it.

If you earn your degree largely via AI, that changes two distinct things.

  1. You might learn different things.

  2. You might signal different things.

Both learning and signaling are under threat if there is too much blatant cheating.

There is too much cheating going on, too blatantly.

Why is that happening? Because the students are choosing to do it.

Ultimately, this is a preview of what will happen everywhere else as well. It is not a coincidence that AI starts its replacement of work in the places where the work is the most repetitive, useless and fake, but its ubiquitousness will not stay confined there. These are problems and also opportunities we will face everywhere. The good news is that in other places the resulting superior outputs will actually produce value.

  1. You Could Take The White Pill, But You Probably Won’t.

  2. Is Our Children Learning.

  3. Cheaters Never Stop Cheating.

  4. If You Know You Know.

  5. The Real Victims Here.

  6. Taking Note.

  7. What You Going To Do About It, Punk?

  8. How Bad Are Things?

  9. The Road to Recovery.

  10. The Whispering Earring.

As I always say, if you have access to AI, you can use it to (A) learn and grow strong and work better, or (B) you can use it to avoid learning, growing and working. Or you can always (C) refuse to use it at all, or perhaps (D) use it in strictly limited capacities that you choose deliberately to save time but avoid the ability to avoid learning.

Choosing (A) and using AI to learn better and smarter is strictly better than choosing (C) and refusing to use AI at all.

If you choose (B) and use AI to avoid learning, you might be better or worse off than choosing (C) and refusing to use AI at all, depending on the value of the learning you are avoiding.

If the learning in question is sufficiently worthless, there’s no reason to invest in it, and (B) is not only better than (C) but also better than (A).

Tim Sweeney: The question is not “is it cheating”, the question is “is it learning”.

James Walsh: AI has made Daniel more curious; he likes that whenever he has a question, he can quickly access a thorough answer. But when he uses AI for homework, he often wonders, If I took the time to learn that, instead of just finding it out, would I have learned a lot more?

I notice I am confused. What is the difference between ‘learning that’ and ‘just finding it out’? And what’s to stop Daniel from walking through the a derivation or explanation with the AI if he wants to do that? I’ve done that a bunch with ML, and it’s great. o3’s example here was being told and memorizing the integral of sin x is -cos x rather than deriving it, but that was what most students always did anyway.

The path you take is up to you.

Ted Chiang: Using ChatGPT to complete tasks is like taking a forklift to the gym: you’ll never improve your cognitive abilities that way.”

Ewan Morrison: AI is demoralising universities. Students who use AI, think “why bother to study or write when AI can do it for me?” Tutors who mark the essays, think “why bother to teach these students & why give a serious grade when 90% of essays are done with AI?”

I would instead ask, why are you assigning essays the AI can do for them, without convincing the students why they should still write the essays themselves?

The problem, as I understand it, is that in general students are more often than not:

  1. Not that interested in learning.

  2. Do not think that their assignments are a good way to learn.

  3. Quite interested in not working.

  4. Quite interested getting good grades.

  5. Know how to use ChatGPT to avoid learning.

  6. Do not know how to use ChatGPT to learn, or it doesn’t even occur to them.

  7. Aware that if they did use ChatGPT to learn, it wouldn’t be via schoolwork.

Meatball Times: has anyone stopped to ask WHY students cheat? would a buddhist monk “cheat” at meditation? would an artist “cheat” at painting? no. when process and outcomes are aligned, there’s no incentive to cheat. so what’s happening differently at colleges? the answer is in the article.

Colin Fraser (being right): “would an artist ‘cheat’ at a painting?”

I mean… yes, famously.

Now that the cost of such cheating is close to zero I expect that we will be seeing a lot more of it!

James Walsh: Although Columbia’s policy on AI is similar to that of many other universities’ — students are prohibited from using it unless their professor explicitly permits them to do so, either on a class-by-class or case-by-case basis — Lee said he doesn’t know a single student at the school who isn’t using AI to cheat. To be clear, Lee doesn’t think this is a bad thing.

If the reward for painting is largely money, which it is, then clearly if you give artists the ability to cheat then many of them will cheat, as in things like forgery, as they often have in the past. The way to stop them is to catch the ones who try.

The reason the Buddhist monk presumably wouldn’t ‘cheat’ at meditation is because they are not trying to Be Observed Performing Meditation, they want to meditate. But yes, if they were getting other rewards for meditation, I’d expect some cheating, sure, even if the meditation also had intrinsic rewards.

Back to the school question. If the students did know how to use AI to learn, why would they need the school, or to do the assignments?

The entire structure of school is based on the thesis that students need to be forced to learn, and that this learning must be constantly policed.

The thesis has real validity. At this point, with not only AI but also YouTube and plenty of other free online materials, the primary educational (non-social, non-signaling) product is that the class schedule and physical presence, and exams and assignments, serve as a forcing function to get you to do the damn work and pay attention, even if inefficiently.

Zito (quoting the NYMag article): The kids are cooked.

Yishan: One of my kids buys into the propaganda that AI is environmentally harmful (not helped by what xAI is doing in Memphis, btw), and so refuses to use AI for any help on learning tough subjects. The kid just does the work, grinding it out, and they are getting straight A’s.

And… now I’m thinking maybe I’ll stop trying to convince the kid otherwise.

It’s entirely not obvious whether it would be a good idea to convince the kid otherwise. Using AI is going to be the most important skill, and it can make the learning much better, but maybe it’s fine to let the kid wait given the downside risks of preventing that?

The reason taking such a drastic (in)action might make sense is that the kids know the assignments are stupid and fake. The whole thesis of commitment devices that lead to forced work is based on the idea that the kids (or their parents) understand that they do need to be forced to work, so they need this commitment device, and also that the commitment device is functional.

Now both of those halves are broken. The commitment devices don’t work, you can simply cheat. And the students are in part trying to be lazy, sure, but they’re also very consciously not seeing any value here. Lee here is not typical in that he goes on to actively create a cheating startup but I mean, hey, was he wrong?

James Walsh: “Most assignments in college are not relevant,” [Columbia student Lee] told me. “They’re hackable by AI, and I just had no interest in doing them.”

While other new students fretted over the university’s rigorous core curriculum, described by the school as “intellectually expansive” and “personally transformative,” Lee used AI to breeze through with minimal effort.

When I asked him why he had gone through so much trouble to get to an Ivy League university only to off-load all of the learning to a robot, he said, “It’s the best place to meet your co-founder and your wife.”

Bingo. Lee knew this is no way to learn. That’s not why he was there.

Columbia can call its core curriculum ‘intellectually expansive’ and ‘personally transformative’ all it wants. That doesn’t make it true, and it definitely isn’t fooling that many of the students.

The key fact about cheaters is that they not only never stop cheating on their own. They escalate the extent of their cheating until they are caught. Once you pop enough times, you can’t stop. Cheaters learn to cheat as a habit, not as the result of an expected value calculation in each situation.

For example, if you put a Magic: the Gathering cheater onto a Twitch stream, where they will leave video evidence of their cheating, will they stop? No, usually not.

Thus, you can literally be teaching ‘Ethics and AI’ and ask for a personal reflection, essentially writing a new line of Ironic, and they will absolutely get it from ChatGPT.

James Walsh: Less than three months later, teaching a course called Ethics and Artificial Intelligence, [Brian Patrick Green] figured a low-stakes reading reflection would be safe — surely no one would dare use ChatGPT to write something personal. But one of his students turned in a reflection with robotic language and awkward phrasing that Green knew was AI-generated.

This is a way to know students are indeed cheating rather than using AI to learn. The good news? Teachable moment.

Lee in particular clearly doesn’t have a moral compass in any of this. He doesn’t get the idea that cheating can be wrong even in theory:

For now, Lee hopes people will use Cluely to continue AI’s siege on education. “We’re going to target the digital LSATs; digital GREs; all campus assignments, quizzes, and tests,” he said. “It will enable you to cheat on pretty much everything.”

If you’re enabling widespread cheating on the LSATs and GREs, you’re no longer a morally ambiguous rebel against the system. Now you’re just a villain.

Or you can have a code:

James Walsh: Wendy, a freshman finance major at one of the city’s top universities, told me that she is against using AI. Or, she clarified, “I’m against copy-and-pasting. I’m against cheating and plagiarism. All of that. It’s against the student handbook.”

Then she described, step-by-step, how on a recent Friday at 8 a.m., she called up an AI platform to help her write a four-to-five-page essay due two hours later.

Wendy will use AI for ‘all aid short of copy-pasting,’ the same way you would use Google or Wikipedia or you’d ask a friend questions, but she won’t copy-and-paste. The article goes on to describe her full technique. AI can generate an outline, and brainstorm ideas and arguments, so long as the words are hers.

That’s not an obviously wrong place to draw the line. It depends on which part of the assignment is the active ingredient. Is Wendy supposed to be learning:

  1. How to structure, outline and manufacture a school essay in particular?

  2. How to figure out what a teacher wants her to do?

  3. ‘How to write’?

  4. How to pick a ‘thesis’?

  5. How to find arguments and bullet points?

  6. The actual content of the essay?

  7. An assessment of how good she is rather than grademaxxing?

Wendy says planning the essay is fun, but ‘she’d rather get good grades.’ As in, the system actively punishes her for trying to think about such questions rather than being the correct form of fake. She is still presumably learning about the actual content of the essay, and by producing it, if there’s any actual value to the assignment, and she pays attention, she’ll pick up the reasons why the AI makes the essay the way it does.

I don’t buy that this is going to destroy Wendy’s ‘critical thinking’ skills. Why are we teaching her that school essay structures and such are the way to train critical thinking? Everything in my school experience says the opposite.

The ‘cheaters’ who only cheat or lie a limited amount and then stop have a clear and coherent model of why what they are doing in the contexts they cheat or lie in is not cheating or why it is acceptable or justified, and this is contrasted with other contexts. Why some rules are valid, and others are not. Even then, it usually takes a far stronger person to hold that line than to not cheat in the first place.

Another way to look at this is, if it’s obvious from the vibes that you cheated, you cheated, even if the system can’t prove it. The level of obviousness varies, you can’t always sneak in smoking gun instructions.

But if you invoke the good Lord Bayes, you know.

James Walsh: Most of the writing professors I spoke to told me that it’s abundantly clear when their students use AI.

Not that they flag it.

Still, while professors may think they are good at detecting AI-generated writing, studies have found they’re actually not. One, published in June 2024, used fake student profiles to slip 100 percent AI-generated work into professors’ grading piles at a U.K. university. The professors failed to flag 97 percent.

But there’s a huge difference between ‘I flag this as AI and am willing to fight over this’ and knowing that something was probably or almost certainly AI.

What about automatic AI detectors? They’re detecting something. It’s noisy, and it’s different, it’s not that hard to largely fool if you care, and it has huge issues (especially for ESL students) but I don’t think either of these responses is an error?

I fed Wendy’s essay through a free AI detector, ZeroGPT, and it came back as 11.74 AI-generated, which seemed low given that AI, at the very least, had generated her central arguments. I then fed a chunk of text from the Book of Genesis into ZeroGPT and it came back as 93.33 percent AI-generated.

If you’re direct block quoting Genesis without attribution, your essay is plagiarized. Maybe it came out of the AI and maybe it didn’t, but it easily could have, it knows Genesis and it’s allowed to quote from it. So 93% seems fine. Whereas Wendy’s essay is written by Wendy, the AI was used to make it conform to the dumb structures and passwords of the course. 11% seems fine.

Colin Fraser: I think we’ve somehow swung to overestimating the number of kids who are cheating with ChatGPT and simultaneously underestimating the amount of grief and hassle this creates for educators.

The guy making the cheating app wants you to think every single other person out there is cheating at everything and you’re falling behind if you’re not cheating. That’s not true. But the spectre a few more plagiarized assignments per term is massively disruptive for teachers.

James Walsh: Many teachers now seem to be in a state of despair.

I’m sorry, what?

Given how estimations work, I can totally believe we might be overestimating the number of kids who are cheating. Of course, the number is constantly rising, especially for the broader definitions of ‘cheating,’ so even if you were overestimating at the time you might not be anymore.

But no, this is not about ‘a few more plagiarized assignments per term,’ both because this isn’t plagiarism it’s a distinct other thing, and also because by all reports it’s not only a few cases, it’s an avalanche even if underestimated.

Doing the assignments yourself is now optional unless you force the student to do it in front of you. Deal with it.

As for this being ‘grief and hassle’ for educators, yes, I am sure it is annoying when your system of forced fake work can be faked back at you more effectively and more often, and when there is a much better source of information and explanations available than you and your textbooks such that very little of what you are doing really has a point to it anymore.

If you think students have to do certain things themselves in order to learn, then as I see it you have two options, you can do either or both.

  1. Use frequent in-person testing, both as the basis of grades and as a forcing function so that students learn. This is a time honored technique.

  2. Use in-person assignments and tasks, so you can prevent AI use. This is super annoying but it has other advantages.

Alternatively or in addition to this, you can embrace AI and design new tasks and assignments that cause students to learn together with the AI. That’s The Way.

Trying to ‘catch’ the ‘cheating’ is pointless. It won’t work. Trying only turns this at best into a battle over obscuring tool use and makes the whole experience adversarial.

If you assign fake essay forms to students, and then grade them on those essays and use those grades to determine their futures, what the hell do you think is going to happen? This form of essay assignment is no longer valid, and if you assign it anyway you deserve what you get.

James Walsh: “I think we are years — or months, probably — away from a world where nobody thinks using AI for homework is considered cheating,” [Lee] said.

I think that is wrong. We are a long way away from the last people giving up this ghost. But seriously it is pretty insane to think ‘using AI for homework’ is cheating. I’m actively trying to get my kids to use AI for homework more, not less.

James Walsh: In January 2023, just two months after OpenAI launched ChatGPT, a survey of 1,000 college students found that nearly 90 percent of them had used the chatbot to help with homework assignments.

What percentage of that 90% was ‘cheating’? We don’t know, and definitions differ, but I presume a lot less than all of them.

Now and also going forward, I think you could say that particular specific uses are indeed really cheating, and it depends how you use it. But if you think ‘use AI to ask questions about the world and learn the answer’ is ‘cheating’ then explain what the point of the assignment was, again?

The whole enterprise is broken, and will be broken while there is a fundamental disconnect between what is measured and what they want to be managing.

James Walsh: Williams knew most of the students in this general-education class were not destined to be writers, but he thought the work of getting from a blank page to a few semi-coherent pages was, above all else, a lesson in effort. In that sense, most of his students utterly failed.

[Jollimore] worries about the long-term consequences of passively allowing 18-year-olds to decide whether to actively engage with their assignments.

The entire article makes clear that students almost never buy that their efforts would be worthwhile. A teacher can think ‘this will teach them effort’ but if that’s the goal then why not go get an actual job? No one is buying this, so if the grades don’t reward effort, why should there be effort?

How dare you let 18-year-olds decide whether to engage with their assignments that produce no value to anyone but themselves.

This is all flat out text.

The ideal of college as a place of intellectual growth, where students engage with deep, profound ideas, was gone long before ChatGPT.

In a way, the speed and ease with which AI proved itself able to do college-level work simply exposed the rot at the core.

There’s no point. Was there ever a point?

“The students kind of recognize that the system is broken and that there’s not really a point in doing this. Maybe the original meaning of these assignments has been lost or is not being communicated to them well.”

The question is, once you know, what do you do about it? How do you align what is measured with what is to be managed? What exactly do you want from the students?

James Walsh: The “true attempt at a paper” policy ruined Williams’s grading scale. If he gave a solid paper that was obviously written with AI a B, what should he give a paper written by someone who actually wrote their own paper but submitted, in his words, “a barely literate essay”?

What is measured gets managed. You either give the better grade to the ‘barely literate’ essay, or you don’t.

My children get assigned homework. The school’s literal justification – I am not making this up, I am not paraphrasing – is that they need to learn to do homework so that they will be prepared to do more homework in the future. Often this involves giving them assignments that we have to walk them through because there is no reasonable way for them to understand what is being asked.

If it were up to me, damn right I’d have them use AI.

It’s not just the students: Multiple AI platforms now offer tools to leave AI-generated feedback on students’ essays. Which raises the possibility that AIs are now evaluating AI-generated papers, reducing the entire academic exercise to a conversation between two robots — or maybe even just one.

Great! Now we can learn.

Another AI application to university is note taking. AI can do excellent transcription and rather strong active note taking. Is that a case of learning, or of not learning? There are competing theories, which I think are true for different people at different times.

  1. One theory says that the act of taking notes is how you learn, by forcing you to pay attention, distill the information and write it in your own words.

  2. The other theory is that having to take notes prevents you from actually paying ‘real’ attention and thinking and engaging, you’re too busy writing down factual information.

AI also means that even if you don’t have it take notes or a transcript, you don’t have to worry as much about missing facts, because you can ask the AI for them later.

My experience is that having to take notes is mostly a negative. Every time I focus on writing something down that means I’m not listening, or not fully listening, and definitely not truly thinking.

Rarely did she sit in class and not see other students’ laptops open to ChatGPT.

Of course your laptop is open to an AI. It’s like being able to ask the professor any questions you like without interrupting the class or paying any social costs, including stupid questions. If there’s a college lecture, and at no point do you want to ask Gemini, Claude or o3 any questions, what are you even doing? That also means everyone gets to learn much better, removing the tradeoff of each question disrupting the rest of the class.

Similarly, devising study materials and practice tests seems clearly good.

The most amazing thing about the AI ‘cheating’ epidemic at universities is the extent to which the universities are content to go quietly into the night. They are mostly content to let nature take its course.

Could the universities adapt to the new reality? Yes, but they choose not to.

Cat Zhang: more depressing than Trump’s funding slashes and legal assaults and the Chat-GPT epidemic is witnessing how many smart, competent people would rather give up than even begin to think of what we could do about it

Tyler Austin Harper: It can’t be emphasized enough: wide swaths of the academy have given up re ChatGPT. Colleges have had since 2022 to figure something out and have done less than nothing. Haven’t even tried. Or tried to try. The administrative class has mostly collaborated with the LLM takeover.

Hardly anyone in this country believes in higher ed, especially the institutions themselves which cannot be mustered to do anything in their own defense. Faced with an existential threat, they can’t be bothered to cry, yawn, or even bury their head in the sand, let alone resist.

It would actually be more respectable if they were in denial, but the pervading sentiment is “well, we had a good run.” They don’t even have the dignity of being delusional. It’s shocking. Three years in and how many universities can you point to that have tried anything really?

If the AI crisis points to anything it’s that higher ed has been dead a long time, before ChatGPT was twinkle in Sam Altman’s eye. The reason the universities can’t be roused to their own defense is that they’re being asked to defend a corpse and the people who run them know it.

They will return to being finishing schools once again.

To paraphrase Alan Moore, this is one of those moments where colleges need to look at what’s on the table and (metaphorically) say: “Thank you, but I’d rather die behind the chemical sheds.” Instead, we get an OpenAI and Cal State partnership. Total, unapologetic capitulation.

The obvious interpretation is that college had long shifted into primarily being a Bryan Caplan style set of signaling mechanisms, so the universities are not moving to defend themselves against students who seek to avoid learning.

The problem is, this also destroys key portions of the underlying signals.

Greg Lukainoff: [Tyler’s statement above is] powerful evidence of the signaling hypothesis, that essentially the primary function of education is to signal to future employers that you were probably pretty smart and conscientious to get into college in the first place, and pretty, as @bryan_caplan puts it, “conservative” in a (non-political sense) to be able to finish it. Therefore graduates may be potentially competent and compliant employees.

Seems like there are far less expensive ways to convey that information.

Clark H: The problem is the signal is now largely false. It takes much less effort to graduate from college now – just crudely ask GPT to do it. There is even a case to be made that, like a prison teaches how to crime, college now teaches how to cheat.

v8pAfNs82P1foT: There’s a third signal of value to future employers: conformity to convention/expectation. There are alternative credible pathways to demonstrate intelligence and sustained diligence. But definitionally, the only way to credibly signal willingness to conform is to conform.

Megan McArdle: The larger problem is that a degree obtained by AI does not signal the information they are trying to convey, so its value is likely to collapse quickly as employers get wise. There will be a lag, because cultural habits die hard, but eventually the whole enterprise will implode unless they figure out how to teach something that employers will pay a premium for.

Matthew Yglesias: I think this is all kind of missing the boat, the same AI that can pass your college classes for you is radically devaluing the skills that a college degree (whether viewed as real learning or just signaling or more plausibly a mix) used to convey in the market.

The AI challenge for higher education isn’t that it’s undermining the assessment protocols (as everyone has noticed you can fix this with blue books or oral exams if you bother trying) it’s that it’s undermining the financial value of the degree!

Megan McArdle: Eh, conscientiousness is likely to remain valuable, I think. They also provide ancillary marriage market and networking services that arguably get more valuable in an age of AI.

Especially at elite schools. If you no longer have to spend your twenties and early thirties prepping for the PUMC rat race, why not get married at 22 and pop out some babies while you still have energy to chase them?

But anyway, yes, this is what I was saying, apparently not clearly enough: the problem is not just that you can’t assess certain kinds of paper-writing skills, it’s that the skills those papers were assessing will decline in value.

Periodically you see talk about how students these days (or kids these days) are in trouble. How they’re stupider, less literate, they can’t pay attention, they’re lazy and refuse to do work, and so on.

“We’re talking about an entire generation of learning perhaps significantly undermined here,” said Green, the Santa Clara tech ethicist. “It’s short-circuiting the learning process, and it’s happening fast.”

The thing is, this is a Pessimists Archive speciality, this pattern dates back at least to Socrates. People have always worried about this, and the opposite has very clearly been true overall. It’s learning, and also many other things, where ‘kids these days’ are always ‘in crisis’ and ‘falling behind’ and ‘at risk’ and so on.

My central understanding for this is that as times change, people compare kids now to kids of old both through rose-colored memory glasses, and also by checking against the exact positive attributes of the previous generations. Whereas as times change, the portfolio of skills and knowledge shifts. Today’s kids are masters at many things that didn’t even exist in my youth. That’s partly going to be a shift away from other things, most of which are both less important than the new priorities and less important than they were.

Ron Arts: Most important sentence in the article: “There might have been people complaining about machinery replacing blacksmiths in, like, the 1600s or 1800s, but now it’s just accepted that it’s useless to learn how to blacksmith.”

George Turner: Blacksmithing is an extremely useful skill. Even if I’m finishing up the part on a big CNC machine or with an industrial robot, there are times when smithing saves me a lot of time.

Bob BTC: Learning a trade is far different than learning to think!

Is it finally ‘learning to think’ this time? Really? Were they reading the sequences? Could previous students have written them?

And yes, people really will use justifications for our university classes that are about as strong as ‘blacksmithing is an extremely useful skill.’

So we should be highly suspicious of yet another claim of new tech destroying kids ability to learn, especially when it is also the greatest learning tool in human history.

Notice how much better it is to use AI than it is to hire to a human to do your homework, if both had the same cost, speed and quality profiles.

For $15.95 a month, Chegg promised answers to homework questions in as little as 30 minutes, 24/7, from the 150,000 experts with advanced degrees it employed, mostly in India. When ChatGPT launched, students were primed for a tool that was faster, more capable.

With AI, you create the prompt and figure out how to frame the assignment, you can ask follow-up questions, you are in control. With hiring a human, you are much less likely to do any of that. It matters.

Ultimately, this particular cataclysm is not one I am so worried about. I don’t think our children were learning before, and they have much better opportunity to do so now. I don’t think they were acting with or being selected for integrity at university before, either. And if this destroys the value of degrees? Mostly, I’d say: Good.

If you are addicted to TikTok, ChatGPT or your phone in general, it can get pretty grim, as was often quoted.

James Walsh: Rarely did she sit in class and not see other students’ laptops open to ChatGPT. Toward the end of the semester, she began to think she might be dependent on the website. She already considered herself addicted to TikTok, Instagram, Snapchat, and Reddit, where she writes under the username maybeimnotsmart. “I spend so much time on TikTok,” she said. “Hours and hours, until my eyes start hurting, which makes it hard to plan and do my schoolwork. With ChatGPT, I can write an essay in two hours that normally takes 12.”

The ‘catch’ that isn’t mentioned is that She Got Better.

Colin Fraser: Kind of an interesting omission. Not THAT interesting or anything but, you know, why didn’t he put that in the article?

I think it’s both interesting and important context. If your example of a student addicted to ChatGPT and her phone beat that addiction, that’s highly relevant. It’s totally within Bounded Distrust rules to not mention it, but hot damn. Also, congrats to maybeimnotsosmart.

Ultimately the question is, if you have access to increasingly functional copies of The Whispering Earring, what should you do with that? If others get access to it, what then? What do we do about educational situations ‘getting there first’?

In case you haven’t read The Whispering Earring, it’s short and you should, and I’m very confident the author won’t mind, so here’s the whole story.

Scott Alexander: Clarity didn’t work, trying mysterianism.

In the treasure-vaults of Til Iosophrang rests the Whispering Earring, buried deep beneath a heap of gold where it can do no further harm.

The earring is a little topaz tetrahedron dangling from a thin gold wire. When worn, it whispers in the wearer’s ear: “Better for you if you take me off.” If the wearer ignores the advice, it never again repeats that particular suggestion.

After that, when the wearer is making a decision the earring whispers its advice, always of the form “Better for you if you…”. The earring is always right. It does not always give the best advice possible in a situation. It will not necessarily make its wearer King, or help her solve the miseries of the world. But its advice is always better than what the wearer would have come up with on her own.

It is not a taskmaster, telling you what to do in order to achieve some foreign goal. It always tells you what will make you happiest. If it would make you happiest to succeed at your work, it will tell you how best to complete it. If it would make you happiest to do a half-assed job at your work and then go home and spend the rest of the day in bed having vague sexual fantasies, the earring will tell you to do that. The earring is never wrong.

The Book of Dark Waves gives the histories of two hundred seventy four people who previously wore the Whispering Earring. There are no recorded cases of a wearer regretting following the earring’s advice, and there are no recorded cases of a wearer not regretting disobeying the earring. The earring is always right.

The earring begins by only offering advice on major life decisions. However, as it gets to know a wearer, it becomes more gregarious, and will offer advice on everything from what time to go to sleep, to what to eat for breakfast. If you take its advice, you will find that breakfast food really hit the spot, that it was exactly what you wanted for breakfast that day even though you didn’t know it yourself. The earring is never wrong.

As it gets completely comfortable with its wearer, it begins speaking in its native language, a series of high-bandwidth hisses and clicks that correspond to individual muscle movements. At first this speech is alien and disconcerting, but by the magic of the earring it begins to make more and more sense. No longer are the earring’s commands momentous on the level of “Become a soldier”. No more are they even simple on the level of “Have bread for breakfast”. Now they are more like “Contract your biceps muscle about thirty-five percent of the way” or “Articulate the letter p”. The earring is always right. This muscle movement will no doubt be part of a supernaturally effective plan toward achieving whatever your goals at that moment may be.

Soon, reinforcement and habit-formation have done their trick. The connection between the hisses and clicks of the earring and the movements of the muscles have become instinctual, no more conscious than the reflex of jumping when someone hidden gives a loud shout behind you.

At this point no further change occurs in the behavior of the earring. The wearer lives an abnormally successful life, usually ending out as a rich and much-beloved pillar of the community with a large and happy family.

When Kadmi Rachumion came to Til Iosophrang, he took an unusual interest in the case of the earring. First, he confirmed from the records and the testimony of all living wearers that the earring’s first suggestion was always that the earring itself be removed. Second, he spent some time questioning the Priests of Beauty, who eventually admitted that when the corpses of the wearers were being prepared for burial, it was noted that their brains were curiously deformed: the neocortexes had wasted away, and the bulk of their mass was an abnormally hypertrophied mid- and lower-brain, especially the parts associated with reflexive action.

Finally, Kadmi-nomai asked the High Priest of Joy in Til Iosophrang for the earring, which he was given. After cutting a hole in his own earlobe with the tip of the Piercing Star, he donned the earring and conversed with it for two hours, asking various questions in Kalas, in Kadhamic, and in its own language. Finally he removed the artifact and recommended that the it be locked in the deepest and most inaccessible parts of the treasure vaults, a suggestion with which the Iosophrelin decided to comply.

This is very obviously not the optimal use of The Whispering Earring, let alone the ability to manufacture copies of it.

But, and our future may depend on the answer, what is your better plan? And in particular, what is your plan for when everyone has access to (a for now imperfect and scope limited but continuously improving) one, and you are at a rather severe disadvantage if you do not put one on?

The actual problem we face is far trickier than that. Both in education, and in general.

Discussion about this post

Cheaters Gonna Cheat Cheat Cheat Cheat Cheat Read More »

report:-doge-supercharges-mass-layoff-software,-renames-it-to-sound-less-dystopian

Report: DOGE supercharges mass-layoff software, renames it to sound less dystopian

“It is not clear how AutoRIF has been modified or whether AI is involved in the RIF mandate (through AutoRIF or independently),” Kunkler wrote. “However, fears of AI-driven mass-firings of federal workers are not unfounded. Elon Musk and the Trump Administration have made no secret of their affection for the dodgy technology and their intentions to use it to make budget cuts. And, in fact, they have already tried adding AI to workforce decisions.”

Automating layoffs can perpetuate bias, increase worker surveillance, and erode transparency to the point where workers don’t know why they were let go, Kunkler said. For government employees, such imperfect systems risk triggering confusion over worker rights or obscuring illegal firings.

“There is often no insight into how the tool works, what data it is being fed, or how it is weighing different data in its analysis,” Kunkler said. “The logic behind a given decision is not accessible to the worker and, in the government context, it is near impossible to know how or whether the tool is adhering to the statutory and regulatory requirements a federal employment tool would need to follow.”

The situation gets even starker when you imagine mistakes on a mass scale. Don Moynihan, a public policy professor at the University of Michigan, told Reuters that “if you automate bad assumptions into a process, then the scale of the error becomes far greater than an individual could undertake.”

“It won’t necessarily help them to make better decisions, and it won’t make those decisions more popular,” Moynihan said.

The only way to shield workers from potentially illegal firings, Kunkler suggested, is to support unions defending worker rights while pushing lawmakers to intervene. Calling on Congress to ban the use of shadowy tools relying on unknown data points to gut federal agencies “without requiring rigorous external testing and auditing, robust notices and disclosure, and human decision review,” Kunkler said rolling out DOGE’s new tool without more transparency should be widely condemned as unacceptable.

“We must protect federal workers from these harmful tools,” Kunkler said, adding, “If the government cannot or will not effectively mitigate the risks of using automated decision-making technology, it should not use it at all.”

Report: DOGE supercharges mass-layoff software, renames it to sound less dystopian Read More »

open-source-project-curl-is-sick-of-users-submitting-“ai-slop”-vulnerabilities

Open source project curl is sick of users submitting “AI slop” vulnerabilities

Ars has reached out to HackerOne for comment and will update this post if we get a response.

“More tools to strike down this behavior”

In an interview with Ars, Stenberg said he was glad his post—which generated 200 comments and nearly 400 reposts as of Wednesday morning—was getting around. “I’m super happy that the issue [is getting] attention so that possibly we can do something about it [and] educate the audience that this is the state of things,” Stenberg said. “LLMs cannot find security problems, at least not like they are being used here.”

This week has seen four such misguided, obviously AI-generated vulnerability reports seemingly seeking either reputation or bug bounty funds, Stenberg said. “One way you can tell is it’s always such a nice report. Friendly phrased, perfect English, polite, with nice bullet-points … an ordinary human never does it like that in their first writing,” he said.

Some AI reports are easier to spot than others. One accidentally pasted their prompt into the report, Stenberg said, “and he ended it with, ‘and make it sound alarming.'”

Stenberg said he had “talked to [HackerOne] before about this” and has reached out to the service this week. “I would like them to do something, something stronger, to act on this. I would like help from them to make the infrastructure around [AI tools] better and give us more tools to strike down this behavior,” he said.

In the comments of his post, Stenberg, trading comments with Tobias Heldt of open source security firm XOR, suggested that bug bounty programs could potentially use “existing networks and infrastructure.” Security reporters paying a bond to have a report reviewed “could be one way to filter signals and reduce noise,” Heldt said. Elsewhere, Stenberg said that while AI reports are “not drowning us, [the] trend is not looking good.”

Stenberg has previously blogged on his own site about AI-generated vulnerability reports, with more details on what they look like and what they get wrong. Seth Larson, security developer-in-residence at the Python Software Foundation, added to Stenberg’s findings with his own examples and suggested actions, as noted by The Register.

“If this is happening to a handful of projects that I have visibility for, then I suspect that this is happening on a large scale to open source projects,” Larson wrote in December. “This is a very concerning trend.”

Open source project curl is sick of users submitting “AI slop” vulnerabilities Read More »

the-third-crisis-dawns-in-foundation-s3-teaser

The Third Crisis dawns in Foundation S3 teaser

We have our first teaser for the upcoming third season of Foundation.

It’s been nearly two years, but the third season of Foundation, Apple TV+’s epic adaptation (or remix) of the Isaac Asimov series, is almost here. The streaming platform released an action-packed teaser of what we can expect from the new ten-episode season: the onset of the Third Crisis, a galactic war, and a shirtless Lee Pace.

(Some spoilers for first two seasons below.)

Showrunner David S. Goyer took great pains in S1 to carefully set up his expansive fictional world, and the scope only broadened in the second season. As previously reported, Asimov’s fundamental narrative arc remains intact, with the series taking place across multiple planets over 1,000 years and featuring a huge cast of characters.

Mathematician Hari Seldon (Jared Harris) developed a controversial theory of “psychohistory,” and his calculations predict the fall of the Empire, ushering in a Dark Age period that will last 30,000 years, after which a second Empire will emerge. The collapse of the Empire is inevitable, but Seldon has a plan to reduce the Dark Ages to a mere 1,000 years through the establishment of a Foundation to preserve all human knowledge so that civilization need not rebuild itself entirely from scratch. He is aided in this endeavor by his math prodigy protegé Gaal Dornick (Lou Llobell).

The biggest change from the books is the replacement of the Empire’s ruling committee with a trio of Eternal Emperor clones called the Cleons—a genetic triune dynasty comprised of Brother Day (Pace), Brother Dusk (Terrence Mann), and Brother Dawn (Cassian Bilton). Technically, they are all perfect incarnations of the same man at different ages, and this is both the source of their strength as a team and of their conflicts. Their guardian is an android, Eto Demerzel (Laura Birn), one of the last surviving androids from the ancient Robot Wars, who is programmed to protect the dynasty at all costs.

The Third Crisis dawns in Foundation S3 teaser Read More »

nasa-scrambles-to-cut-iss-activity-after-trump-budget—its-options-are-not-great

NASA scrambles to cut ISS activity after Trump budget—its options are not great

NASA has not publicly announced the astronauts who will fly on Crew-12 next year, but according to sources, it has already assigned veteran astronaut Jessica Meir and newcomer Jack Hathaway, a former US Navy fighter pilot who joined NASA’s astronaut corps in 2021. If these changes go through, presumably one of these two would be removed from the mission.

Will this actually happen?

The cuts are by no means a certainty. The president’s budget proposal is just the beginning of a monthslong process in which the White House Office of Management and Budget will work with Congress to establish funding levels and programmatic priorities for fiscal year 2026. If this budget process is like those in years past, a final budget may not even be set by the start of the fiscal year this October.

Congress has been broadly supportive of the space station, which is slated to fly through 2030 before being decommissioned. The Trump White House nominee to lead NASA, Jared Isaacman, also spoke in favor of “maximizing” science on the space station during his confirmation hearing last month. In subsequent answers to written questions, Isaacman reaffirmed this position.

“My priority would be to maximize the remaining value of the ISS before it is decommissioned,” Isaacman wrote. “We must prioritize the highest-potential science and research that can be conducted on the station—and do everything possible to ‘crack the code’ on an on orbit economy.”

This comment reflects a desire to focus on science that will help jump-start a commercial economy in low-Earth orbit, as opposed to the White House budget’s desire to focus on research related to the Moon and Mars.

Isaacman has not been confirmed yet—that should happen within the next couple of weeks—so he did not have direct input into setting the White House budget proposal. That process was led by Russell Vought, who leads the White House Office of Management and Budget.

NASA scrambles to cut ISS activity after Trump budget—its options are not great Read More »

zuckerberg’s-dystopian-ai-vision

Zuckerberg’s Dystopian AI Vision

You think it’s bad now? Oh, you have no idea. In his talks with Ben Thompson and Dwarkesh Patel, Zuckerberg lays out his vision for our AI future.

I thank him for his candor. I’m still kind of boggled that he said all of it out loud.

We will start with the situation now. How are things going on Facebook in the AI era?

Oh, right.

Sakib: Again, it happened again. Opened Facebook and I saw this. I looked at the comments and they’re just unsuspecting boomers congratulating the fake AI gen couple😂

Deepfates: You think those are real boomers in the comments?

This continues to be 100% Zuckerberg’s fault, and 100% an intentional decision.

The algorithm knows full well what kind of post this is. It still floods people with them, especially if you click even once. If they wanted to stop it, they easily could.

There’s also the rather insane and deeply embarrassing AI bot accounts they have tried out on Facebook and Instagram.

Compared to his vision of the future? You aint seen nothing yet.

Ben Thompson interviewed Mark Zuckerberg, centering on business models.

It was like if you took a left wing caricature of why Zuckerberg is evil, combined it with a left wing caricature about why AI is evil, and then fused them into their final form. Except it’s coming directly from Zuckerberg, as explicit text, on purpose.

It’s understandable that many leave such interviews and related stories saying this:

Ewan Morrison: Big tech atomises you, isolates you, makes you lonely and depressed – then it rents you an AI friend, and AI therapist, an AI lover.

Big tech are parasites who pretend they are here to help you.

When asked what he wants to use AI for, Zuckerberg’s primary answer is advertising, in particular an ‘ultimate black box’ where you ask for a business outcome and the AI does what it takes to make that outcome happen. I leave all the ‘do not want’ and ‘misalignment maximalist goal out of what you are literally calling a black box, film at 11 if you need to watch it again’ and ‘general dystopian nightmare’ details as an exercise to the reader. He anticipates that advertising will then grow from the current 1%-2% of GDP to something more, and Thompson is ‘there with’ him, ‘everyone should embrace the black box.’

His number two use is ‘growing engagement on the customer surfaces and recommendations.’ As in, advertising by another name, and using AI in predatory fashion to maximize user engagement and drive addictive behavior.

In case you were wondering if it stops being this dystopian after that? Oh, hell no.

Mark Zuckerberg: You can think about our products as there have been two major epochs so far.

The first was you had your friends and you basically shared with them and you got content from them and now, we’re in an epoch where we’ve basically layered over this whole zone of creator content.

So the stuff from your friends and followers and all the people that you follow hasn’t gone away, but we added on this whole other corpus around all this content that creators have that we are recommending.

Well, the third epoch is I think that there’s going to be all this AI-generated content…

So I think that these feed type services, like these channels where people are getting their content, are going to become more of what people spend their time on, and the better that AI can both help create and recommend the content, I think that that’s going to be a huge thing. So that’s kind of the second category.

The third big AI revenue opportunity is going to be business messaging.

And the way that I think that’s going to happen, we see the early glimpses of this because business messaging is actually already a huge thing in countries like Thailand and Vietnam.

So what will unlock that for the rest of the world? It’s like, it’s AI making it so that you can have a low cost of labor version of that everywhere else.

Also he thinks everyone should have an AI therapist, and that people want more friends so AI can fill in for the missing humans there. Yay.

PoliMath: I don’t really have words for how much I hate this

But I also don’t have a solution for how to combat the genuine isolation and loneliness that people suffer from

AI friends are, imo, just a drug that lessens the immediate pain but will probably cause far greater suffering

Well, I guess the fourth one is the normal ‘everyone use AI now,’ at least?

And then, the fourth is all the more novel, just AI first thing, so like Meta AI.

He also blames Llama-4’s terrible reception on user error in setup, and says they now offer an API so people have a baseline implementation to point to, and says essentially ‘well of course we built a version of Llama-4 specifically to score well on Arena, that only shows off how easy it is to steer it, it’s good actually.’ Neither of them, of course, even bothers to mention any downside risks or costs of open models.

The killer app of Meta AI is that it will know all about all your activity on Facebook and Instagram and use it against for you, and also let you essentially ‘talk to the algorithm’ which I do admit is kind of interesting but I notice Zuckerberg didn’t mention an option to tell it to alter the algorithm, and Thompson didn’t ask.

There is one area where I like where his head is at:

I think one of the things that I’m really focused on is how can you make it so AI can help you be a better friend to your friends, and there’s a lot of stuff about the people who I care about that I don’t remember, I could be more thoughtful.

There are all these issues where it’s like, “I don’t make plans until the last minute”, and then it’s like, “I don’t know who’s around and I don’t want to bug people”, or whatever. An AI that has good context about what’s going on with the people you care about, is going to be able to help you out with this.

That is… not how I would implement this kind of feature, and indeed the more details you read the more Zuckerberg seems determined to do even the right thing in the most dystopian way possible, but as long as it’s fully opt-in (if not, wowie moment of the week) then at least we’re trying at all.

Also interviewing Mark Zuckerberg is Dwarkesh Patel. There was good content here, Zuckerberg in many ways continues to be remarkably candid. But it wasn’t as dense or hard hitting as many of Patel’s other interviews.

One key difference between the interviews is that when Zuckerberg lays out his dystopian vision, you get the sense that Thompson is for it, whereas Patel is trying to express that maybe we should be concerned. Another is that Patel notices that there might be more important things going on, whereas to Thompson nothing could be more important than enhancing ad markets.

  1. When asked what changed since Llama 3, Zuckerberg leads off with the ‘personalization loop.’

  2. Zuckerberg still claims Llama 4 Scout and Maverick are top notch. Okie dokie.

  3. He doubles down on ‘open source will become most used this year’ and that this year has been Great News For Open Models. Okie dokie.

  4. His heart’s clearly not in claiming it’s a good model, sir. His heart is in it being a good model for Meta’s particular commercial purposes and ‘product value’ as per people’s ‘revealed preferences.’ That’s the modes he talked about with Thompson.

  5. He’s very explicit about this. OpenAI and Anthropic are going for AGI and a world of abundance, with Anthropic focused on coding and OpenAI towards reasoning. Meta wants fast, cheap, personalized, easy to interact with all day, and (if you add what he said to Thompson) to optimize feeds and recommendations for engagement, and to sell ads. It’s all for their own purposes.

  6. He says Meta is specifically creating AI tools to write their own code for internal use, but I don’t understand what makes that different from a general AI coder? Or why they think their version is going to be better than using Claude or Gemini? This feels like some combination of paranoia and bluff.

  7. Thus, Meta seems to at this point be using the open model approach as a recruiting or marketing tactic? I don’t know what else it’s actually doing for them.

  8. As Dwarkesh notes, Zuckerberg is basically buying the case for superintelligence and the intelligence explosion, then ignoring it to form an ordinary business plan, and of course to continue to have their safety plan be ‘lol we’re Meta’ and release all their weights.

  9. I notice I am confused why their tests need hundreds of thousands or millions of people to be statistically significant? Impacts must be very small and also their statistical techniques they’re using don’t seem great. But also, it is telling that his first thought of experiments to run with AI are being run on his users.

  10. In general, Zuckerberg seems to be thinking he’s running an ordinary dystopian tech company doing ordinary dystopian things (except he thinks they’re not dystopian, which is why he talks about them so plainly and clearly) while other companies do other ordinary things, and has put all the intelligence explosion related high weirdness totally out of his mind or minimized it to specific use cases, even though he intellectually knows that isn’t right.

  11. He, CEO of Meta, says people use what is valuable to them and people are smart and know what is valuable in their lives, and when you think otherwise you’re usually wrong. Queue the laugh track.

  12. First named use case is talking through difficult conversations they need to have. I do think that’s actually a good use case candidate, but also easy to pervert.

  13. (29: 40) The friend quote: The average American only has three friends ‘but has demand for meaningfully more, something like 15… They want more connection than they have.’ His core prediction is that AI connection will be a compliment to human connection rather than a substitute.

    1. I tentatively agree with Zuckerberg, if and only if the AIs in question are engineered (by the developer, user or both, depending on context) to be complements rather than substitutes. You can make it one way.

    2. However, when I see Meta’s plans, it seems they are steering it the other way.

  14. Zuckerberg is making a fully general defense of adversarial capitalism and attention predation – if people are choosing to do something, then later we will see why it turned out to be valuable for them and why it adds value to their lives, including virtual therapists and virtual girlfriends.

    1. But this proves (or implies) far too much as a general argument. It suggests full anarchism and zero consumer protections. It applies to heroin or joining cults or being in abusive relationships or marching off to war and so on. We all know plenty of examples of self-destructive behaviors. Yes, the great classical liberal insight is that mostly you are better off if you let people do what they want, and getting in the way usually backfires.

    2. If you add AI into the mix, especially AI that moves beyond a ‘mere tool,’ and you consider highly persuasive AIs and algorithms, asserting ‘whatever the people choose to do must be benefiting them’ is Obvious Nonsense.

    3. I do think virtual therapists have a lot of promise as value adds, if done well. And also great danger to do harm, if done poorly or maliciously.

  15. Dwarkesh points out the danger of technology reward hacking us, and again Zuckerberg just triples down on ‘people know what they want.’ People wouldn’t let there be things constantly competing for their attention, so the future won’t be like that, he says. Is this a joke?

  16. I do get that the right way to design AI-AR glasses is as great glasses that also serve as other things when you need them and don’t flood your vision, and that the wise consumer will pay extra to ensure it works that way. But where is this trust in consumers coming from? Has Zuckerberg seen the internet? Has he seen how people use their smartphones? Oh, right, he’s largely directly responsible.

    1. Frankly, the reason I haven’t tried Meta’s glasses is that Meta makes them. They do sound like a nifty product otherwise, if execution is good.

  17. Zuckerberg is a fan of various industrial policies, praising the export controls and calling on America to help build new data centers and related power sources.

  18. Zuckerberg asks, would others be doing open models if Meta wasn’t doing it? Aren’t they doing this because otherwise ‘they’re going to lose?’

    1. Do not flatter yourself, sir. They’re responding to DeepSeek, not you. And in particular, they’re doing it to squash the idea that r1 means DeepSeek or China is ‘winning.’ Meta’s got nothing to do with it, and you’re not pushing things in the open direction in a meaningful way at this point.

  19. His case for why the open models need to be American is because our models embody an America view of the world in a way that Chinese models don’t. Even if you agree that is true, it doesn’t answer Dwarkesh’s point that everyone can easily switch models whenever they want. Zuckerberg then does mention the potential for backdoors, which is a real thing since ‘open model’ only means open weights, they’re not actually open source so you can’t rule out a backdoor.

  20. Zuckerberg says the point of Llama Behemoth will be the ability to distill it. So making that an open model is specifically so that the work can be distilled. But that’s something we don’t want the Chinese to do, asks Padme?

  21. And then we have a section on ‘monetizing AGI’ where Zuckerberg indeed goes right to ads and arguing that ads done well add value. Which they must, since consumers choose to watch them, I suppose, per his previous arguments?

To be fair, yes, it is hard out there. We all need a friend and our options are limited.

Roman Helmet Guy (reprise from last week): Zuckerberg explaining how Meta is creating personalized AI friends to supplement your real ones: “The average American has 3 friends, but has demand for 15.”

Daniel Eth: This sounds like something said by an alien from an antisocial species that has come to earth and is trying to report back to his kind what “friends” are.

Sam Ro: imagine having 15 friends.

Modest Proposal (quoting Chris Rock): “The Trenchcoat Mafia. No one would play with us. We had no friends. The Trenchcoat Mafia. Hey I saw the yearbook picture it was six of them. I ain’t have six friends in high school. I don’t got six friends now.”

Kevin Roose: The Meta vision of AI — hologram Reelslop and AI friends keeping you company while you eat breakfast alone — is so bleak I almost can’t believe they’re saying it out loud.

Exactly how dystopian are these ‘AI friends’ going to be?

GFodor.id (being modestly unfair): What he’s not saying is those “friends” will seem like real people. Your years-long friendship will culminate when they convince you to buy a specific truck. Suddenly, they’ll blink out of existence, having delivered a conversion to the company who spent $3.47 to fund their life.

Soible_VR: not your weights, not your friend.

Why would they then blink out of existence? There’s still so much more that ‘friend’ can do to convert sales, and also you want to ensure they stay happy with the truck and give it great reviews and so on, and also you don’t want the target to realize that was all you wanted, and so on. The true ‘AI ad buddy’ plays the long game, and is happy to stick around to monetize that bond – or maybe to get you to pay to keep them around, plus some profit margin.

The good ‘AI friend’ world is, again, one in which the AI friends are complements, or are only substituting while you can’t find better alternatives, and actively work to help you get and deepen ‘real’ friendships. Which is totally something they can do.

Then again, what happens when the AIs really are above human level, and can be as good ‘friends’ as a person? Is it so impossible to imagine this being fine? Suppose the AI was set up to perfectly imitate a real (remote) person who would actually be a good friend, including reacting as they would to the passage of time and them sometimes reaching out to you, and also that they’d introduce you to their friends which included other humans, and so on. What exactly is the problem?

And if you then give that AI ‘enhancements,’ such as happening to be more interested in whatever you’re interested in, having better information recall, watching out for you first more than most people would, etc, at what point do you have a problem? We need to be thinking about these questions now.

I do get that, in his own way, the man is trying. You wouldn’t talk about these plans in this way if you realized how the vision would sound to others. I get that he’s also talking to investors, but he has full control of Meta and isn’t raising capital, although Thompson thinks that Zuckerberg has need of going on a ‘trust me’ tour.

In some ways this is a microcosm of key parts of the alignment problem. I can see the problems Zuckerberg thinks he is solving, the value he thinks or claims he is providing. I can think of versions of these approaches that would indeed be ‘friendly’ to actual humans, and make their lives better, and which could actually get built.

Instead, on top of the commercial incentives, all the thinking feels alien. The optimization targets are subtly wrong. There is the assumption that the map corresponds to the territory, that people will know what is good for them so any ‘choices’ you convince them to make must be good for them, no matter how distorted you make the landscape, without worry about addiction to Skinner boxes or myopia or other forms of predation. That the collective social dynamics of adding AI into the mix in these ways won’t get twisted in ways that make everyone worse off.

And of course, there’s the continuing to model the future world as similar and ignoring the actual implications of the level of machine intelligence we should expect.

I do think there are ways to do AI therapists, AI ‘friends,’ AI curation of feeds and AI coordination of social worlds, and so on, that contribute to human flourishing, that would be great, and that could totally be done by Meta. I do not expect it to be at all similar to the one Meta actually builds.

Discussion about this post

Zuckerberg’s Dystopian AI Vision Read More »