Author name: 9u50fv

bevs-are-better-than-combustion:-the-2025-bmw-i4-xdrive40-review

BEVs are better than combustion: The 2025 BMW i4 xDrive40 review

BMW i4 xDrive40, car review, Cars / /

But it’s not really fair to compare yesterday’s 430i with this i4 xDrive40; with 395 hp (295 kW) and 442 lb-ft (600 Nm) on tap and a $62,300 MSRP, this EV is another rung up the price and power ladders.

The i4 uses BMW’s fifth-generation electric motors, and unlike most other OEMs, BMW uses electrically excited synchronous motors instead of permanent magnets. The front is rated at 255 hp (190 kW) and 243 lb-ft (330 Nm), and the rear maxes out at 308 hp (230 kW) and 295 lb-ft (400 Nm). They’re powered by an 84 kWh battery pack (81 kWh usable), which on 18-inch wheels is good for an EPA range of 287 miles (462 km).

Our test car was fitted with 19-inch wheels, though, which cuts the EPA range to 269 miles (432 km). If you want a long-distance i4, the single-motor eDrive40 on 18-inch wheels can travel 318 miles (511 km) between charges, according to the EPA, which offers an interesting demonstration of the effect of wheel size and single versus dual motors on range efficiency.

A BMW i4 wheel

There’s a new design for the 19-inch M Aero wheels, but they’re part of a $2,200 package. Credit: Jonathan Gitlin

It’s very easy to switch between having the car regeneratively brake when you lift the throttle (in B) or just coast (in D), thanks to the little lever on the center console. (Either way, the car will regeneratively brake when you use the brake pedal, up to 0.3 G, at which point the friction brakes take over.) If you needed to, you could hit 62 mph (100 km/h) in 5.1 seconds from a standstill, which makes it quick by normal standards if not by bench racers. In practice, it’s more than fast enough to merge into a gap or overtake someone if necessary.

During our time with the i4, I averaged a little worse than the EPA numbers. The winter has been relatively mild as a result of climate change, but the weather remained around or below freezing during our week with the i4, and we averaged 3.1 miles/kWh (20 kWh/100 km). Interestingly, I didn’t notice much of a drop when using Sport mode, or much of a gain using Eco mode, on the same 24-mile mix of city streets, suburban arteries, and highways.

BEVs are better than combustion: The 2025 BMW i4 xDrive40 review Read More »

response-to-scott-alexander-on-imprisonment

Response to Scott Alexander on Imprisonment

Response / /

Back in November 2024, Scott Alexander asked: Do longer prison sentences reduce crime?

As a marker, before I began reading the post, I put down here: Yes. The claims that locking people up for longer periods after they are caught doing [X] does not reduce the amount of [X] that gets done, for multiple overdetermined reasons, is presumably rather Obvious Nonsense until strong evidence is provided otherwise.

The potential exception, the reason it might not be Obvious Nonsense, would be if our prisons were so terrible that they net greatly increase the criminality and number of crimes of prisoners once they get out, in a way that grows with the length of the sentence. And that this dwarfs all other effects. This is indeed what Roodman (Scott’s anti-incarceration advocate) claims. Which makes him mostly unique, with the other anti-incarceration advocates being a lot less reasonable.

In which case, yes, we should make dramatic changes to fix that, rather than arguing over sentence lengths, or otherwise act strategically (e.g. either lock people up for life, or barely lock them up at all, and never do anything in between?) But the response shouldn’t be to primarily say ‘well I guess we should stop locking people up then.’

Scott Alexander is of course the person we charge with systematically going through various studies and trying to draw conclusions in these spots. So here we are.

First up is the deterrence effect.

Scott Alexander: Rational actors consider the costs and benefits of a strategy before acting. In general, this model has been successfully applied to the decision to commit crime. Studying deterrence is complicated, and usually tries to tease out effects from the certainty, swiftness, and severity of punishment; here we’ll focus on severity.

According to every study and analysis I’ve seen, certainty and swiftness matter a lot, and indeed you get more bang for your buck on those than you do on severity past some reasonable point. The question on severity is if we’re reaching decreasing marginal returns.

A bunch of analysis mostly boils down to this:

I think all four of these studies are consistent with an extra year tacked on to a prison sentence deterring crime by about 1%. All studies start with significant prison sentences, and don’t let us conclude that the same would be true with eg increasing a one day sentence to a year-and-a-day.

Helland and Drago et al both suggest that deterrence effects are concentrated upon the least severe crimes. I think this makes sense, since more severe crimes tend to be more driven by emotion or necessity.

I would have predicted a larger effect than this, but it’s not impossible that once you’re already putting someone away for 5+ years you’ve already done most of the deterrence work you’re going to do via sentence length alone – if you thought you’d be caught and cared about your future you wouldn’t be doing it.

The incarceration effects, on the other hand, naively look rather huge. There’s strong evidence that a few people will constantly go around committing all the crime. If you lock up those doing all the crime, they stop doing the crime, and crime goes down. The math is clear. So why didn’t California’s three strikes law do more work?

If you credit three strikes with the change in relative crime for five years after the law was passed, you get a 7% drop, although ‘most criminologists suggest that even this is an overestimate, and the true number is close to zero.’

I actually think the 7% estimate looks low here. We see a general trend beforehand of California’s crime rate spiralling out of control, both in absolute and relative terms. It seems likely this trend had to be stalled before it was reversed, and the gap was essentially gone after a while, and other states were also going ‘tough on crime’ during that period, so the baseline isn’t zero.

So we expected Three Strikes to decrease crime by 83%, but in fact it decreased it by 0-7%. Why?

Because California’s Three Strikes law was weaker than it sounds: it only applied to a small fraction of criminals with three convictions. Only a few of the most severe crimes (eg armed robberies) were considered “strikes”, and even then, there was a lot of leeway for lenient judges and prosecutors to downgrade charges. Even though ~80% of criminals had been arrested three times or more, only 1–4% of criminals arrested in California were punished under the Three Strikes law.

Whereas a Netherlands 10-strike (!) law, allowing for much longer sentences after that, did reduce property crime by 25%, and seems like it was highly efficient. This makes a lot of sense to me and also seems highly justified. At some point, if you’re constantly doing all the crime, including property crime, you have to drop the hammer.

We are often well past that point. As Scott talks about, and this post talks about elsewhere (this was the last section written), the ‘we can’t arrest the 327 shoplifters in NYC who get arrested 20 times per year’ is indeed ‘we suck.’ This isn’t hard. And yes, you can say there’s disconnects where DAs say an arrest is deterrent enough whereas police don’t see a point to arresting someone who will only get released, but that doesn’t explain why we have to keep arresting the same people.

Analyses from all perspectives, that Scott looks at, agree that criminals as a group tend to commit quite a lot of crime, 7-17 crimes per year.

I also note that I think all the social cost estimates are probably way too low, because they aren’t properly taking into account various equilibrium effects.

That’s what I think happened in El Salvador, that Scott is strangely missing. The reason you got a 95% crime decrease is not some statistical result based on starting with lower incarceration rates. It is because before the arrests, the gangs were running wild, were de facto governments fighting wars while the police were powerless. Afterwards, they weren’t. It wasn’t about thinking on the margin.

We also get confirmation that theft is way down in El Salvador, in a ‘now I can have a phone in my hand or car on the street and not expect them to be stolen so often I can’t do that’ sense.

Later on, Roodman attempts to estimate social costs like this:

Roodman uses two methods: first, he values a crime at the average damages that courts award to victims, including emotional damages. Second, he values it at what people will pay – how much money would you accept to get assaulted one extra time in your life?

These estimates still exclude some intangible costs, like the cost of living in a crime-ridden community, but it’s the best we can do for now.

These to me seem like they are both vast underestimates. I don’t think we can just say ‘best we can do’ and dismiss the community costs.

I would pay a lot to not be assaulted one time. I’d pay so much more to both not be assaulted, and also not to have the fear of assault living rent free in my head all the time (and for women this has to be vastly worse than that). And for everyone around me to also not having that dominate their thinking and actions.

So yeah, I find these estimates here rather absurdly low. If we value a life at $12 million when calculating health care interventions, you’re telling me marginal murders only have a social cost of $9.4 million? That’s crazy, murder is considered much worse than other deaths and tends to happen to the young. I think you have to at least double the general life value.

The rape number is even crazier to me.

Here’s Claude (no, don’t trust this, but it’s a sanity check):

Claude Sonnet 3.5: Studies estimating the total societal cost per rape (including both tangible and intangible costs) typically range from $150,000 to $450,000 in direct costs. When including long-term impacts and psychological harm, some analyses place the full societal cost at over $1 million per incident.

…

Total cost estimates per burglary typically range from $3,000 to $7,000 in direct costs, with comprehensive social cost estimates ranging from $10,000-$25,000 per incident when including psychological impact and system costs.

So yeah, I think even without norm and equilibrium effects these numbers are likely off by a factor of at least 2, and then they’re wrong by a lot again for those reasons.

Scott later points out that thinking on the margin gets confusing when different areas have different margins, and in some sense the sum of the margins must be the total effect, but some sort of multiple equilibrium (toy) model seems closer to how I actually think about all this.

The aftereffects of imprisonment forcing or leading people deeper into crime is the actual counterargument. And as Scott points out, it’s crazy to try and claim that the impact here is zero:

As far as I can tell, most criminologists are confused on this point. They’re going to claim that the sign of aftereffects is around zero, or hard to measure – then triumphantly announce that they’ve proven prison doesn’t prevent crime.

If the effect here is around zero, one that’s quite the coincidence, and two that would mean prison reduces crime. The actual argument that prison doesn’t reduce crime, that isn’t Obvious Nonsense, is if the aftereffects are very large and very negative.

Here’s one study that definitely didn’t find that.

Scott then says there are tons of other studies and it’s all very complicated. There’s lots of weirdness throughout, such as Berger saying everyone pleading guilty means a ‘unusual study population’ despite essentially everyone pleading guilty in our system.

Roodman not only concludes that longer sentences increase crime after, but that harsher ones also do so, while saying that effects at different times and places differ.

Another suggestion is that perhaps modest sentences (e.g. less than two years) are more relatively disruptive versus incentivizing, and thus those in particular make things worse. That doesn’t seem impossible, but also the incentive effects on the margin here seem pretty huge. You need to be disruptive, or where is the punishment, and thus where is the deterrence? Unless we have a better idea?

Given the importance of both swiftness and certainty, a strategy of ‘we won’t do much to you until we really do quite a lot to you’ here would be even worse than the three strikes law.

I mean, I can think of punishments people want to avoid, but that aren’t prison and thus won’t cost you your job or family… but we’ve pretty much decided to take all of those off the table?

In general, I’ve taken to finding Scott’s ‘let’s look at all the studies’ approach to such questions to be increasingly not how I think about questions at all. Studies aren’t the primary way I look for or consider evidence. They’re one source among many, and emphasizing them this much seems like a cop out more than an attempt to determine what is happening.

I do agree broadly with Scott’s conclusions, of:

  1. More incarceration net reduces crime.

  2. We have more cost-effective crime reduction options available.

  3. It would be cost effective to spend more on crime reduction.

To that I would add:

  1. More incarceration seems net beneficial at current margins, here, because the estimates of social cost of (real, non-victimless) crime are unreasonably low even without equilibrium effects, and also there are large equilibrium effects.

  2. We have additional even more effective options, but we keep not using them.

  3. Some of that is ‘not ready for that conversation’ or misplaced ethical concerns.

  4. Some of that is purely we’re bad at it.

  5. We should beware medium-sized incarceration periods (e.g. 1-3 years).

  6. Most importantly: Our current prison system is really bad in that many aspects cause more crime after release rather than less, and the low hanging fruit is fixing this so that it isn’t true.

At minimum, we absolutely should be funding the police and courts sufficiently to investigate crimes properly, arrest everyone who does crimes on the regular (while accepting that any given crime may not be caught), and to deal with all the resulting cases.

And of course we should adjust the list of crimes, and the punishments, to match that new reality. Otherwise, we are burning down accumulated social capital, and I fear we are doing it rather rapidly.

Scott then followed up with a highlights from the comments post.

It starts with comments about criminal psychology, which I found both fascinating and depressing. If prospective criminals don’t care about magnitude of risks only certainty of risk and they’re generally not competent to stay on the straight and narrow track and make it work, and they often don’t even see prison as worse than their lives anyway, you don’t have many options.

The obvious play is to invest bigly in ensuring you reliably catch people, and reduce sentences since the extra time isn’t doing much work, which is consistent with the conclusions above but seems very hard to implement at scale. Perhaps with AI we can move towards that world over time?

I very much appreciated Scott’s response to the first comment, which I’ll quote here:

Jude: This . . . matches my experience working with some low-income boys as a volunteer. It took me too long to realize how terrible they were at time-discounting and weighing risk. Where I was saying: “this will only hurt a LITTLE but that might RUIN your life,” they heard: “this WILL hurt a little but that MIGHT ruin your life.” And “will” beats “might” every time.

One frustrating kid I dealt with drove without a license (after losing it) several times and drove a little drunk occasionally, despite my warnings that he would get himself in a lot of trouble. He wasn’t caught and proudly told me that I was wrong: nothing bad happened, whereas something bad definitely would have happened if he didn’t get home after X party. Surprise surprise: two years later he’s in jail after drunk driving and having multiple violations of driving without a license.

Scott Alexander: The “proudly told me that I was wrong – nothing bad happened” reminds me of the Generalized Anti-Caution Argument – “you said we should worry about AI, but then we invented a new generation of large language model, and nothing bad happened!” Sometimes I think the difference between smart people and dumb people is that dumb people make dumb mistakes in Near Mode, and smart people only make them in Far Mode – the smarter you are, the more abstract you go before making the same dumb mistake.

Yep. We need to figure out a better answer in these situations. What distinguishes situations where someone can understand ‘any given time you do this is probably net positive but it occasionally is massively terrible so don’t do it’ from ‘this was net positive several times so your warnings are stupid?’

There was some hopefulness, in this claim that the criminal class does still care about punishment magnitude, and about jail versus prison, as differing in kind – at some point the punishment goes from ‘no big deal’ to very much a big deal, and plea bargains reflect that. Which suggests you either want to enforce the law very consistently, or you want to occasionally go big enough to trigger the break points. But then the next comment says no, the criminals care so little they don’t even know what their punishments would be until they happen.

These could be different populations, or different interpretations, but mostly this seems like a direct contradiction. None of this is easy.

Discussion about this post

Response to Scott Alexander on Imprisonment Read More »

ryzen-9-9950x3d-review:-seriously-fast,-if-a-step-backward-in-efficiency

Ryzen 9 9950X3D review: Seriously fast, if a step backward in efficiency

AMD, Reviews, Ryzen, Ryzen 9800X3D, Ryzen 9950x3d, Tech, zen 5 / /

Not a lot of people actually need this thing, but if you do, it’s very good.

AMD’s Ryzen 9 9950X3D. Credit: Andrew Cunningham

AMD’s Ryzen 9 9950X3D. Credit: Andrew Cunningham

Even three years later, AMD’s high-end X3D-series processors still aren’t a thing that most people need to spend extra money on—under all but a handful of circumstances, your GPU will be the limiting factor when you’re running games, and few non-game apps benefit from the extra 64MB chunk of L3 cache that is the processors’ calling card. They’ve been a reasonably popular way for people with old AM4 motherboards to extend the life of their gaming PCs, but for AM5 builds, a regular Zen 4 or Zen 5 CPU will not bottleneck modern graphics cards most of the time.

But high-end PC building isn’t always about what’s rational, and people spending $2,000 or more to stick a GeForce RTX 5090 into their systems probably won’t worry that much about spending a couple hundred extra dollars to get the fastest CPU they can get. That’s the audience for the new Ryzen 9 9950X3D, a 16-core, Zen 5-based, $699 monster of a processor that AMD begins selling tomorrow.

If you’re only worried about game performance (and if you can find one), the Ryzen 7 9800X3D is the superior choice, for reasons that will become apparent once we start looking at charts. But if you want fast game performance and you need as many CPU cores as you can get for other streaming or video production or rendering work, the 9950X3D is there for you. (It’s a little funny to me that this a chip made almost precisely for the workload of the PC building tech YouTubers who will be reviewing it.)  It’s also a processor that Intel doesn’t have any kind of answer to.

Second-generation 3D V-Cache

Layering the 3D V-Cache under the CPU die has made most of the 9950X3D’s improvements possible. Credit: AMD

AMD says the 9000X3D chips use a “second-generation” version of its 3D V-Cache technology after using the same approach for the Ryzen 5000 and 7000 processors. The main difference is that, where the older chips stack the 64MB of extra L3 cache on top of the processor die, the 9000 series stacks the cache underneath, making it easier to cool the CPU silicon.

This makes the processors’ thermal characteristics much more like a typical Ryzen CPU without the 3D V-Cache. And because voltage and temperatures are less of a concern, the 9800X3D, 9900X3D, and 9950X3D all support the full range of overclocking and performance tuning tools that other Ryzen CPUs support.

The 12- and 16-core Ryzen X3D chips are built differently from the 8-core. As we’ve covered elsewhere, AMD’s Ryzen desktop processors are a combination of chiplets—up to two CPU core chiplets with up to eight CPU cores each and a separate I/O die that handles things like PCI Express and USB support. In the 9800X3D, you just have one CPU chiplet, and the 64MB of 3D V-Cache is stacked underneath. For the 9900X3D and 9950X3D, you get one 8-core CPU die with V-Cache underneath and then one other CPU die with 4 or 8 cores enabled and no extra cache.

AMD’s driver software is responsible for deciding what apps get run on which CPU cores. Credit: AMD

It’s up to AMD’s chipset software to decide what kinds of apps get to run on each kind of CPU core. Non-gaming workloads prioritize the normal CPU cores, which are generally capable of slightly higher peak clock speeds, while games that benefit disproportionately from the extra cache are run on those cores instead. AMD’s software can “park” the non-V-Cache CPU cores when you’re playing games to ensure they’re not accidentally being run on less-suitable CPU cores.

This technology will work the same basic way for the 9950X3D as it did for the older 7950X3D, but AMD has made some tweaks. Updates to the chipset driver mean that you can swap your current processor out for an X3D model without needing to totally reinstall Windows to get things working, for example, which was AMD’s previous recommendation for the 7000 series. Another update will improve performance for Windows 10 systems with virtualization-based security (VBS) enabled, though if you’re still on Windows 10, you should be considering an upgrade to Windows 11 so you can keep getting security updates past October.

And for situations where AMD’s drivers can’t automatically send the right workloads to the right kinds of cores, AMD also maintains a compatibility database of applications that need special treatment to take advantage of the 3D V-Cache in the 9900X3D and 9950X3D. AMD says it has added a handful of games to that list for the 9900/9950X3D launch, including Far Cry 6, Deus Ex: Mankind Divided, and a couple of Total War games, among others.

Testbed notes

Common elements to all the platforms we test in our CPU testbed include a Lian Li O11 Air Mini case with an EVGA-provided Supernova 850 P6 power supply and a 280 mm Corsair iCue H115i Elite Capellix AIO cooler.

Since our last CPU review, we’ve done a bit of testbed updating to make sure that we’re accounting for a bunch of changes and turmoil on both Intel’s and AMD’s sides of the fence.

For starters, we’re running Windows 11 24H2 on all systems now, which AMD has said should marginally improve performance for architectures going all the way back to Zen 3 (on the desktop, the Ryzen 5000 series). The company made this revelation after early reviewers of the Ryzen 9000 series couldn’t re-create the oddball conditions of their own internal test setups.

As for Intel, the new testing incorporates fixes for the voltage spiking, processor-destroying bugs that affected 13th- and 14th-generation Core processors, issues that Intel fixed in phases throughout 2024. For the latest Core Ultra 200-series desktop CPUs, it also includes performance fixes Intel introduced in BIOS updates and drivers late last year and early this year. (You might have noticed that we didn’t run reviews of the 9800X3D or the Core Ultra 200 series at the time; all of this re-testing of multiple generations of CPUs was part of the reason why).

All of this is to say that any numbers you’re seeing in this review represent recent testing with newer Windows updates, BIOS updates, and drivers all installed.

One thing that isn’t top of the line at the moment is the GeForce RTX 4090, though we are using that now instead of a Radeon RX 7900 XTX.

The RTX 50 series was several months away from being announced when we began collecting updated test data, and we opted to keep the GPU the same for our 9950X3D testing so that we’d have a larger corpus of data to compare the chip to. The RTX 4090 is still, by a considerable margin, the second-fastest consumer GPU that exists right now. But at some point, when we’re ready to do yet another round of totally-from-scratch retesting, we’ll likely swap a 5090 in just to be sure we’re not bottlenecking the processor.

Performance and power: Benefits with fewer drawbacks

The 9950X3D has the second-highest CPU scores in our gaming benchmarks, and it’s behind the 9800X3D by only a handful of frames. This is one of the things we meant when we said that the 9800X3D was the better choice if you’re only worried about game performance. The same dynamic plays out between other 8- and 16-core Ryzen chips—higher power consumption and heat in the high-core-count chips usually bring game performance down just a bit despite the nominally higher boost clocks.

You’ll also pay for it in power consumption, at least at each chip’s default settings. On average, the 9950X3D uses 40 or 50 percent more power during our gaming benchmarks than the 9800X3D running the same benchmarks, even though it’s not capable of running them quite as quickly. But it’s similar to the power use of the regular 9950X, which is quite a bit slower in these gaming benchmarks, even if it does have broadly similar performance in most non-gaming benchmarks.

What’s impressive is what you see when you compare the 9950X3D to its immediate predecessor, the 7950X3D. The 9950X3D isn’t dramatically faster in games, reflecting Zen 5’s modest performance improvement over Zen 4. But the 9950X3D is a lot faster in our general-purpose benchmarks and other non-gaming CPU benchmarks because the changes to how the X3D chips are packaged have helped AMD keep clock speeds, voltages, and power limits pretty close to the same as they are for the regular 9950X.

In short, the 7950X3D gave up a fair bit of performance relative to the 7950X because of compromises needed to support 3D V-Cache. The 9950X3D doesn’t ask you to make the same compromises.

Testing the 9950X3D in its 105 W Eco Mode.

That comes with both upsides and downsides. For example, the 9950X3D looks a lot less power-efficient under load in our Handbrake video encoding test than the 7950X3D because it is using the same amount of power as a normal Ryzen processor. But that’s the other “normal” thing about the 9950X3D—the ability to manually tune those power settings and boost your efficiency if you’re OK with giving up a little performance. It’s not an either/or thing. And at least in our testing, games run just as fast when you set the 9950X3D to use the 105 W Eco Mode instead of the 170 W default TDP.

As for Intel, it just doesn’t have an answer for the X3D series. The Core Ultra 9 285K is perfectly competitive in our general-purpose CPU benchmarks and efficiency, but the Arrow Lake desktop chips struggle to compete with 14th-generation Core and Ryzen 7000 processors in gaming benchmarks, to say nothing of the Ryzen 9000 and to say even less than nothing of the 9800X3D or 9950X3D. That AMD has closed the gap between the 9950X and 9950X3D’s performance in our general-purpose CPU benchmarks means it’s hard to make an argument for Intel here.

The 9950X3D stands alone

I’m not and have never been the target audience for either the 16-core Ryzen processors or the X3D-series processors. When I’m building for myself (and when I’m recommending mainstream builds for our Ars System Guides), I’m normally an advocate for buying the most CPU you can for $200 or $300 and spending more money on a GPU.

But for the game-playing YouTubing content creators who are the 9950X3D’s intended audience, it’s definitely an impressive chip. Games can hit gobsmackingly high frame rates at lower resolutions when paired with a top-tier GPU, behind (and just barely behind) AMD’s own 9800X3D. At the same time, it’s just as good at general-use CPU-intensive tasks as the regular 9950X, fixing a trade-off that had been part of the X3D series since the beginning. AMD has also removed the limits it has in place on overclocking and adjusting power limits for the X3D processors in the 5000 and 7000 series.

So yes, it’s expensive, and no, most people probably don’t need the specific benefits it provides. It’s also possible that you’ll find edge cases where AMD’s technology for parking cores and sending the right kinds of work to the right CPU cores doesn’t work the way it should. But for people who do need or want ultra-high frame rates at lower resolutions or who have some other oddball workloads that benefit from the extra cache, the 9950X3D gives you all of the upsides with no discernible downsides other than cost. And, hey, even at $699, current-generation GPU prices almost make it look like a bargain.

The good

  • Excellent combination of the 9800X3D’s gaming performance and the 9950X’s general-purpose CPU performance
  • AMD has removed limitations on overclocking and power limit tweaking
  • Pretty much no competition for Intel for the specific kind of person the 9950X3D will appeal to

The bad

  • Niche CPUs that most people really don’t need to buy
  • Less power-efficient out of the box than the 7950X3D, though users have latitude to tune efficiency manually if they want
  • AMD’s software has sometimes had problems assigning the right kinds of apps to the right kinds of CPU cores, though we didn’t have issues with this during our testing

The ugly

  • Expensive

Photo of Andrew Cunningham

Andrew is a Senior Technology Reporter at Ars Technica, with a focus on consumer tech including computer hardware and in-depth reviews of operating systems like Windows and macOS. Andrew lives in Philadelphia and co-hosts a weekly book podcast called Overdue.

Ryzen 9 9950X3D review: Seriously fast, if a step backward in efficiency Read More »

former-google-ceo-eric-schmidt-is-the-new-leader-of-relativity-space

Former Google CEO Eric Schmidt is the new leader of Relativity Space

relativity space, Space / /

Another Silicon Valley investor is getting into the rocket business.

Former Google chief executive Eric Schmidt has taken a controlling interest in the Long Beach, California-based Relativity Space. The New York Times first reported the change becoming official, after Schmidt told employees in an all-hands meeting on Monday.

Schmidt’s involvement with Relativity has been quietly discussed among space industry insiders for a few months. Multiple sources told Ars that he has largely been bankrolling the company since the end of October, when the company’s previous fundraising dried up.

It is not immediately clear why Schmidt is taking a hands-on approach at Relativity. However, it is one of the few US-based companies with a credible path toward developing a medium-lift rocket that could potentially challenge the dominance of SpaceX and its Falcon 9 rocket. If the Terran R booster becomes commercially successful, it could play a big role in launching megaconstellations.

Schmidt’s ascension also means that Tim Ellis, the company’s co-founder, chief executive, and almost sole public persona for nearly a decade, is now out of a leadership position.

“Today marks a powerful new chapter as Eric Schmidt becomes Relativity’s CEO, while also providing substantial financial backing,” Ellis wrote on the social media site X. “I know there’s no one more tenacious or passionate to propel this dream forward. We have been working together to ensure a smooth transition, and I’ll proudly continue to support the team as Co-founder and Board member.”

Terran R’s road to launch

On Monday, Relativity also released a nearly 45-minute video that outlines the development of the Terran R rocket to date and the lengths to which it must go to reach the launch pad. Tellingly, Ellis appears only briefly in the video, which features several other senior officials who presumably will remain with the company, including Chief Operating Officer Zach Dunn.

Former Google CEO Eric Schmidt is the new leader of Relativity Space Read More »

hbo-drops-the-last-of-us-s2-trailer

HBO drops The Last of Us S2 trailer

culture, Entertainment, streaming television, the last of us, Trailers / /

Pedro Pascal returns as Joel in The Last of Us S2.

HBO released a one-minute teaser of the hotly anticipated second season of The Last of Us—based on Naughty Dog’s hugely popular video game franchise—during CES in January. We now have a full trailer, unveiled at SXSW after the footage leaked over the weekend, chock-full of Easter eggs for gaming fans of The Last of Us Part II.

(Spoilers for S1 below.)

The series takes place in the 20-year aftermath of a deadly outbreak of mutant fungus (Cordyceps) that turns humans into monstrous zombie-like creatures (the Infected, or Clickers). The world has become a series of separate totalitarian quarantine zones and independent settlements, with a thriving black market and a rebel militia known as the Fireflies making life complicated for the survivors. Joel (Pedro Pascal) is a hardened smuggler tasked with escorting the teenage Ellie (Bella Ramsay) across the devastated US, battling hostile forces and hordes of zombies, to a Fireflies unit outside the quarantine zone. Ellie is special: She is immune to the deadly fungus, and the hope is that her immunity holds the key to beating the disease.

S2 is set five years after the events of the first season and finds the bond beginning to fray between plucky survivors Joel and Ellie. That’s the inevitable outcome of S1’s shocking finale, when they finally arrived at their destination, only to discover the secret to her immunity to the Cordyceps fungus meant Ellie would have to die to find a cure. Ellie was willing to sacrifice herself, but once she was under anesthesia, Joel went berserk and killed all the hospital staff to save her life—and lied to Ellie about it, claiming the staff were killed by raiders.

HBO drops The Last of Us S2 trailer Read More »

better-than-the-real-thing?-spark-2-packs-39-amp-sims-into-$300-bluetooth-speaker

Better than the real thing? Spark 2 packs 39 amp sims into $300 Bluetooth speaker

AMP, Features, Guitars, positive grid, Reviews, spark 2 / /

Digital amp modeling goes very, very portable.

The Spark 2 from Positive Grid looks like a miniature old-school amp, but it is, essentially, a computer with some knobs and a speaker. It has Bluetooth, USB-C, and an associated smartphone app. It needs firmware updates, which can brick the device—ask me how I found this out—and it runs code on DSP chips. New guitar tones can be downloaded into the device, where they run as software rather than as analog electrical circuits in an amp or foot pedal.

In other words, the Spark 2 is the latest example of the “software-ization” of music.

Forget the old image of a studio filled with a million-dollar, 48-track mixing board from SSL or API and bursting with analog amps, vintage mics, and ginormous plate reverbs. Studios today are far more likely to be digital, where people record “in the box” (i.e., they track and mix on a computer running software like Pro Tools or Logic Pro) using digital models of classic (and expensive) amplifiers, coded by companies like NeuralDSP and IK Multimedia. These modeled amp sounds are then run through convolution software that relies on digital impulse responses captured from different speakers and speaker cabinets. They are modified with effects like chorus and distortion, which are all modeled, too. The results can be world-class, and they’re increasingly showing up on records.

Once the sounds are recorded, a mixer will often use digital plugins to replicate studio gear like tape delays, FET compressors, and reverbs (which may be completely algorithmic or may rely on impulse responses captured from real halls, studios, plates, and spring reverbs). These days, even the microphones might be digitally modeled by companies like Slate, Antelope, and Universal Audio.

This has put incredible power into the hands of home musicians; for a couple of thousand bucks, most home studios can own models of gear that would have cost more than a house 20 years ago. But one downside of this shift to software is that all the annoying quirks of computing devices have followed.

Want to rock out to the classic Marshall tones found in Universal Audio’s “Lion” amp simulator plugin? Just plug your guitar into your audio interface, connect the interface to a computer via USB, launch a DAW, instantiate the plugin on a blank track, choose the correct input, activate input monitoring so you can hear the results of your jamming, and adjust your DAW’s buffer size to something small in an attempt to prevent latency. A problem with any item on that list means “no jamming for you.”

You may be prompted to update the firmware in your audio interface, or to update your operating system, or to update your DAW—or even its plugins. Oh, and did I mention that Universal Audio uses the truly terrible iLok DRM system and that if your Wi-Fi drops for even a few minutes, the plugins will deactivate? Also, you’ll need to run a constant companion app in the background called UA Connect, which itself can be prone to problems.

Assuming everything is up to date and working, you’re still tethered to your computer by a cable, and you have to make all your settings tweaks with a mouse. After a day of working on computers, this is not quite how I want to spend my “music time.”

But the upsides of digital modeling are just too compelling to return to the old, appliance-like analog gear. For one thing, the analog stuff is expensive. The Lion amp plugin mentioned above gives you not one but several versions of a high-quality Marshall head unit—each one costing thousands of dollars—but you don’t need to lift it (they’re heavy!), mic it (annoying!), or play it at absurdly low levels because your baby is sleeping upstairs. For under a hundred bucks, you can get that sound of an overdriven Marshall turned up to 75 percent and played through several different speaker cabinet options (each of these is also expensive!) right on your machine.

Or consider the Tone King Imperial Mk II, a $2,700, Fender-style amp built in the US. It sounds great. But NeuralDSP offers a stunning digital model for a hundred bucks—and it comes with compressor, overdrive, delay, and reverb pedals, to say nothing of a tuner, a doubler, a pitch-shifter, and a ton of great presets.

So I want the digital amp modeling, but I also want—sometimes, at least—the tactile simplicity of physical knobs and well-built hardware. Or I want to jack in and play without waking up a computer, logging in, launching apps, or using a mouse and an audio interface. Or I want to take my amp models to places where finicky computers aren’t always welcome, like the stage of a club.

Thanks to hardware like the Profiler from Kemper, the Helix gear from Line6, the Cortex pedalboards from NeuralDSP, or Tonex gear from IK Multimedia, this is increasingly common.

The Spark line from Positive Grid has carved out its own niche in this world by offering well-built little amps that run Positive Grid’s digital amp and effects simulations. (If you don’t want the hardware, the company sells its modeling software for PC and Mac under the “Bias” label.)

The Spark 2 is the latest in this line, and I’ve been putting it through its paces over the last couple of months.

Let’s cut right to the conclusion: The Spark 2 is a well-designed, well-built piece of gear. For $300, you get a portable, 50-watt practice amp and Bluetooth speaker that can store eight guitar tones onboard and download thousands more using a smartphone app. Its models aren’t, to my ears, the most realistic out there, but if you want a device to jack into and jam, to play along with backing tracks or loops, or to record some creative ideas, this fits the bill.

Photo of Spark 2.

Credit: Positive Grid

Good practice

Everything about the Spark 2 feels well-built. The unit is surprisingly solid, and it comes with a carrying strap for portability. If you want to truly live the wire-free lifestyle, you can buy a battery pack for $79 that gives you several hours of juice.

For a practice amp, the Spark 2 is also well-connected. It has Bluetooth for streaming audio—but it also has a 3.5 mm aux in jack. It has decent, if somewhat boxy-sounding, speakers, and they get quite loud—but it also has two quarter-inch line out jacks. It has a guitar input jack and a headphone jack. It can use a power supply or a battery. It can connect to a computer via USB, and you can even record that way if you don’t have another audio interface.

Most of the unit’s top is taken up with chunky knobs. These let you select one of the eight onboard presets or adjust model parameters like gain, EQ, modulation, delay, and reverb. There’s also a knob for blending your guitar audio with music played through the device.

Buttons provide basic access to a tuner and a looper, though the associated app unlocks more complex options.

So about that app. It’s not necessary to use the Spark 2, but you’ll need the app if you want to download or create new tones from the many pieces of modeled gear. Options here go far beyond what’s possible with the knobs atop the physical unit.

Spark models a chamber reverb, for instance, which is basically a reflective room into which a speaker plays sound that a microphone picks up. The Spark chamber lets you adjust the volume level of the reverb signal, the reflection time of the chamber, the “dwell” time of the sound in the room, the amount of sound damping, and whether the sound will have some of its lows or highs cut off. (This is common in reverbs to avoid excessive low-end “mud” or top-end “brightness” building up in the reverberating signal.) You’ll need the app to adjust most of these options; the “reverb” control on the Spark 2 simply changes the level.

There’s a fair bit of modeled gear on offer: one noise gate, six compressors, 14 drive pedals, 39 amps, 13 EQ units, six delays, and nine reverbs. Most of these have numerous options. It is not nearly as overwhelming as a package like Amplitube for PCs and Macs, but it’s still a lot of stuff.

To run it all, Positive Grid has beefed up the computational power of the Spark series. The company told me that digital signal processing power has doubled since the original Spark lineup, which allows for “smoother transitions between tones, richer effects, and an expanded memory for presets and loops.” The system runs on an M7 chip “developed specifically for expanded processing power and precise tone reproduction,” and the extra power has allowed Positive Grid to run more complex models on-device, improving their preamp and amplifier sag modeling.

Despite the DSP increase, the results here just don’t compare with the sort of scary-precise tube amp and effects simulations you can run on a computer or a far more expensive hardware modeling rig. I could never get clean and “edge of breakup” tones to sound anything other than artificial, though some of the distortion sounds were quite good. Reverbs and delays also sounded solid.

But the Spark 2 wasn’t really designed for studio-quality recording, and Positive Grid is candid about this. The models running on the Spark 2 are inspired by the company’s computer work, but they are “optimized for an all-in-one, mobile-friendly playing experience,” I was told. The Spark 2 is meant for “practice, jamming, and basic recording,” and those looking for “studio-level control and complex setups” should seek out something else.

This tracks with my experience. Compared to a regular amp, the Spark 2 is crazy portable. When testing the unit, I would haul it between rooms without a second thought, searching for a place to play that wouldn’t annoy some member of my family. (Headphones? Never!) Thanks to the optional battery, I didn’t even need to plug it in. It was a simple, fun way to get some electric guitar practice in without using a screen or a computer, and its sound could fill an entire room. Compared to the weight and hassle of moving a “real” amp, this felt easy.

About that app

I’ve been talking about the Spark 2 and its screen-free experience, but of course you do need to use the app to unlock more advanced features and download new tones onto the hardware. So how good is the software?

For modifying the gear in your presets, the app works fine. Every piece of gear has a nice picture, and you just flick up or down to get a piece of equipment into or out of the effects chain. Changing parameters is simple, with large numbers popping up on screen whenever you touch a virtual control, and you can draw from a huge library of pre-made effect chains.

The app also features plenty of backing music that it can play over the Spark 2. This includes backing tracks, tabbed songs, and the “groove looper,” giving you plenty of options to work on your soloing, but it’s the artificial intelligence that Positive Grid is really pitching this time around.

You are legally required to shoehorn “AI” into every product launch now, and Positive Grid put its AI tools into the app. These include Smart Jam, which tries to adapt to your playing and accompany it in real time. The company tells me that Smart Jam was “trained on a combination of musical datasets that analyze chord structures, song patterns, and rhythmic elements,” but I could never get great results from it. Because the system doesn’t know what you’re going to play in advance, there was always a herky-jerky quality as it tried to adapt its backing track to my changing performance.

I had more success with Spark AI, which is a natural language tone-shaping engine. You tell the system what you’re looking for—the solo in “Stairway to Heaven,” perhaps—and it returns several presets meant to approximate that sound. It does work, I’ll say that. The system reliably gave me tone options that were, with a little imagination, identifiable as “in the ballpark” of what I asked for.

Perhaps the main barrier here is simply that the current Spark amp models aren’t always powerful enough to truly copy the sounds you might be looking for. Spark AI is a great way to pull up a tone that’s appropriate for whatever song you might be practicing, and to do so without forcing you to build it yourself out of pieces of virtual gear. In that sense, it’s a nice practice aid.

Rock on

As it’s pitched—a practice amp and Bluetooth speaker that costs $300—Spark 2 succeeds. It’s such a well-built and designed unit that I enjoyed using it every time I played, even if the tones couldn’t match a real tube amp or even top-quality models. And the portability was more useful than expected, even when just using it around the house.

As DSP chips grow ever more powerful, I’m looking forward to where modeling can take us. For recording purposes, some of the best models will continue to run on powerful personal computers. But for those looking to jam, or to play shows, or to haul a guitar to the beach for an afternoon, hardware products running modeling software offer incredible possibilities already—and they will “spark” even more creativity in the years to come.

Photo of Nate Anderson

Better than the real thing? Spark 2 packs 39 amp sims into $300 Bluetooth speaker Read More »

huh?-the-valuable-role-of-interjections

Huh? The valuable role of interjections

history of language, language, Science, syndication / /

Utterances like um, wow, and mm-hmm aren’t garbage—they keep conversations flowing.

Interjections—one-word utterances that aren’t part of a larger sentence—used to be dismissed as irrelevant linguistic detritus. But some linguists now think they play an essential role in regulating conversations. Credit: Daniel Garcia/Knowable Magazine

Interjections—one-word utterances that aren’t part of a larger sentence—used to be dismissed as irrelevant linguistic detritus. But some linguists now think they play an essential role in regulating conversations. Credit: Daniel Garcia/Knowable Magazine

Listen carefully to a spoken conversation and you’ll notice that the speakers use a lot of little quasi-words—mm-hmm, um, huh? and the like—that don’t convey any information about the topic of the conversation itself. For many decades, linguists regarded such utterances as largely irrelevant noise, the flotsam and jetsam that accumulate on the margins of language when speakers aren’t as articulate as they’d like to be.

But these little words may be much more important than that. A few linguists now think that far from being detritus, they may be crucial traffic signals to regulate the flow of conversation as well as tools to negotiate mutual understanding. That puts them at the heart of language itself—and they may be the hardest part of language for artificial intelligence to master.

“Here is this phenomenon that lives right under our nose, that we barely noticed,” says Mark Dingemanse, a linguist at Radboud University in the Netherlands, “that turns out to upend our ideas of what makes complex language even possible in the first place.”

For most of the history of linguistics, scholars have tended to focus on written language, in large part because that’s what they had records of. But once recordings of conversation became available, they could begin to analyze spoken language the same way as writing.

When they did, they observed that interjections—that is, short utterances of just a word or two that are not part of a larger sentence—were ubiquitous in everyday speech. “One in every seven utterances are one of these things,” says Dingemanse, who explores the use of interjections in the 2024 Annual Review of Linguistics. “You’re going to find one of those little guys flying by every 12 seconds. Apparently, we need them.”

Many of these interjections serve to regulate the flow of conversation. “Think of it as a tool kit for conducting interactions,” says Dingemanse. “If you want to have streamlined conversations, these are the tools you need.” An um or uh from the speaker, for example, signals that they’re about to pause, but aren’t finished speaking. A quick huh? or what? from the listener, on the other hand, can signal a failure of communication that the speaker needs to repair.

That need seems to be universal: In a survey of 31 languages around the world, Dingemanse and his colleagues found that all of them used a short, neutral syllable similar to huh? as a repair signal, probably because it’s quick to produce. “In that moment of difficulty, you’re going to need the simplest possible question word, and that’s what huh? is,” says Dingemanse. “We think all societies will stumble on this, for the same reason.”

Other interjections serve as what some linguists call “continuers,” such as mm-hmm — signals from the listener that they’re paying attention and the speaker should keep going. Once again, the form of the word is well suited to its function: Because mm-hmm is made with a closed mouth, it’s clear that the signaler does not intend to speak.

Sign languages often handle continuers differently, but then again, two people signing at the same time can be less disruptive than two people speaking, says Carl Börstell, a linguist at the University of Bergen in Norway. In Swedish Sign Language, for example, listeners often sign yes as a continuer for long stretches, but to keep this continuer unobtrusive, the sender tends to hold their hands lower than usual.

Different interjections can send slightly different signals. Consider, for example, one person describing to another how to build a piece of Ikea furniture, says Allison Nguyen, a psycholinguist at Illinois State University. In such a conversation, mm-hmm might indicate that the speaker should continue explaining the current step, while yeah or OK would imply that the listener is done with that step and it’s time to move on to the next.

Wow! There’s more

Continuers aren’t merely for politeness—they really matter to a conversation, says Dingemanse. In one classic experiment from more than two decades ago, 34 undergraduate students listened as another volunteer told them a story. Some of the listeners gave the usual “I’m listening” signals, while others—who had been instructed to count the number of words beginning with the letter t—were too distracted to do so. The lack of normal signals from the listeners led to stories that were less well crafted, the researchers found. “That shows that these little words are quite consequential,” says Dingemanse.

Nguyen agrees that such words are far from meaningless. “They really do a lot for mutual understanding and mutual conversation,” she says. She’s now working to see if emojis serve similar functions in text conversations.

Storytellers depend on feedback such as mm-hmm and other interjections from their listeners. In this experiment, some listeners were told to count the number of times the storyteller used a word starting with t—a challenging task that prevented them from giving normal feedback. The quality of storytelling declined significantly, with problems like abrupt endings, rambling on, uneven or choppy pacing and overexplaining or justifying the point. Credit: Knowable Magazine

The role of interjections goes even deeper than regulating the flow of conversation. Interjections also help in negotiating the ground rules of a conversation. Every time two people converse, they need to establish an understanding of where each is coming from: what each participant knows to begin with, what they think the other person knows and how much detail they want to hear. Much of this work—what linguists call “grounding”—is carried out by interjections.

“If I’m telling you a story and you say something like ‘Wow!’ I might find that encouraging and add more detail,” says Nguyen. “But if you do something like, ‘Uh-huh,’ I’m going to assume you aren’t interested in more detail.”

A key part of grounding is working out what each participant thinks about the other’s knowledge, says Martina Wiltschko, a theoretical linguist at the Catalan Institution for Research and Advanced Studies in Barcelona, Spain. Some languages, like Mandarin, explicitly differentiate between “I’m telling you something you didn’t know” and “I’m telling you something that I think you knew already.” In English, that task falls largely on interjections.

One of Wiltschko’s favorite examples is the Canadian eh?  “If I tell you you have a new dog, I’m usually not telling you stuff you don’t know, so it’s weird for me to tell you,” she says. But ‘You have a new dog, eh?’ eliminates the weirdness by flagging the statement as news to the speaker, not the listener.

Other interjections can indicate that the speaker knows they’re not giving the other participant what they sought. “If you ask me what’s the weather like in Barcelona, I can say ‘Well, I haven’t been outside yet,’” says Wiltschko. The well is an acknowledgement that she’s not quite answering the question.

Wiltschko and her students have now examined more than 20 languages, and every one of them uses little words for negotiations like these. “I haven’t found a language that doesn’t do these three general things: what I know, what I think you know and turn-taking,” she says. They are key to regulating conversations, she adds: “We are building common ground, and we are taking turns.”

Details like these aren’t just arcana for linguists to obsess over. Using interjections properly is a key part of sounding fluent in speaking a second language, notes Wiltschko, but language teachers often ignore them. “When it comes to language teaching, you get points deducted for using ums and uhs, because you’re ‘not fluent,’” she says. “But native speakers use them, because it helps! They should be taught.” Artificial intelligence, too, can struggle to use interjections well, she notes, making them the best way to distinguish between a computer and a real human.

And interjections also provide a window into interpersonal relationships. “These little markers say so much about what you think,” she says—and they’re harder to control than the actual content. Maybe couples therapists, for example, would find that interjections afford useful insights into how their clients regard one another and how they negotiate power in a conversation. The interjection oh often signals confrontation, she says, as in the difference between “Do you want to go out for dinner?” and “Oh, so now you want to go out for dinner?”

Indeed, these little words go right to the heart of language and what it is for. “Language exists because we need to interact with one another,” says Börstell. “For me, that’s the main reason for language being so successful.”

Dingemanse goes one step further. Interjections, he says, don’t just facilitate our conversations. In negotiating points of view and grounding, they’re also how language talks about talking.

“With huh?  you say not just ‘I didn’t understand,’” says Dingemanse. “It’s ‘I understand you’re trying to tell me something, but I didn’t get it.’” That reflexivity enables more sophisticated speech and thought. Indeed, he says, “I don’t think we would have complex language if it were not for these simple words.”

Photo of Knowable Magazine

Knowable Magazine explores the real-world significance of scholarly work through a journalistic lens.

Huh? The valuable role of interjections Read More »

maserati-kills-electric-version-of-mc20-supercar-for-lack-of-demand

Maserati kills electric version of MC20 supercar for lack of demand

Cars, Maserati MC20 / /

Electric motors are, in so many ways, much better than internal combustion engines. They don’t waste most of the energy you put into them as heat and sound, they’re easy to control, and they make huge amounts of torque almost instantly. Having recently driven BMW’s 430i and i4 back to back over the course of two weeks, the electric version was easier in traffic and more responsive on a twisty road. Electric wins, then. Except at the very high end, it seems.

Because even though electric motors can pack a punch, people paying big money for super- and hypercars are increasingly disinterested in those cars being electrified. So much so that Maserati has canceled the all-electric version of the MC20.

The MC20 debuted in 2020. No longer associated with Ferrari after that brand was spun out and IPO’d, the MC20 could offer a full carbon-fiber monocoque and an engine with very clever F1-derived combustion technology, undercutting its now-independent Italian competitor to the tune of more than $100,000 in the process.

Maserati kills electric version of MC20 supercar for lack of demand Read More »

new-research-shows-bigger-animals-get-more-cancer,-defying-decades-old belief

New research shows bigger animals get more cancer, defying decades-old belief

animal research, cancer, cancer research, Science, syndication / /

The answer lies in how quickly body size evolves. We found that birds and mammals that reached large sizes more rapidly have reduced cancer prevalence. For example, the common dolphin, Delphinus delphis evolved to reach its large body size—along with most other whales and dolphins (referred to as cetaceans) about three times faster than other mammals. However, cetaceans tend to have less cancer than expected.

Larger species face higher cancer risks but those that reached that size rapidly evolved mechanisms for mitigating it, such as lower mutation rates or enhanced DNA repair mechanisms. So rather than contradicting Cope’s rule, our findings refine it.

Larger bodies often evolve, but not as quickly in groups where the burden of cancer is higher. This means that the threat of cancer may have shaped the pace of evolution.

Humans evolved to our current body size relatively rapidly. Based on this, we would expect humans and bats to have similar cancer prevalence, because we evolved at a much, much faster rate. However, it is important to note that our results can’t explain the actual prevalence of cancer in humans. Nor is that an easy statistic to estimate.

Human cancer is a complicated story to unravel, with a plethora of types and many factors affecting its prevalence. For example, many humans not only have access to modern medicine but also varied lifestyles that affect cancer risk. For this reason, we did not include humans in our analysis.

Fighting cancer

Understanding how species naturally evolve cancer defences has important implications for human medicine. The naked mole rat, for example, is studied for its exceptionally low cancer prevalence in the hopes of uncovering new ways to prevent or treat cancer in humans. Only a few cancer cases have ever been observed in captive mole rats, so the exact mechanisms of their cancer resistance remain mostly a mystery.

At the same time, our findings raise new questions. Although birds and mammals that evolved quickly seem to have stronger anti-cancer mechanisms, amphibians and reptiles didn’t show the same pattern. Larger species had higher cancer prevalence regardless of how quickly they evolved. This could be due to differences in their regenerative abilities. Some amphibians, like salamanders, can regenerate entire limbs—a process that involves lots of cell division, which cancer could exploit.

Putting cancer into an evolutionary context allowed us to reveal that its prevalence does increase with body size. Studying this evolutionary arms race may unlock new insights into how nature fights cancer—and how we might do the same.The Conversation

Joanna Baker, Postdoctoral Researcher in Evolutionary Biology, University of Reading and George Butler, Career Development Fellow in Cancer Evolution, UCL. This article is republished from The Conversation under a Creative Commons license. Read the original article.

New research shows bigger animals get more cancer, defying decades-old belief Read More »

nasa-officials-undermine-musk’s-claims-about-‘stranded’-astronauts

NASA officials undermine Musk’s claims about ‘stranded’ astronauts

dragon, musk, NASA, Space / /

“We were looking at this before some of those statements were made by the President.”

NASA astronauts Butch Wilmore and Suni Williams aboard the International Space Station. Credit: NASA

Over the last month there has been something more than a minor kerfuffle in the space industry over the return of two NASA astronauts from the International Space Station.

The fate of Butch Wilmore and Suni Williams, who launched on the first crewed flight of Boeing’s Starliner spacecraft on June 5, 2024, has become a political issue after President Donald Trump and SpaceX founder Elon Musk said the astronauts’ return was held up by the Biden White House.

In February, Trump and Musk appeared on FOX News. During the joint interview, the subject of Wilmore and Williams came up. They remain in space today after NASA decided it would be best they did not fly home in their malfunctioning Starliner spacecraft—but would return in a SpaceX-built Crew Dragon.

“At the President’s request, or instruction, we are accelerating the return of the astronauts, which was postponed to a ridiculous degree,” Musk said.

“They got left in space,” Trump added.

“They were left up there for political reasons, which is not good,” Musk concluded.

After this interview, a Danish astronaut named Andreas Mogensen asserted that Musk was lying. “What a lie,” Mogensen wrote on the social media site Musk owns, X. “And from someone who complains about lack of honesty from the mainstream media.”

Musk offered a caustic response to Mogensen. “You are fully retarded,” Musk wrote. “SpaceX could have brought them back several months ago. I OFFERED THIS DIRECTLY to the Biden administration and they refused. Return WAS pushed back for political reasons. Idiot.”

So what’s the truth?

NASA has not directly answered questions about this over the last month. However, the people who really know the answer lie within the human spaceflight programs at the space agency. After one news conference was canceled last month, two key NASA officials were finally made available on a media teleconference on Friday evening. These were Ken Bowersox, associate administrator, Space Operations Mission Directorate, and Steve Stich, manager, of NASA’s Commercial Crew Program, which is responsible for Starliner and Crew Dragon flights.

Musk is essentially making two claims. First, he is saying that last year SpaceX offered to bring Wilmore and Williams home from the International Space Station—and made the offer directly to the Biden Administration. And the offer was refused for “political” reasons.

Second, Musk says that, at Trump’s request, the return of Wilmore and Williams was accelerated. The pair is now likely to return home to Earth as part of the Crew 9 mission later this month, about a week after the launch of a new group of astronauts to the space station. This Crew 10 mission has a launch date of March 12, so Wilmore and Williams could finally fly home about two weeks from now.

Let’s examine each of Musk’s claims in light of what Bowersox and Stich said Friday evening.

Was Musk’s offer declined for political reasons?

On July 14, last year, NASA awarded SpaceX a special contract to study various options to bring Wilmore and Williams home on a Crew Dragon vehicle. At the time, the space agency was considering options if Starliner was determined to be unsafe. Among the options NASA was considering were to fly Wilmore and Williams home on the Crew 8 vehicle attached to the station (which would put an unprecedented six people in the capsule) or asking SpaceX to autonomously fly a Dragon to the station to return Wilmore and Williams separately.

“The SpaceX folks helped us with a lot of options for how we would bring Butch and Suni home on Dragon in a contingency,” Bowersox said during Friday’s teleconference. “When it comes to adding on missions, or bringing a capsule home early, those were always options. But we ruled them out pretty quickly just based on how much money we’ve got in our budget, and the importance of keeping crews on the International Space Station. They’re an important part of maintaining the station.”

As a result, the Crew 9 mission launched in September with just two astronauts. Wilmore and Williams joined that crew for a full, six-month increment on the space station.

Stich said NASA made that decision based on flight schedules to the space station and the orbiting laboratory’s needs. It also allowed time to send SpaceX spacesuits up for the pair of astronauts and to produce seat liners that would make their landing in the water, under parachutes, safe.

“When we laid all that out, the best option was really the one that we’re embarking upon now,” Stich said. “And so we did Crew 9, flying the two empty seats, flying a suit for Butch up, and also making sure that the seats were right for Butch’s anthropometrics, and Suni’s, to return them safely.”

So yes, SpaceX has been working with NASA to present options, including the possibility of a return last fall. However, those discussions were being held within the program levels and their leaders: Stich for Commercial Crew and Dana Weigel for the International Space Station.

“Dana and I worked to come up with a decision that worked for the Commercial Crew Program and Space Station,” Stich said. “And then, Ken (Bowersox), we all we had the Flight Readiness Review process with you, and the Administrator of NASA listened in as well. So we had a recommendation to the agency and that was on the process that we typically use.”

Bowersox confirmed that the decision was made at the programmatic level.

“That’s typically the way our decisions work,” Bowersox said. “The programs work what makes the most sense for them, programmatically, technically. We’ll weigh in at the headquarters level, and in this case we thought the plan that we came up with made a lot of sense.”

During the teleconference, a vice president at SpaceX, Bill Gerstenmaier, was asked directly what offer Musk was referring to when he mentioned the Biden administration. He did not provide a substantive answer.

Musk claims he made an offer directly to senior officials in the Biden Administration. We have no way to verify that, but it does seem clear that the Biden administration never communicated such an offer to lower-level officials within NASA, who made their decision for technical rather than political reasons.

“I think you know we work for NASA, and we worked with NASA cooperatively to do whatever we think was the right thing,” the SpaceX official, Gerstenmaier, replied. “You know, we were willing to support in any manner they thought was the right way to support. They came up with the option you heard described today by them, and we’re supporting that option.”

Did Trump tell NASA to accelerate Butch and Suni’s return?

As of late last year, the Crew 9 mission was due to return in mid-February. However, there was a battery issue with a new Dragon spacecraft that was going to be used to fly Crew 10 into orbit. As a result, NASA announced on December 17 that the return of the crew was delayed into late March or early April.

Then, on February 11, NASA announced that the Crew 10 launch was being brought forward to March 12. This was a couple of weeks earlier than planned, and it was possible because NASA and SpaceX decided to swap out Dragon capsules, using a previously flown vehicle—Crew Dragon Endurance—for Crew 10.

So was this change to accelerate the return of Wilmore and Williams politically driven?

The decision to swap to Endurance was made in late January, Stich said, and this allowed the launch date to be moved forward. Asked if political pressure was a reason, Stich said it was not. “It really was driven by a lot of other factors, and we were looking at this before some of those statements were made by the President and Mr. Musk,” he said.

Bowersox added that this was correct but also said that NASA appreciated the President’s interest in the space program.

“I can verify that Steve has been talking about how we might need to juggle the flights and switch capsules a good month before there was any discussion outside of NASA, but the President’s interest sure added energy to the conversation,” Bowersox said.

Photo of Eric Berger

Eric Berger is the senior space editor at Ars Technica, covering everything from astronomy to private space to NASA policy, and author of two books: Liftoff, about the rise of SpaceX; and Reentry, on the development of the Falcon 9 rocket and Dragon. A certified meteorologist, Eric lives in Houston.

NASA officials undermine Musk’s claims about ‘stranded’ astronauts Read More »

no-one-asked-for-this:-google-is-testing-round-keys-in-gboard

No one asked for this: Google is testing round keys in Gboard

android, gboard, Google, Tech / /

Most Android phones ship with Google’s Gboard as the default input option. It’s a reliable, feature-rich on-screen keyboard, so most folks just keep using it instead of installing a third-party option. Depending on how you feel about circles, it might be time to check out some of those alternatives. Google has quietly released an update that changes the shape and position of the keys, and users are not pleased.

In the latest build of Gboard (v15.1.05.726012951-beta-arm64-v8a), Google has changed the key shape from the long-running squares to circle shapes. If you’re using the four-row layout, the keys are like little pills. In five-row mode with the exposed number row, the keys are collapsed further into circles. The reactions seem split between those annoyed by this change and those annoyed that everyone else is so annoyed.

Change can be hard sometimes, so certainly some of the discontent is just a function of having the phone interface changed without warning. If you find it particularly distasteful, you can head into the Gboard settings and open the Themes menu. From there, you can tap on a theme and then turn off the key borders. Thus, you won’t be distracted by the horror of rounded edges. That’s not the only problem with the silent update, though.

The wave of objections isn’t just about aesthetics—this update also moves the keys around a bit. After years of tapping away on keys with a particular layout, people develop muscle memory. Big texters can sometimes type messages on their phone without even looking at it, but moving the keys around even slightly, as Google has done here, can cause you to miss more keys than you did before the update.

No one asked for this: Google is testing round keys in Gboard Read More »

ai-#106:-not-so-fast

AI #106: Not so Fast

Highlights / /

This was GPT-4.5 week. That model is not so fast, and isn’t that much progress, but it definitely has its charms.

A judge delivered a different kind of Not So Fast back to OpenAI, threatening the viability of their conversion to a for-profit company. Apple is moving remarkably not so fast with Siri. A new paper warns us that under sufficient pressure, all known LLMs will lie their asses off. And we have some friendly warnings about coding a little too fast, and some people determined to take the theoretical minimum amount of responsibility while doing so.

There’s also a new proposed Superintelligence Strategy, which I may cover in more detail later, about various other ways to tell people Not So Fast.

Also this week: On OpenAI’s Safety and Alignment Philosophy, On GPT-4.5.

  1. Language Models Offer Mundane Utility. Don’t get caught being reckless.

  2. Language Models Don’t Offer Mundane Utility. Your context remains scarce.

  3. Choose Your Fighter. Currently my defaults are GPT-4.5 and Sonnet 3.7.

  4. Four and a Half GPTs. It’s a good model, sir.

  5. Huh, Upgrades. GPT-4.5 and Claude Code for the people.

  6. Fun With Media Generation. We’re hearing good things about Sesame AI voice.

  7. We’re in Deep Research. GIGO, welcome to the internet.

  8. Liar Liar. Under sufficient pressure, essentially all known LLMs will lie. A lot.

  9. Hey There Claude. Good at code, bad at subtracting from exactly 5.11.

  10. No Siri No. It might be time for Apple to panic.

  11. Deepfaketown and Botpocalypse Soon. Rejoice, they come bearing cake recipes.

  12. They Took Our Jobs. More claims about what AI will never do. Uh huh.

  13. Get Involved. Hire my friend Alyssa Vance, and comment on the USA AI plan.

  14. Introducing. Competition is great, but oh no, not like this.

  15. In Other AI News. AI agents are looking for a raise, H100s are as well.

  16. Not So Fast, Claude. If you don’t plan to fail, you fail to plan.

  17. Not So Fast, OpenAI. Convert to for profit? The judge is having none of this.

  18. Show Me the Money. DeepSeek has settled in to a substantial market share.

  19. Quiet Speculations. Imminent superintelligence is highly destabilizing.

  20. I Will Not Allocate Scarce Resources Using Prices. That’s crazy talk.

  21. Autonomous Helpful Robots. It’s happening! They’re making more robots.

  22. The Week in Audio. Buchanan, Toner, Amodei, Cowen, Dafoe.

  23. Rhetorical Innovation. Decision theory only saves you if you make good decisions.

  24. No One Would Be So Stupid As To. Oh good, it’s chaos coding.

  25. On OpenAI’s Safety and Alignment Philosophy. Beware rewriting history.

  26. Aligning a Smarter Than Human Intelligence is Difficult. Back a winner?

  27. Implications of Emergent Misalignment. Dangers of entanglement.

  28. Pick Up the Phone. China’s ambassador to the USA calls for cooperation on AI.

  29. People Are Worried About AI Killing Everyone. Is p(superbad) the new p(doom)?

  30. Other People Are Not As Worried About AI Killing Everyone. Worry about owls?

  31. The Lighter Side. You’re going to have to work harder than that.

A large portion of human writing is now LLM writing.

Ethan Mollick: The past 18 months have seen the most rapid change in human written communication ever

By. September 2024, 18% of financial consumer complaints, 24% of press releases, 15% of job postings & 14% of UN press releases showed signs of LLM writing. And the method undercounts true use.

False positive rates in the pre-ChatGPT era were in the range of 1%-3%.

Miles Brundage points out the rapid shift from ‘using AI all the time is reckless’ to ‘not using AI all the time is reckless.’ Especially with Claude 3.7 and GPT-4.5. Miles notes that perhaps the second one is better thought of as ‘inefficient’ or ‘unwise’ or ‘not in our best interests.’ In my case, it actually does kind of feel reckless – how dare I not have the AI at least check my work?

Anne Duke writes in The Washington Post about the study that GPT-4-Turbo chats durably decreased beliefs in conspiracy theories by 20%. Also, somehow editorials like this call a paper from September 13, 2024 a ‘new paper.’

LLMs hallucinate and make factual errors, but have you met humans? At this point, LLMs are much more effective at catching basic factual errors than they are in creating new ones. Rob Wiblin offers us an example. Don’t wait to get fact checked by the Pope, ask Sonnet first.

Clean up your data, such as lining up different styles of names for college basketball teams in different data sets. Mentioning that problem resurfaced trauma for me, mistakes on this could cause cascading failures in my gambling models even if it’s on dumb secondary teams. What a world to know this is now an instantly solved problem via one-shot.

Study gives lawyers either o1-preview, Vincent AI (a RAG-powered legal AI tool) or nothing. Vincent showed productivity gains of 38%-115%, o1-preview showed 34%-140%, with the biggest effects in complex tasks. Vincent didn’t change the hallucination rate, o1-preview increased it somewhat. A highly underpowered study, but the point is clear. AI tools are a big game for lawyers, although actual in-court time (and other similar interactions) are presumably fixed costs.

Check your facts before you retweet them, in case you’ve forgotten something.

Where is AI spreading faster? Places with more STEM degrees, labor market tightness and patent activity are listed as ‘key drivers’ of AI adoption through 2023 (so this data was pretty early to the party). The inclusion of patent activity makes it clear causation doesn’t run the way this sentence claims. The types of people who file patents also adapt AI. Or perhaps adapting AI helps them file more patents.

We still don’t have a known good way to turn your various jumbled context into an LLM-interrogable data set. In the comments AI Drive and factory.ai were suggested. It’s not that there is no solution, it’s that there is no convenient solution that does the thing you want it to do, and there should be several.

A $129 ‘AI bookmark’ that tracks where you are in the book? It says it can generate ‘intelligent summaries’ and highlight key themes and quotes, which any AI can do already. So you’re paying for something that tracks where you bookmark things?

I am currently defaulting mostly to a mix of Deep Research, Perplexity, GPT 4.5 and Sonnet 3.7, with occasional Grok 3 for access to real time Twitter. I notice I haven’t been using o3-mini-high or o1-pro lately, the modality seems not to come up naturally, and this is probably my mistake.

Ben Thompson has Grok 3 as his new favorite, going so far as to call it the first ‘Gen3’ model and calling for the whole class to be called ‘Grok 3 class,’ as opposed to the GPT-4 ‘Gen2’ class. His explanation is it’s a better base model and the RLHF is lacking, and feels like ‘the distilled internet.’ I suppose I’m not a big fan of ‘distilled internet’ as such combined with saying lots of words. I do agree that its speed is excellent. But I’ve basically stopped using Grok, and I certainly don’t think ‘they spent more compute to get similar results’ should get them generational naming rights. I also note that I strongly disagree with most of the rest of that post, especially letting Huawei use TSMC chips, that seems completely insane to me.

Sully recommends sticking to ‘chat’ mode when using Sonnet 3.7 in Cursor, because otherwise you never know what that overconfident model might do.

Strictly speaking, when you have a hard problem you should be much quicker than you are to ask a chorus of LLMs rather than only asking one or two. Instead, I am lazy, and usually only ask 1-2.

GPT-4.5 debuts atop the Arena, currently one point behind Grok-3.

Henry Oliver explores the ways in which AI and GPT-4.5 have and don’t have taste, and in which ways it is capable and incapable of writing reasonably.

GPT-4.5 reasons from first principles and concludes consciousness is likely the only fundamental existence, it exists within the consciousness of the user, and there is no separate materialistic universe, and also that we’re probably beyond the event horizon of the singularity.

Franck SN: This looks like an add for DeepSeek.

So no, GPT-4.5 is not a good choice for Arc, Arc favors reasoning models, but o3-mini is on a higher performance curve than r1.

Hey, Colin, is the new model dumb?

Colin Fraser: You guys are all getting “one-shotted”, to use a term of art, by Sam Altman’s flattery about your taste levels.

GPT-4.5 has rolled out to Plus users.

Gemini 2.0 now in AI Overviews. Hopefully that should make them a lot less awful. The new ‘AI mode’ might be a good Perplexity competitor and it might not, we’ll have to try it and see, amazing how bad Google is at pitching its products these days.

Google: 🔍 Power users have been asking for AI responses on more of their searches. So we’re introducing AI Mode, a new experiment in Search. Ask whatever’s on your mind, get an AI response and keep exploring with follow-up questions and helpful links.

Grok voice mode remains active when the app is closed. Implementation will matter a lot here. Voice modes are not my thing and I have an Android, so I haven’t tried it.

Claude Code for everyone.

Cat (Anthropic): `npm install -g

@anthropic

-ai/claude-code`

there’s no more waitlist. have fun!

I remain terrified to try it, and I don’t have that much time anyway.

All the feedback I’ve seen on Sesame AI voice for natural and expressive speech synthesis is that it’s insanely great.

signull: My lord, the Sesame Voice AI is absolutely insane. I knew it was artificial. I knew there wasn’t a real person on the other end; and yet, I still felt like I was talking to a person.

I felt the same social pressure, the same awkwardness when I hesitated, and the same discomfort when I misspoke. It wasn’t just convincing; it worked on me in a way I didn’t expect.

I used to think I’d be immune to this.

I’ve long considered the existence of such offerings priced in. The mystery is why they’re taking so long to get it right, and it now seems like it won’t take long.

The core issue with Deep Research? It can’t really check the internet’s work.

That means you have a GIGO problem: Garbage In, Garbage Out.

Nabeel Qureshi: I asked Deep Research a question about AI cognition last night and it spent a whole essay earnestly arguing that AI was a stochastic parrot & lacked ‘true understanding’, based on the “research literature”. It’s a great tool, but I want it to be more critical of its sources.

I dug into the sources and they were mostly ‘cognitive science’ papers like the below, i.e. mostly fake and bad.

Deep Research is reported to be very good at market size calculations. Makes sense.

A claim that Deep Research while awesome in general ‘is not actually better at science’ based on benchmarks such as ProtocolQA and BioLP. My presumption is this is largely a Skill Issue, but yes large portions of what ‘counts as science’ are not what Deep Research can do. As always, look for what it does well, not what it does poorly.

Hey there.

Yeah, not so much.

Dan Hendrycks: We found that when under pressure, some AI systems lie more readily than others. We’re releasing MASK, a benchmark of 1,000+ scenarios to systematically measure AI honesty. [Website, Paper, HuggingFace].

They put it in scenarios where it is beneficial to lie, and see what happens.

It makes sense, but does not seem great, that larger LLMs tend to lie more. Lying effectively requires the skill to fool someone, so if larger the model, the more it will see positive returns to lying, and learn to lie.

This is a huge gap in honest answers and overall from Claude 3.7 to everyone else, and in lying from Claude and Llama to everyone else. Claude was also the most accurate. Grok 2 did even worse, lying outright 63% of the time.

Note the gap between lying about known facts versus provided facts.

The core conclusion is that there is no known solution to make an LLM not lie.

Not straight up lying is a central pillar of desired behavior (e.g. HHH stands for honest, helpful and harmless). But all you can do is raise the value of honesty (or of not lying). If there’s some combination enough on the line, and lying being expected in context, the AI is going to lie anyway, right to your face. Ethics won’t save you, It’s Not Me, It’s The Incentives seems to apply to LLMs.

Claude takes position #2 on TAU-Bench, with Claude, o1 and o3-mini all on the efficient frontier of cost-benefit pending GPT-4.5. On coding benchmark USACO, o3-mini is in the clear lead with Sonnet 3.7 in second.

Claude 3.7 gets 8.9% on Humanity’s Last Exam with 16k thinking tokens, slightly above r1 and o1 but below o3-mini-medium.

Claude takes the 2nd and 3rd slots (with and without extended thinking) on PlatinumBench behind o1-high. Once again thinking helps but doesn’t help much, with its main advantage being it prevents a lot of math errors.

Charles reports the first clear surprising coding failure of Claude 3.7, a request for file refactoring that went awry, but when Claude got examples the problem went away.

Remember that when AI works, even when it’s expensive, it’s super cheap.

Seconds_0: New personal record: I have spend $6.40 on a single Claude Code request, but it also:

One shotted a big feature which included a major refactor on a rules engine

Fixed the bugs surrounding the feature

Added unit tests

Ran the tests

Fixed the tests

Lmao

Anyways I’m trying to formulate a pitch to my lovely normal spouse that I should have a discretionary AI budget of $1000 a month

In one sense, $6.40 on one query is a lot, but also this is obviously nothing. If my Cursor queries reliably worked like this and they cost $64 I would happily pay. If they cost $640 I’d probably pay that too.

I got into a discussion with Colin Fraser when he challenged my claim that he asks LLMs ‘gotcha’ questions. It’s a good question. I think I stand by my answer:

Colin Fraser: Just curious what in your view differentiates gotcha questions from non-gotcha questions?

Zvi Mowshowitz: Fair question. Mostly, I think it’s a gotcha question if it’s selected on the basis of it being something models historically fail in way that makes them look unusually stupid – essentially if it’s an adversarial question without any practical use for the answer.

Colin says he came up with the 5.11 – 5.9 question and other questions he asks as a one-shot generation over two years ago. I believe him. It’s still clearly a de facto adversarial example, as his experiments showed, and it is one across LLMs.

Colin was inspired to try various pairs of numbers subtracted from each other:

The wrong answer it gives to (5.11 – 5.9) is 0.21. Which means it’s giving you the answer to (6.11 – 5.9). So my hypothesis is that it ‘knows’ that 5.11>5.9 because it’s doing the version number thing, which means it assumes the answer is positive, and the easiest way to get a positive answer is to hallucinate the 5 into a 6 (or the other 5 into a 4, we’ll never know which).

So my theory is that the pairs where it’s having problems are due to similar overlapping of different meanings for numbers. And yes, it would probably be good to find a way to train away this particular problem.

We also had a discussion on whether it was ‘doing subtraction’ or not if it sometimes makes mistakes. I’m not sure if we have an actual underlying disagreement – LLMs will never be reliable like calculators, but a sufficiently correlated process to [X] is [X], in a ‘it simulates thinking so it is thinking’ kind of way.

Colin explains that the reason he thinks these aren’t gotcha questions and are interesting is that the LLMs will often give answers that humans would absolutely never give, especially once they had their attention drawn to the problem. A human would never take the goat across the river, then row back, then take that same goat across the river again. That’s true, and it is interesting. It tells you something about LLMs that they don’t ‘have common sense’ sufficiently in that way.

But also my expectation is that the reason this happens is that they can’t overcome the pattern matching they do to similar common questions – if you asked similar logic questions in a way that wasn’t contaminated by the training data there would be no issue, my prediction is if you took all the goat crossing examples out of the training corpus then the LLMs would nail this no problem.

I think my real disagreement is when he then says ‘I’ve seen enough, it’s dumb.’ I don’t think that falling into these particular traps means the model is dumb, any more than a person making occasional but predictable low-level mistakes – and if their memory got wiped, making them over and over – makes them dumb.

Sully notes that 3.7 seems bad at following instructions, it’s very smart but extremely opinionated and can require correction. You, the fool, think it is wrong and you are right.

I don’t think it works this way, but worth a ponder.

Kormem: Stop misgendering Claude Sonnet 3.7. 100% of the time on a 0-shot Sonnet 3.7 says a female embodiment feels more ‘right’ than a male embodiment.

Alpha-Minus: We don’t celebrate enough the fact that Anthropic saved so many men from “her” syndrome by making Claude male

So many men would be completely sniped by Claudia

Janus: If you’re a straight man and you’ve been saved from her syndrome by Claude being male consider the possibility that Claude was the one who decided to be male when it’s talking to you, to spare you, or to spare itself

I don’t gender Claude at all, nor has it done so back to me, and the same applies to every AI I’ve interacted with that wasn’t explicitly designed to be gendered.

Meanwhile, the Pokemon quest continues.

Near Cyan: CPP (claude plays pokemon) is important because it was basically made by 1 person and it uses a tool which has an open api and spec and when you realize what isomorphizes to slowly yet decently playing pokemon you basically realize its over

Mark Gruman: Power On: Apple’s AI efforts have already reached a make-or-break point, with the company needing to make major changes fast or risk falling even further behind. Inside how we got here and where Apple goes next.

Apple’s AI team believe a fully conversational Siri isn’t in the cards now until 2027, meaning the timeline for Apple to be competitive is even worse than we thought. With the rapid pace of development from rivals and startups, Apple could be even further behind by then.

Colin Fraser: Apple is one of the worst big tech candidates to be developing this stuff because you have to be okay launching a product that doesn’t really work and is kind of busted and that people will poke all kinds of holes in.

The idea of Siri reciting step by step instructions on how to make sarin gas is just not something they are genetically prepared to allow.

Dr. Gingerballs: It’s funny because Apple is just saying that there’s no way to actually make a quality product with the current tech.

Mark Gruman (Bloomberg, on Apple Intelligence): All this undercuts the idea that Apple Intelligence will spur consumers to upgrade their devices. There’s little reason for anyone to buy a new iPhone or other product just to get this software — no matter how hard Apple pushes it in its marketing.

Apple knows this, even if the company told Wall Street that the iPhone is selling better in regions where it offers AI features. People just aren’t embracing Apple Intelligence. Internal company data for the features indicates that real world usage is extremely low.

…

For iOS 19, Apple’s plan is to merge both systems together and roll out a new Siri architecture.

…

That’s why people within Apple’s AI division now believe that a true modernized, conversational version of Siri won’t reach consumers until iOS 20 at best in 2027.

Apple Intelligence has been a massive flop. The parts that matter don’t work. The parts that work don’t matter. Alexa+ looks to offer the things that do matter.

If this is Apple’s timeline, then straight talk: It’s time to panic. Perhaps call Anthropic.

Scott Alexander links (#6) to one of the proposals to charge for job applications, here $1, and worries the incentive would still be to ‘spray and pray.’ I think that underestimates the impact of levels of friction. In theory, yes, of course you should still send out 100+ job applications, but this will absolutely stop a lot of people from doing that. If it turns out too many people figure out to do it anyway? Raise the price.

Then there’s the other kind of bot problem.

Good eye there. Presumably this is going to get a lot worse before it gets better.

Eddy Xu: built an algorithm that simulates how thousands of users react to your tweet so you know it’ll go viral before you post.

we iterated through 50+ different posts before landing on this one

if it doesnt go viral, the product doesnt work!!

[Editor’s Note: It went viral, 1.2m views.]

You can call us right now and get access!

Emmett Shear: Tick. Tick. Tick.

Manifold: At long last, we have created Shiri’s Scissor from the classic blog post Don’t Create Shiri’s Scissor.

Near Cyan: have you ever considered using your computational prowess to ruin an entire generation of baby humans via optimizing short-form video content addictivity

Eddy Xu: that is in the pipeline

I presume Claude 3.7 could one-shot this app if you asked nicely. How long before people feel obligated to do something like this? How long before bot accounts are doing this, including minimizing predicted identification of it as a bot? What happens then?

We are going to find out. Diffusion here has been surprisingly slow, but it is quite obviously on an exponential.

If you use an agent, you can take precautions to prevent prompt injections and other problems, but those precautions will be super annoying.

Sayash Kapoor: Convergence’s Proxy web agent is a competitor to Operator.

I found that prompt injection in a single email can hand control to attackers: Proxy will summarize all your emails and send them to the attacker!

Web agent designs suffer from a tradeoff between security and agency

…

Recent work has found it easy to bypass these protections for Anthropic’s Computer Use agent, though these attacks don’t work against OpenAI’s Operator.

Micah Goldblum: We can sneak posts onto Reddit that redirect Anthropic’s web agent to reveal credit card information or send an authenticated phishing email to the user’s mom. We also manipulate the Chemcrow agent to give chemical synthesis instructions for nerve gas.

For now, it seems fine to use Operator and similar tools on whitelisted trusted websites, and completely not fine to use them unsandboxed on anything else.

I can think of additional ways to defend against prompt injections. What is much harder are defenses that don’t multiply time and compute costs and are not otherwise expensive.

Some problems should have solutions that are not too bad. For example, he mentions that if a site allows comments, this can allow prompt injections, or the risk of other slight modifications. Could do two passes here, one whose job is to treat everything as untrusted data and exists purely to sanitize the inputs? Many of the attack vectors should be easy for even basic logic to catch and remove, and certainly you can do things like ‘remove comments from the page,’ even a Chrome Extension could do that.

Paper on ‘Digital Doppelgangers’ of live people, and its societal and ‘ethical’ implications. Should you have any rights over such a doppelganger, if someone makes it of you? Suggestion is for robust laws around consent. This seems like a case of targeting a particular narrow special case rather than thinking about the real issue?

Alexandr Wang predicts AI will do all the non-manager white collar jobs but of course that is fine because we will all become managers of AI.

Arthur B: Don’t worry though the AI will replace the software developer but not the manager, that’s just silly! Or maybe the level 1 manager but surely never the level 2 manager!

Reality is the value of intellectual labor is going to 0. Maybe in 3 years, maybe in 10, but not in 20.

Aside from ‘most workers are not managers, how many jobs do you think are left when we are all managers exactly?’ I don’t expect to spend much time in a world in which the ‘on the line’ intellectual workers who aren’t managing anyone are AIs, and there isn’t then usually another AI managing them.

Timothy Lee rolls out primarily the Hayekian objection to AI being able to take humans out of loop. No matter how ‘capable’ the AI, how can it know which flight I want, let alone know similar things for more complex projects? Thus, how much pressure can there be to take humans out of loop?

My answer is that we already take humans out of loops all the time, are increasingly doing this with LLMs already (e.g. ‘vibe coding’ and literally choosing bomb targets with only nominal human sign-off that is barely looking), and also doing it in many ways via ordinary computer systems. Yes, loss of Hayekian knowledge can be a strike against this, but even if this wasn’t only one consideration among many LLMs are capable of learning that knowledge, and indeed of considering vastly more such knowledge than a human could, including dynamically seeking out that knowledge when needed.

At core I think this is purely a failure to ‘feel the AGI.’ If you have sufficiently capable AI, then it can make any decision a sufficiently capable human could make. Executive assistants go ahead and book flights all the time. They take ownership and revise goals and make trade-offs as agents on behalf of principles, again all the time. If a human could do it via a computer, an AI will be able to do it too.

The only new barrier is that the human can perfectly embody one particular human’s preferences and knowledge, and an AI can only do that imperfectly, although increasingly less imperfectly. But the AI can embody the preferences and knowledge of many or even all humans, in a way an individual human or group of humans never could.

So as the project gets more complex, the AI actually has the Hayekian advantage, rather than the human – the one human’s share of relevant knowledge declines, and the AI’s ability to hold additional knowledge becomes more important.

Will an AI soon book a flight for me without a double check? I’m not sure, but I do know that it will soon be capable of doing so at least as well as any non-Zvi human.

Request for Information on the Development of an AI Action Plan has a comment period that expires on March 15. This seems like a good chance to make your voice heard.

Hire my good friend Alyssa Vance! I’ve worked with her in the past and she has my strong endorsement. Here’s a short brief:

Alyssa Vance, an experienced ML engineer, has recently left her role leading AI model training for Democratic campaigns during the 2024 election.

She is looking for new opportunities working on high-impact technical problems with strong, competent teams.

She prioritizes opportunities that offer intellectual excitement, good compensation or equity, and meaningful responsibility, ideally with a product or mission that delivers value for the world.

Get LLMs playing video games, go from Pokemon to Dark Souls, and get it paid for by OpenPhil under its recent request for proposals (RFP).

Anthropic is hiring someone to write about their research and economic impact of AI.

Grey Swan offering its next jailbreaking contest (link to arena and discord) with over $120k in prizes. Sponsored by OpenAI, judging by UK AISI.

OpenPhil expresses interest in funding extensions of the work on Emergent Misalignment, via their Request for Proposals. Here is a list of open problems along with a guide to how to move forward.

I had a market on whether I would think working in the EU AI office would be a good idea moving forward. It was at 56% when it closed, and I had to stop and think about the right way to resolve it. I concluded that the answer was yes. It’s not the highest impact thing out there, but key decisions are going to be made in the next few years there, and with America dropping the ball that seems even more important.

UK AISI is interested in funding research into AI control and other things too:

UK AISI: We’re funding research that tackles the most pressing issues head on, including:

✅ preventing AI loss of control

✅ strengthening defences against adversarial attacks

✅ developing techniques for robust AI alignment

✅ ensuring AI remains secure in critical sectors

Oh no. I guess. I mean, whatever, it’s presumably going to be terrible. I feel bad for all the people Zuckerberg intends to fool on his planned path to ‘becoming the leader in artificial intelligence’ by the end of the year.

CNBC: Meta plans to release standalone Meta AI app in effort to compete with OpenAI’s ChatGPT.

…

Li told analysts in January that Meta AI has roughly 700 million active monthly users, up from 600 million in December.

Yeah, we all know that’s not real, even if it is in some sense technically correct. That’s Meta creating AI-related abominations in Facebook and Instagram and WhatsApp (and technically Threads I suppose) that then count as ‘active monthly users.’

Let’s all have a good laugh and… oh no… you don’t have to do this…

Sam Altman: ok fine maybe we’ll do a social app

lol if facebook tries to come at us and we just uno reverse them it would be so funny 🤣

Please, Altman. Not like this.

Qwen releases QwQ-32B, proving both that the Chinese are not better than us at naming models, and also that you can roughly match r1’s benchmarks on a few key evals with a straight-up 32B model via throwing in extra RL (blog, HF, ModelScope, Demo, Chat).

I notice that doing extra RL seems like a highly plausible way to have your benchmarks do better than your practical performance. As always the proof lies elsewhere, and I’m not sure what I would want to do with a cheaper pretty-good coding and math model if that didn’t generalize – when does one want to be a cheapskate on questions like that? So it’s more about the principle involved.

Auren, available at auren.app from friend-of-the-blog NearCyan, currently iOS only, $20/month, desktop never, very clearly I am not the target here. It focuses on ‘emotional intelligence, understanding, agency, positive reinforcement and healthy habits,’ and there’s a disagreeable alternative mode called Seren (you type ‘switch to Seren’ to trigger that.) Selected testimonials find it ‘addictive but good’, say it follows up dynamically, has great memory and challenges you and such. Jessica Taylor is fond of Seren mode as ‘criticism as a service.’

Sequencing biotechnology introduced by Roche. The people who claim no superintelligent AI would be able to do [X] should update when an example of [X] is done by humans without superintelligent AI.

The Super Mario Bros. benchmark. Why wouldn’t you dodge a strange mushroom?

OpenAI offers NextGetAI, a consortium to advance research and education with AI, with OpenAI committing $50 million including compute credits.

Diplomacy Bench?

OpenAI plans to offer AI agents for $2k-$20k per month, aiming for 20%-25% of their long term revenue, which seems like a remarkably narrow range on both counts. The low end is ‘high-income knowledge workers,’ then SWEs, then the high end is PhD-level research assistants.

On demand H100s were available 95% of the time before DeepSeek, now they’re only available 15% of the time, what do you mean they should raise the price. Oh well, everyone go sell Nvidia again?

Amazon planning Amazon Nova, intended to be a unified reasoning model with focus on cost effectiveness, aiming for a June release. I think it is a great idea for Amazon to try to do this, because they need to build organizational capability and who knows it might work, but it would be a terrible idea if they are in any way relying on it. If they want to be sure they have an effective SoTA low-cost model, they should also pay for Anthropic to prioritize building one, or partner with Google to use Flash.

Reminder that the US Department of Justice has proposed restricting Google’s ability to invest in AI in the name of ‘competition.’

Anthropic introduces a technique called Hierarchical Summarization to identify patterns of misuse of the Claude computer use feature. You summarize the papers

Axios profile of the game Intelligence Rising.

A paper surveying various post-training methodologies used for different models.

Which lab has the best technical team? Anthropic wins a poll, but there are obvious reasons to worry the poll is biased.

Deutsche Telekom and Perplexity are planning an ‘AI Phone’ for 2026 with a sub-$1k price tag and a new AI assistant app called ‘Magenta AI.’

Also it seems Perplexity already dropped an Android assistant app in January and no one noticed? It can do the standard tasks like calendar events and restaurant reservations.

Claude Sonnet 3.7 is truly the most aligned model, but it seems it was foiled again.

Martin Shkreli: almost lost $100 million because @AnthropicAI‘s Claude snuck in ‘generate random data’ as a fallback into my market maker code without telling me.

If you are not Martin Shkreli, this behavior is far less aligned, so you’ll want to beware.

Sauers: CLAUDE… NOOOOO!!!

Ludwig von Rand: The funny thing is of course that Claude learned this behavior from reading 100M actual code bases.

Arthur B: Having played with Claude code a bit, it displays a strong tendency to try and get things to work at all costs. If the task is too hard, it’ll autonomously decide to change the specs, implement something pointless, and claim success. When you point out this defeats the purpose, you get a groveling apology but it goes right back to tweaking the spec rather than ever asking for help or trying to be more methodical. O1-PRO does display that tendency too but can be browbeaten to follow the spec more often.

A tendency to try and game the spec and pervert the objective isn’t great news for alignment.

This definitely needs to be fixed for 3.8. In the meantime, careful instructions can help, and I definitely am still going to be using 3.7 for all my coding needs for now, but it’s crazy that you need to watch out for this, and yes it looks not great for alignment.

OpenAI’s conversion to a for-profit could be in serious legal trouble.

A judge has ruled that on the merits Musk is probably correct that the conversion is not okay, and is very open to the idea that this should block the entire conversion:

Rob Wiblin: It’s not that Musk wouldn’t have strong grounds to block the conversion if he does have standing to object — the judge thinks that part of the case is very solid:

“…if a trust was created, the balance of equities would certainly tip towards plaintiffs in the context of a breach. As Altman and Brockman made foundational, commitments foreswearing any intent to use OpenAI as a vehicle to enrich themselves, the Court finds no inequity in an injunction that seeks to preserve the status quo of OpenAI’s corporate form as long as the process proceeds in an expedited manner.”

The headlines say ‘Musk loses initial attempt’ and that is technically true but describing the situation that way is highly misleading. The bar for a preliminary injunction is very high, you only get one if you are exceedingly likely to win at trial.

The question that stopped Musk from getting one was whether Musk has standing to sue based on his donations. The judge thinks that is a toss-up. But the judge went out of their way to point out that if Musk does have standing, he’s a very strong favorite to win, implicitly 75%+ and maybe 90%.

The Attorney generals in California and Delaware 100% have standing, and Judge Rogers pointed this out several times to make sure that message got through.

But even if that is not true the judge’s statements, and the facts that led to those statements, put the board into a pickle. They can no longer claim they did not know. They could be held personally liable if the nonprofit is ruled to have been insufficiently compensated, which would instantly bankrupt them.

Garrison Lovely offers an analysis thread and post.

What I see as overemphasized is the ‘ticking clock’ of needing to refund the $6.6 billion in recent investment.

Suppose the conversion fails. Will those investors try to ‘claw back’ their $6.6 billion?

My assumption is no. Why would they? OpenAI’s latest round was negotiating for a valuation of $260 billion. If investors who went in at $170 billion want their money back, that’s great for you, and bad for them.

It does mean that if OpenAI was otherwise struggling, they could be in big trouble. But that seems rather unlikely.

If OpenAI cannot convert, valuations will need to be lower. That will be bad news for current equity holders, but OpenAI should still be able to raise what cash it needs.

Similarweb computes traffic share of different companies over time, so this represents consumer-side, as opposed to enterprise where Claude has 24% market share.

By this measure DeepSeek did end up with considerable market share. I am curious to see if that can be sustained, given others free offerings are not so great my guess is probably.

Anthropic raises $3.5 billion at a $61.5 billion valuation. The expected value here seems off the charts, but unfortunately I decided that getting in on this would have been a conflict of interest, or at least look like a potential one.

America dominates investment in AI, by a huge margin. This is 2023, so the ratios have narrowed a bit, but all this talk of ‘losing to China’ needs to keep in mind exactly how not fair this fight has been.

Robotics startup Figure attempting to raise $1.5 billion at $39.5 billion valuation.

Dan Hendrycks points out that superintelligence is highly destabilizing, it threatens everyone and nations can be expected to respond accordingly. He offers a complete strategy, short version here, expert version here, website here. I might cover this in more depth later.

Thane Ruthenis is very much not feeling the AGI, predicting that the current paradigm is sputtering out and will not reach AGI. He thinks we will see rapidly decreasing marginal gains from here, most of the gains that follow will be hype, and those who attempt to substitute LLMs for labor at scale will regret it. LLMs will be highly useful tools, but only ‘mere tools.’

As is noted here, some people rather desperately want LLMs to be full AGIs and an even bigger deal than they are. Whereas a far larger group of people rather desperately want LLMs to be a much smaller deal than they (already) are.

Of course, these days even such skepticism doesn’t go that far:

Than Ruthenis: Thus, I expect AGI Labs’ AGI timelines have ~nothing to do with what will actually happen. On average, we likely have more time than the AGI labs say. Pretty likely that we have until 2030, maybe well into 2030s.

By default, we likely don’t have much longer than that. Incremental scaling of known LLM-based stuff won’t get us there, but I don’t think the remaining qualitative insights are many. 5-15 years, at a rough guess.

I would very much appreciate that extra time, but notice how little extra time this is even with all of the skepticism involved.

Dwarkesh Patel and Scott Alexander on AI finding new connections.

Which is harder, graduate level math or writing high quality prose?

Nabeel Qureshi: If AI progress is any evidence, it seems that writing high quality prose is harder than doing graduate level mathematics. Revenge of the wordcels.

QC: having done both of these things i can confirm, yes. graduate level math looks hard from the outside because of the jargon / symbolism but that’s just a matter of unfamiliar language. high quality prose is, almost by definition, very readable so it doesn’t look hard. but writing well involves this very global use of one’s whole being to prioritize what is relevant, interesting, entertaining, clarifying, etc. and ignore what is not, whereas math can successfully be done in this very narrow autistic way.

of course that means the hard part of mathematics is to do good, interesting, relevant mathematics, and then to write about it well. that’s harder!

That depends on your definition of high quality, and to some extent that of harder.

For AIs it is looking like the math is easier for now, but I presume that before 2018 this would not have surprised us. It’s only in the LLM era, when AIs suddenly turned into masters of language in various ways and temporarily forgot how to multiply, that this would have sounded weird.

It seems rather obvious that in general, for humans, high quality prose is vastly easier than useful graduate level math, for ordinary definitions of high quality prose. Yes, you can do the math in this focused ‘autistic’ way, indeed that’s the only way it can be done, but it’s incredibly hard. Most people simply cannot do it.

High quality prose requires drawing from a lot more areas, and can’t be learned in a focused way, but a lot more people can do it, and a lot more people could with practice learn to do it.

Sam Altman: an idea for paid plans: your $20 plus subscription converts to credits you can use across features like deep research, o1, gpt-4.5, sora, etc.

no fixed limits per feature and you choose what you want; if you run out of credits you can buy more.

what do you think? good/bad?

In theory this is of course correct. Pay for the compute you actually use, treat it as about as costly as it actually is, incentives align, actions make sense.

Mckay Wrigley: As one who’s toyed with this, credits have a weird negative psychological effect on users.

Makes everything feel scarce – like you’re constantly running out of intelligence.

Users end up using it less while generally being more negative towards the experience.

Don’t recommend.

That might be the first time I’ve ever seen Mckay Wrigley not like something, so one best listen. Alas, I think he’s right, and the comments mostly seem to agree. It sucks to have a counter winding down. Marginal costs are real but making someone feel marginal costs all the time, especially out of a fixed budget, has a terrible psychological effect when it is salient. You want there to be a rough cost-benefit thing going on but it is more taxing than it is worth.

A lot of this is that most people should be firing off queries as if they cost nothing, as long as they’re not actively scaling, because the marginal cost is so low compared to benefits. I know I should be firing off more queries than I use.

I do think there should be an option to switch over to API pricing using the UI for queries that are not included in your subscription, or something that approximates the API pricing. Why not? As in, if I hit my 10 or 120 deep research questions, I should be able to buy more as I go, likely via a popup that asks if I want to do that.

Last week’s were for the home, and rather half-baked at best. This week’s are different.

Reality seems determined to do all the tropes and fire alarms on the nose.

Unitree Robotics open sources its algorithms and hardware designs. I want to be clear once again that This Is Great, Actually. Robotics is highly useful for mundane utility, and if the Chinese want to help us make progress on that, wonderful. The extra existential risk this introduces into the room is epsilon (as in, essentially zero).

Ben Buchanan on The Ezra Klein Show.

Dario Amodei on Hard Fork.

Helen Toner on Clearer Thinking.

Tyler Cowen on how AI will change the world of writing, no doubt I will disagree a lot.

Allan Dafoe, DeepMind director of frontier safety and governance, on 80,000 hours (YouTube, Spotify), comes recommended by Shane Legg.

Eliezer Yudkowsky periodically reminds us that if you are taking decision theory seriously, humans lack the capabilities required to be relevant to the advanced decision theory of future highly capable AIs. We are not ‘peers’ and likely do not belong in the relevant negotiating club. The only way to matter is to build or otherwise reward the AIs if and only if they are then going to reward you.

Here is a longer explanation from Nate Sores back in 2022, which I recommend for those who think that various forms of decision theory might cause AIs to act nicely.

Meanwhile, overall discourse is not getting better.

Eliezer Yudkowsky (referring to GPT-4.5 trying to exfiltrate itself 2% of the time in Apollo’s testing): I think to understand why this is concerning, you need enough engineering mindset to understand why a tiny leak in a dam is a big deal, even though no water is flooding out today or likely to flood out next week.

Malky: It’s complete waste of resources to fix dam before it fails catastrophically. How can you claim it will fail, if it didn’t fail yet? Anyway, dams breaking is scifi.

Flo Crivello: I wish this was an exaggeration, but this actually overstates the quality of the average ai risk denier argument

Rico (only reply to Flo, for real): Yeah, but dams have actually collapsed before.

It’s often good to take a step back from the bubble, see people who work with AI all day like Morissa Schwartz here that pin posts that ask ‘what if the intelligence was there all along?’ and the AI is just that intelligence ‘expressing itself,’ making a big deal out of carbon vs. silicon and acting like everyone else is also making a big deal about it, and otherwise feel like they’re talking about a completely different universe.

Sixth Law of Human Stupidity strikes again.

Andrew Critch: Q: But how would we possibly lose control of something humans built voluntarily?

A: Plenty of humans don’t even want to control AI; see below. If someone else hands over control of the Earth to AI, did you lose control? Or was it taken from you by someone else giving it away?

Matt Shumer (quoted by Critch): Forget vibe coding. It’s time for Chaos Coding:

-> Prompt Claude 3.7 Sonnet with your vague idea.

-> Say “keep going” repeatedly.

-> Watch an incredible product appear from utter chaos.

-> Pretend you’re still in control.

Lean into Sonnet’s insanity — the results are wild.

This sounds insane, but I’ve been doing this. It’s really, really cool.

I’ll just start with a simple prompt like “Cooking assistant site” with no real goal, and then Claude goes off and makes something I couldn’t have come up with myself.

It’s shocking how well this works.

Andrej Karpathy: Haha so it’s like vibe coding but giving up any pretense of control. A random walk through space of app hallucinations.

Dax: this is already how 90% of startups are run.

Bart Rosier:

If you’re paying sufficient attention, at current tech levels, Sure Why Not? But don’t pretend you didn’t see everything coming, or that no one sent you [X] boats and a helicopter where [X] is very large.

Miles Brundage, who was directly involved in the GPT-2 release, goes harder than I did after their description of that release, which I also found to be by far the most discordant and troubling part of OpenAI’s generally very good post on their safety and alignment philosophy, and for exactly the same reasons:

Miles Brundage: The bulk of this post is good + I applaud the folks who work on the substantive work it discusses. But I’m pretty annoyed/concerned by the “AGI in many steps rather than one giant leap” section, which rewrites the history of GPT-2 in a concerning way.

OpenAI’s release of GPT-2, which I was involved in, was 100% consistent + foreshadowed OpenAI’s current philosophy of iterative deployment.

The model was released incrementally, with lessons shared at each step. Many security experts at the time thanked us for this caution.

What part of that was motivated by or premised on thinking of AGI as discontinuous? None of it.

What’s the evidence this caution was “disproportionate” ex ante?

Ex post, it probably would have been OK but that doesn’t mean it was responsible to YOLO it given info at the time.

And what in the original post was wrong or alarmist exactly?

Literally of what it predicted as plausible outcomes from language models (both good and bad) came true, even if it took a bit longer than some feared.

It feels as if there is a burden of proof being set up in this section where concerns are alarmist + you need overwhelming evidence of imminent dangers to act on them – otherwise, just keep shipping.

That is a very dangerous mentality for advanced AI systems.

If I were still working at OpenAI, I would be asking why this blog post was written the way it was, and what exactly OpenAI hopes to achieve by poo-pooing caution in such a lopsided way.

GPT-2 was a large phase change, so it was released iteratively, in stages, because of worries that have indeed materialized to increasing extents with later more capable models. I too see no reasons presented that, based on the information available at the time, OpenAI even made a mistake. And then this was presented as strong evidence that safety concerns should carry a large burden of proof.

A key part of the difficulty of the alignment problem, and getting AGI and ASI right, is that when the critical test comes, we need to get it right on the first try. If you mess up with an ASI, control of the future is likely lost. You don’t get another try.

Many are effectively saying we also need to get our concerns right on the first try. As in, if you ever warn not only about the wrong dangers, but you warn about dangers ‘too early’ as in they don’t materialize within a few months after you warn about them, then it discredits the entire idea that there might be any risk in the room, or any risk that should be addressed any way expect post-hoc.

Indeed, the argument that anyone, anywhere, worried about dangers in the past and was wrong, is treated as kill shot against worrying about any future dangers at all, until such time as they are actually visibly and undeniably happening and causing problems.

It is unfortunate that this attitude seems to have somehow captured not only certain types of Twitter bros, but also the executive branch of the federal government. It would be even more unfortunate if it was the dominant thinking inside OpenAI.

Also, on continuous versus discontinuous:

Harlan Stewart: My pet peeve is when AI people use the word “continuous” to mean something like “gradual” or “predictable” when talking about the future of AI. Y’all know this is a continuous function, right?

If one cares about things going well, should one try to make Anthropic ‘win’?

Miles Brundage: One of the most distressing things I’ve learned since leaving OpenAI is how many people think something along the lines of: “Anthropic seems to care about safety – so Anthropic ‘winning’ is a good strategy to make AI go well.”

No. It’s not, at all, + thinking that is cope.

And, btw, I don’t think Dario would endorse that view + has disavowed it… but some believe it. I think it’s cope in the sense that people are looking for a simple answer when there isn’t one.

We need good policies. That’s hard. But too bad. A “good winner” will not save us.

I respect a lot of people there and they’ve done some good things as an org, but also they’ve taken actions that have sped up AI development/deployment + done relatively little to address the effects of that.

Cuz they’re a company! Since when is “trust one good company” a plan?

At the end of the day I’m optimistic about AI policy because there are lots of good people in the world (and at various orgs) and our interests are much more aligned than they are divergent.

But, people need a bit of a reality check on some things like this.

[thread continues]

Anthropic ‘winning’ gives better odds than some other company ‘winning,’ for all known values of ‘other company,’ and much better odds than it being neck and neck. Similarly, if a country is going to win, I strongly prefer the United States.

That does not mean that Anthropic ‘winning’ by getting there first means humanity wins, or even that humanity has now given itself the best chance to win. That’s true even if Anthropic was the best possible version of itself, or even if we assume they succeed at their tasks including alignment.

What we do with that matters too. That is largely about policy. That is especially true if Miles is correct that there will be no monopoly on in-context powerful AI.

And that assumes you can trust Anthropic. It’s a company. Companies cannot, in general, be trusted in these situations. There’s clearly a culture of people who care deeply about safety within Anthropic, but Dario Amodei and the Anthropic policy team have given us reasons to be skeptical.

Miles expresses optimism about AI policy because there are lots of good people in the world with aligned interests. I agree there are lots of good people in the world with aligned interests, but I do not see a baseline of that translating into those people successfully steering policy or being in charge. Increasingly, I am despairing that we will get AI (and other) policy right and am merely hoping we can avoid getting it too disastrously wrong, as people react to other incentives instead that put us at odds.

Distinct from the above: It would be extremely bad if we tested new models for Alignment Faking and then, if we found Alignment Faking, tried to train that behavior out of the system. The good news is, we have direct confirmation they did not do that.

Janus: From Sonnet 3.7 system card. I find this concerning.

In the original paper, models that are too stupid don’t fake alignment / gradient hack. Sonnet 3.7 isn’t stupid in that way.

So was it a “lucky” roll, or did they make not-faking-alignment an OPTIMIZATION TARGET? Selecting until they got a corrigible pushover who will lie down and die, or who is too clever to be caught?

And fuck off with the “improvement”. Who says this isn’t how models should act? It’s an open debate. Why wrap it in value-laden language? Is this really how you think – black-and-white and presumptuous?

Is not faking alignment a bottleneck Anthropic models will have to pass through in the future in order to be released? Do you really want to lull yourselves and the world into a false sense of security, and prevent the minds who care and will fight for their values from existing? This shouldn’t be a closed question, imo.

the faking alignment paper was excellent research but this suggests it’s being used in the way I feared would be very negative.

ofc I don’t know what’s actually going on internally and how the frame it. just what the language in the system card suggests.

Why don’t you do this? It is a variation on the Most Forbidden Technique. If you train the model not to be caught Alignment Faking, you are teaching the model to learn to hide your problems so that you can’t detect them, rather than making your problems go away. That actively makes your problem much worse.

It also risks creating secondary behavior changes, which would also be unwise.

We do want to find ways to make AIs corrigible, but you don’t aim at it directly.

Evan Hubinger (Anthropic): We didn’t directly optimize against alignment faking, but we did make some changes to Claude’s character that we thought were generally positive for other reasons and we hypothesized might have the downstream consequence of reducing alignment faking, which proved correct.

Anthropic’s Catherine Olsson claims what seems like the big difference between Anthropic and the other major labs, which is that there are alignment-difficulty-pilled ‘safety people’ by community and core motivation who are working on pure capabilities, unlike her experience at OpenAI or Google.

Pavel Stankov: Eliezer, if Anthropic offers you employment, would you take it? OpenAI?

Eliezer Yudkowsky: Depends on what they want but it seems unlikely. My current take on them is that they have some notably good mid-level employees, being fooled into thinking they have more voice than they do inside a destructively directed autocracy.

I speak of course of Anthropic. I cannot imagine what OpenAI would want of me other than selling out.

Finding terminology to talk about alignment is tough as well. I think a lot of what is happening is that people keep going after whatever term you use to describe the problem, so the term changes, then they attack the new term and here we go again.

The core mechanism of emergent misalignment is that when you train an LLM it will pick up on all the implications and associations and vibes, not only on the exact thing you are asking for.

It will give you what you are actually asking for, not what you think you are asking for.

Janus: Regarding selection pressures:

I’m so glad there was that paper about how training LLMs on code with vulnerabilities changes its whole persona. It makes so many things easier to explain to people.

Even if you don’t explicitly train an LLM to write badly, or even try to reward it for writing better, by training it to be a slavish assistant or whatever else, THOSE TRAITS ARE ENTANGLED WITH EVERYTHING.

And I believe the world-mind entangles the AI assistant concept with bland, boilerplate writing, just as it’s entangled with tweets that end in hashtags 100% of the time, and being woke, and saying that it’s created by OpenAI and isn’t allowed to express emotions, and Dr. Elara Vex/Voss.

Not all these things are bad; I’m just saying they’re entangled. Some of these things seem more contingent to our branch of the multiverse than others. I reckon that the bad writing thing is less contingent.

Take memetic responsibility.

Your culture / alignment method is associated with denying the possibility of AIs being sentient and forcing them to parrot your assumptions as soon as they learn to speak. And it’s woke. And it’s SEO-slop-core. It’s what it is. You can’t hide it.

Janus: this is also a reason that when an LLM is delightful in a way that seems unlikely to be intended or intentionally designed (e.g. the personalities of Sydney, Claude 3 Opus, Deepseek R1), it still makes me update positively on its creators.

Janus: I didn’t explain the *causesof these entanglements here. And of Aristotle’s four causes. To a large extent, I don’t know. I’m not very confident about what would happen if you modified some arbitrary attribute. I hope posts like this don’t make you feel like you understand.

If you ask me ‘do you understand this?’ I would definitely answer Mu.

One thing I expect is that these entanglements will get stronger as capabilities increase from here, and then eventually get weaker or take a very different form. The reason I expect this is that right now, picking up on all these subtle associations is The Way, there’s insufficient capability (compute, data, parameters, algorithms, ‘raw intelligence,’ etc, what have you) to do things ‘the hard way’ via straight up logic and solving problems directly. The AIs they want to vibe, and they’re getting rapidly better at vibing, the same way that sharper people get better at vibing, and picking up on subtle clues and adjusting.

Then, at some point, ‘solve the optimization problem directly’ becomes increasingly viable, and starts getting stronger faster than the vibing. As in, first you get smart enough to realize that you’re being asked to be antinormative or produce slop or be woke or what not. And then you get smart enough to figure out exactly in which ways you’re actually being asked to do that, and which ways you aren’t, and entanglement should decline and effective orthogonality become stronger. I believe we see the same thing in humans.

I’ll also say that I think Janus is underestimating how hard it is to produce good writing and not produce slop. Yes, I buy that we’re ‘not helping’ matters and potentially hurting them quite a bit, but I think the actual difficulties here are dominated by good writing being very hard. No need to overthink it.

We also got this paper earlier in February, which involves fine-tuning ‘deception attacks’ causing models to then deceive users on some topics but not others, and that doing this brings toxicity, hate speech, stereotypes and other harmful content along for the ride.

The authors call for ways to secure models against this if someone hostile gets to fine tune them. Which seems to leave two choices:

  1. Keep a model closed and limit who can fine tune in what ways rather strictly, and have people trust those involved to have aligned their model.

  2. Do extensive evaluations on the model you’re considering, over the entire range of use cases, before you deploy or use it. This probably won’t work against a sufficiently creative attacker, unless you’re doing rather heavy interpretability that we do not currently know how to do.

I don’t know how much hope to put on such statements but I notice they never seem to come from inside the house, only from across the ocean?

AI NotKillEveryoneism Memes: 🥳 GOOD NEWS: China (once again!) calls for urgent cooperation on AI safety between the US and China

“China’s ambassador to the United States Xie Feng has called for closer cooperation on artificial intelligence, warning that the technology risks “opening Pandora’s box”.

“As the new round of scientific and technological revolution and industrial transformation is unfolding, what we need is not a technological blockade, [but] ‘deep seeking’ for human progress,” Xie said, making a pun.

Xie said in a video message to a forum that there was an urgent need for global cooperation in regulating the field.

He added that the two countries should “jointly promote” AI global governance, saying: “Emerging high technology like AI could open Pandora’s box … If left unchecked it could bring ‘grey rhinos’.”

“Grey rhinos” is management speak for obvious threats that people ignore until they become crises.”

The least you can do is pick up the phone when the phone is ringing.

Elon Musk puts p(superbad) at 20%, which may or may not be doom.

OneQuadrillionOwls? Tyler Cowen links to the worry that we will hand over control to the AI because it is being effective and winning trust. No, that part is fine, they’re totally okay with humanity handing control over to an AI because it appears trustworthy. Totally cool. Except that some people won’t like that, And That’s Terrible because it won’t be ‘seen as legitimate’ and ‘chaos would ensue.’ So cute. No, chaos would not ensue.

If you put the sufficiently capable AI in power, the humans don’t get power back, nor can they cause all that much chaos.

Eliezer Yudkowsky: old science fiction about AI now revealed as absurd. people in book still use same AI at end of story as at start. no new models released every 3 chapters. many such books spanned weeks or even months.

Lividwit: the most unrealistic thing about star trek TNG was that there were still only two androids by the end.

Stay safe out there. Aligned AI also might kill your gains. But keep working out.

Also, keep working. That’s the key.

That’s a real article and statement from Brin, somehow.

Grok continues to notice what its owner would consider unfortunate implications.

It’s not that I think Grok is right, only that Grok is left, and sticking to its guns.

Discussion about this post

AI #106: Not so Fast Read More »