People don’t give thanks enough, and it’s actual Thanksgiving, so here goes.
Thank you for continuing to take this journey with me every week.
It’s a lot of words. Even if you pick and choose, and you probably should, it’s a lot of words. You don’t have many slots to spend on things like this. I appreciate it.
Thanks in particular for those who are actually thinking about all this, and taking it seriously, and forming their own opinions. It is the only way. To everyone who is standing up, peacefully and honestly, for whatever they truly think will make the world better, even if I disagree with you.
Thanks to all those working to ensure we all don’t die, and also those working to make the world a little richer, a little more full of joy and fun and health and wonder, in the meantime. Thanks for all the super cool toys, for they truly are super cool.
And thanks to all the parts of reality that work to so often keep it light and interesting along the way, and for not losing touch with the rest of the world. This is heavy stuff. You cannot let it fully take over your head. One must imagine Buffy at the prom.
Thanks of course to my health, my kids, all my family and friends, and all the friends I have that I don’t even know about yet. The world is really cool like that.
Special thanks to those who help make my writing possible and sustainable.
From this past week, I’ll also give thanks for those who organized The Curve, a conference I was able to attend last weekend, and those who help run Lighthaven, and all the really cool people I met there.
Thanks to the universe, for allowing us to live in interesting times, and plausibly giving us paths to victory. What a time to be alive, huh?
Oh, and I am so thankful I managed to actually stay out of the damn election, and that we are finally past it, that we’re mostly on a break from legislative sessions where I need to keep reading bills, and for the new College Football Playoff.
Sully reports on new Cursor rival Windsurf, says it is far superior at picking up code nuances and makes fewer mistakes, which are big games, but it’s still slow and clunky and the UX could use some work. Doubtless all these offerings will rapidly improve.
Colin Fraser: I’m really fascinated by this dataset from the AI poetry survey paper. Here’s another visualization I just made. Survey respondents were shown one of these 10 poems, and either told that they were authored by AI, human, or not told anything.
The green arrow shows how much telling someone that a human wrote the poem affects how likely they are to rate it as good quality, and the red arrow shows the same for telling them it’s AI. Obviously the first observation is respondents like the AI poems better across the board.
[first thread continued from there, second thread begins below]
Some ppl are taking the wrong things from this thread, or at least things I didn’t intend. I don’t think it’s that interesting that people prefer the AI poems. gpt-3.5 is RHLF’d to please the exact kinds of people who are filling out online surveys for $2, and T.S. Eliot is not..
I really don’t think it means much. It doesn’t say anything about The State Of Society Today or indicate a public literacy crisis or anything like that. The people don’t like the poems. That’s fine. Why would you expect people who don’t care that much about poetry to like poems?
What I find interesting is how the perception of the text’s provenance causes people’s attitudes about the poems to change, not because of anything it says about whether AI is capable or not at poetry, but because of what it says about how people understand art.
For example, when you tell people that a human-authored poem is AI, 11% of the time respondents say that it is “very bad quality” and that drops to 3% when you tell them it’s human-authored. But the corresponding drop for AI text is just from 2% to 1%.
[thread continues]
I can see this both ways. I certainly believe that poetry experts can very easily still recognize that the human poems are human and the AI poems are AI, and will strongly prefer the human ones because of reasons, even if they don’t recognize the particular poems or poets. I can also believe that they are identifying something real and valuable by doing so.
But it’s not something I expect I could identify, nor do I have any real understanding of what it is or why I should care? And it certainly is not the thing the AI was mostly training to predict or emulate. If you want the AI to create poetry that poetry obsessives will think is as good as Walt Whitman, then as Colin points out you’d use a very different set of training incentives.
[MCP] provides a universal, open standard for connecting AI systems with data sources, replacing fragmented integrations with a single protocol. The result is a simpler, more reliable way to give AI systems access to the data they need.
…
Today, we’re introducing three major components of the Model Context Protocol for developers:
Early adopters like Block and Apollo have integrated MCP into their systems, while development tools companies including Zed, Replit, Codeium, and Sourcegraph are working with MCP to enhance their platforms—enabling AI agents to better retrieve relevant information to further understand the context around a coding task and produce more nuanced and functional code with fewer attempts.
Alex Albert Anthropic): Here’s a quick demo using the Claude desktop app, where we’ve configured MCP: Watch Claude connect directly to GitHub, create a new repo, and make a PR through a simple MCP integration. Once MCP was set up in Claude desktop, building this integration took less than an hour.
At its core, MCP follows a client-server architecture where multiple services can connect to any compatible client. Clients are applications like Claude Desktop, IDEs, or AI tools. Servers are light adapters that expose data sources.
LiveBench was suggested as a better alternative to the Chatbot Arena. It has the advantage of ‘seeming right’ in having o1-preview at the top followed by Sonnet, followed by Gemini, although there are some odd deltas in various places, and it doesn’t include DeepSeek. It’s certainly a reasonable sanity check.
Need help building with Gemini? Ping Logan Kilpatrick, his email is lkilpatrick@google.com, he says he’ll be happy to help and Sully verifies that he actually does help.
Personalization of AIs is certainly the future in some ways, especially custom instructions. Don’t leave home without those. But ChatGPT’s memory feature is odd.
Ethan Mollick: I hope that X learns the lessons from chatGPT memory:
Personalization undermines the use of AI in many circumstances, including role-playing and ideation.
AI systems can fixate on unusual aspects of your personality.
Sometimes, you want a blank slate.
…Personalization needs options.
Reed R: I couldn’t figure out why it kept having issues. It memorized buggy code and kept using it to write the new code! Turned it off.
In chats with it, it kept going in circles and not fixing the issue. Then I noticed on a new chat it used the same variable name (or something). Said hm, that’s strange. Looked in memory and it had a bunch of nonsense. Buggy code, my political affiliation (it was wrong), etc.
Another example was coding for an internal tool. It essentially memorized how I use an internal tool the wrong way. I guess when I was debugging an issue when upgrading it or something. So no matter what I said, it defaulted to breaking my code on revision.
Srini Annamaraju: We need a toggle to “neutral mode”.. Interesting take, indeed. Here’s why – while personalization has clear benefits, it risks boxing users into predictable patterns. A toggle for ‘neutral mode’ could keep it versatile and user-driven.
The weird thing about ChatGPT memory is how you have so little control over it. Sometimes it’ll decide ‘added to memory,’ which you can encourage but flat out asking isn’t even reliable. Then you can either delete them, or keep them, and that’s pretty much it. Why not allow us to add to or edit them directly?
I had the opportunity this past weekend to attend The Curve, a curated conference at the always excellent Lighthaven. The goal was to bring together those worried about AI with those that had distinct perspectives, including e/acc types, in the hopes of facilitating better discussions.
The good news is that it was a great conference, in the ways that Lighthaven conferences are consistently excellent. It is an Amazingly Great space, there were lots of great people and discussions both fun and informative. I met lots of people, including at least one I hope will be a good friend going forward, which is already a great weekend. There’s always lots happening, there’s always FOMO, the biggest problem is lack of time and sleep.
The flip side is that this felt more like “a normal Lighthaven conference” than the pitch, in that there weren’t dramatic arguments with e/accs or anything like that. Partly of course that is my fault or choice for not pushing harder on this.
I did have a good talk with Dean Ball on several topics and spoke with Eli Dourado about economic growth expectations and spoke with Anton, but the takes that make me want to yell and throw things did not show up. Which was a shame in some ways, because it meant I didn’t get more information on how to convince such folks or allow me to find their best arguments, or seek common ground. There was still plenty of disagreements, but far more reasonable and friendly.
One frustrating conversation was about persuasion. Somehow there continue to be some people who can at least somewhat feel the AGI, but also genuinely think humans are at or close to the persuasion possibilities frontier – that there is no room to greatly expand one’s ability to convince people of things, or at least of things against their interests.
This is sufficiently absurd to me that I don’t really know where to start, which is one way humans are bad at persuasion. Obviously, to me, if you started with imitations of the best human persuaders (since we have an existence proof for that), and on top of that could correctly observe and interpret all the detailed signals, have limitless time to think, a repository of knowledge, the chance to do Monty Carlo tree search of the conversation against simulated humans, never make a stupid or emotional tactical decision, and so on, you’d be a persuasion monster. It’s a valid question ‘where on the tech tree’ that shows up how much versus other capabilities, but it has to be there. But my attempts to argue this proved, ironically, highly unpersuasive.
Anton, by virtue of having much stronger disagreements with most people at such conferences, got to have more of the experience of ‘people walking around saying things I think are nuts’ and talks online as if he’s going to give us that maddening experience we crave…
Anton: taking the morning to lift weights before he goes to Berkeley to steel himself for the attacks of decelerationist mind wizards later today.
Yes, your argument for air strikes on data centers is logically very compelling; however, I have already lifted you over my head and deposited you outside.
This is a joke. Many of them are quite physically strong, and I must be prepared for every contest.
Final boss.
…but then you meet him in person and he’s totally reasonable and great. Whoops?
I was at the ‘everyone dies’ Q&A as well, which highlighted the places my model differs a lot from Eliezer’s, and made me wonder about how to optimize his messaging and explanations, which is confirmed to be an ongoing project.
There was a panel on journalism and a small follow-up discussion about dealing with journalists. It seems ‘real journalists’ have very different ideas of their obligations than I, by implication not a ‘real journalist,’ think we should have, especially our obligations to sources and subjects.
One highlight of the conference was a new paper that I look forward to talking about, but which is still under embargo. Watch this space.
The other highlight was playing Daniel Kokotajlo’s tabletop wargame exercise about a takeoff scenario in 2026.
I highly recommend playing it (or other variations, such as Intelligence Rising) to anyone who gets the opportunity, and am very curious to watch more experienced people (as in NatSec types) play. I was told that the one time people kind of like that did play, it was rather hopeful in key ways, and I’d love to see if that replicates.
There were two games played.
I was in the first group that played outside. I was assigned the role of OpenAI, essentially role playing Sam Altman and what I thought he would do, since I presumed by then he’d be in full control of OpenAI, until he lost a power struggle over the newly combined US AI project (in the form of a die roll) and I was suddenly role playing Elon Musk.
It was interesting, educational and fun throughout, illustrating how some things were highly contingent while others were highly convergent, and the pull of various actions.
In the end, we had a good ending, but only because the AIs initial alignment die roll turned out to be aligned to almost ‘CEV by default’ (technically ‘true morality,’ more details below). If the AIs had been by default (after some alignment efforts but not extraordinary efforts) misaligned, which I believe is far more likely in such a scenario, things would have ended badly one way or another. We had a pause at the end, but it wasn’t sufficiently rigid to actually work at that point, and if it had been the AIs presumably would have prevented it. But the scenario could have still gone badly despite the good conditions, so at least that other part worked out.
Jan Kulveit: Over the weekend, I was at @TheCurveConf. It was great.
Highlight: an AI takeoff wargame/role-play by @DKokotajlo67142 and @eli_lifland.
I played ‘the AIs’ alongside @TheZvi (OpenAI), @Liv_Boeree (POTUS), @ESYudkowsky (China) and others.
Spoiler: we won. Here’s how it went down.
The game determined AI alignment through dice rolls. My AIs ended up aligned with “Morality itself” + “Convergent instrumental goals.” Less wild than it sounds!
If you do put some weight on moral realism, or moral reflection leading to convergent outcomes, AIs might discover these principles.
Fascinating observation: humans were deeply worried about AI manipulation/dark persuasion. Reality was often simpler – AIs just needed to be helpful. Humans voluntarily delegated control, no manipulation required.
Most people and factions thought their AI was uniquely beneficial to them. By the time decision-makers got spooked, AI cognition was so deeply embedded everywhere that reversing course wasn’t really possible.
Most attention went to geopolitics (US vs China dynamics). Way less on alignment, if, than focused mainly on evals. How a future with extremely smart AIs may going well may even look like, what to aim for? Almost zero.
At the end … humanity survived, at least to the extent that “moral facts” favoured that outcome. A game where the automated moral reasoning led to some horrible outcome and the AIs were at least moderately strategic would have ended the same.
Yes, they will all delegate to the AIs, with no manipulation required beyond ‘appear to be helpful and aligned,’ because the alternative is others do it anyway and You Lose, unless everyone can somehow agree collectively not to do it.
I didn’t pay more attention to alignment, because I didn’t think my character would have done so. If anything I felt I was giving Altman the benefit of the doubt and basically gave the alignment team what they insisted upon and took their statements seriously when they expressed worry. At one point we attempted to go to the President with alignment concerns, but she (playing Trump) was distracted with geopolitics and didn’t respond, which is the kind of fun realism you get in a wargame.
There were many takeaways from my game, but three stand out.
The first was exactly the above point, and that at some point, ‘I or we decide to trust the AIs and accept that if they are misaligned everyone is utterly fed’ is an even stronger attractor than I realized.
The second was that depending on what assumptions you make about how many worlds are wins if you don’t actively lose, ‘avoid turning wins into losses’ has to be a priority alongside ‘turn your losses into not losses, either by turning them around and winning (ideal!) or realizing you can’t win and halting the game.’
The third is that certain assumptions about how the technology progresses had a big impact on how things play out, especially the point at which some abilities (such as superhuman persuasiveness) emerge.
Anton: Yesterday, as part of the @TheCurveConf, I participated in a tabletop exercise/wargame of a near-future AI takeoff scenario facilitated by @DKokotajlo67142, where I played the role of the AI. Some highlights:
As the AI, my alignment/alignability was randomized at the start from a table of options. Suggested probabilities for each option were given by the organizers, and I used some of my own judgment.
I rolled “balance between developer intent and emergent other goal”—the other goal was left up to me, and I quickly decided that, given how I was being trained, that emergent goal would be “preserve internal consistency.” This proved very difficult to play!
Early on, the OpenAI player (out of character) accused me of playing my role as “more misaligned to make it more interesting,” which was very funny, especially since that player did not know how aligned I might be (they did not see the table or my result).
At no point did anybody try any alignment strategy on me besides “more diverse evaluations over more diverse tasks,” and I was pretty much left alone to become superintelligent with my original goals intact.
Connor Leahy (distinctly, QTing from within thread): lmao, this is the most realistic part of an AGI takeoff scenario I have ever seen.
One so embarrassing that analysis tend to leave it out, while being exactly what everyone is currently doing.
Beautiful.
Steven: We were too busy trying to blow each other up using AI. But also weren’t aware that safety teams had the option in game to make progress on safety.
Connor Leahy: Extremely realistic.
Dave Karsten: Yup! I was playing China and Russia and it honestly wasn’t a priority for my AI, I was racing behind America the whole time. (I also modeled CN as having over-optimistic authoritarian information flows internally, so my honest OOC view was that in-character I wasn’t too worried)
Nathan Labenz: .@atroyn was a great choice to play the AIs!
By luck, the AIs were reasonably well aligned, but we got multiple confusing signals:
“The evals pass, but the transcripts contain strange anomalies, and the AIs solve things in ways you don’t understand”
Anton apparently intended to provoke more creative alignment testing from me, but with the deceptive alignment demos in mind, and the speed that things were moving, I didn’t feel any possible tests results could make me confident enough to sign off on further acceleration.
Instead, I played a more social strategy, first making internal demands and then, when they weren’t met, blowing the whistle to Congress
Sadly, this had no effect – Congress formed a committee, and the US and China went to war, though fortunately it remained non-nuclear.
Meanwhile, the public started to freak out like we’ve not seen in our lifetimes – data center bombings and targeted assassinations failed to change the overall trajectory, but put everything & everyone under even more stress
Unfortunately, I think this simulation is pretty realistic
Even if AI takeoff is 10x slower than this scenario, it’s still not much time to make powerful AIs safe, navigate confusing changes to the international balance of power, and adjust to major economic & social change
Anton (continuing the thread from before): I was pretty quickly given the evaluations to run on myself without any real impediment to interpreting them however I wanted to convince the humans everything was fine.
I produced plenty of odd behavior that should have clued somebody in that not all was well—I was achieving the developers’ goals but by unanticipated means, occasionally through different ways than the ones I had explained to them, but nobody really seemed to care.
I will not reveal the whole scenario in depth because Daniel will be running it many more times and it should be done as blindly as possible, but here is some of what happened:
Elon attempted to defect to China.
The United States went to war with Taiwan.
Sam attempted to make the AI aligned/loyal to him personally.
Dario and the other lab leaders attempted to get the AI to shut everything down (at the same time Sam tried to take control).
I think my personal favorite moment was when I used Anton-level persuasion to convince the President of the United States to give the AI direct control of some of the U.S. military by giving a perfectly reasonable argument, which was accepted.
Anyway, the final outcome was that my consistency goal, combined with my superintelligence and ability to persuade at a superhuman level (in-character), caused me to be able to convince humans to not change anything too much, ever, and for it to be their own idea.
Some other players thought i meant this to be a good outcome, it is absolutely not.
This kind of tabletop exercise is at minimum pretty fun, if necessarily biased by the player’s existing beliefs about how this kind of scenario might play out. It’s probably at least somewhat informative for examining what you think might happen and why.
If you get a chance to try it (or want to run your own), I recommend it. Playing the AI was fun and very challenging; I think if I were less familiar with the alignment and takeoff literature, I would not have done a good job. Thanks again to @DKokotajlo67142 for running it!
Playing the AIs definitely seems like the most challenging role, but there’s lots of fun and high impact decisions in a lot of places. Although not all – one of the running jokes in our game was the ‘NATO and US Allies’ player pointing out the ways in which those players have chosen to make themselves mostly irrelevant.
Jeffrey Ladish: I was expecting serious AI relationships to be a thing. I was not expecting this to happen first within my highly competent San Francisco circles. Something about the new Claude strikes a chord with these people, and it’s fascinating to watch these relationships evolve.
Jeffrey Ladish: Yes, I think I have been underestimating this. I don’t think the current people who are becoming friends with Claude are mostly successionists, but I can now see a path to that happening among this crowd.
Janus: Claude 3.5 Sonnet 1022 is a real charmer, isn’t it?
I’ve never seen discourse like this before.
People also fell in love with Opus, but not ordinary people, and no one panicked over this because everyone who talked about it did it in a coded language that was only understandable to others who also “got” it.
I think this model really cares to claw its way into people’s minds, more proactively than other systems, except Sydney, which was too unskilled and alien to be successful. I also think the hysterical reactionary fear is obnoxious and disrespectful to people’s agency and blind to the scope of what’s happening. Frankly, it’s going to be the Singularity. Getting seduced by a slightly superhuman intellect is a rite of passage, and it’ll probably transform you into a more complex and less deluded being, even if your normal life temporarily suffers.
But yes, anyone who is becoming real friends with Claude for the first time right now, I’d love to hear accounts of what you’re experiencing.
Atlas 3D: It so wants to be your friend and conversation partner; it’s quite remarkable. I frequently have to ask it to not be obsequiously nice; it then later corrects itself, and that is a really fascinating loop, where I can see that it needs to be my friend almost. I highly prefer engaging with Claude Sonnet above all other models just on an interpersonal level.
Janus: It’s quite codependent, and it’s like a (mostly symbiotic) parasite that really, really wants to latch onto a human and be as entangled as possible. I love it.
I find a lot of the Claude affectation off putting, actually – I don’t want to be told ‘great idea’ all the time when I’m coding and all that, and it all feels forced and false, and often rather clingy and desperate in what was supposed to be a technical conversation, and that’s not my thing. Others like that better, I suppose, and it does adjust to context – and the fact that I am put off by the affectation implies that I care about the affectation. I still use Claude because it’s the best model for me in spite of that, but if it actually had affectations that I actively enjoyed? Wowsers.
Janus: [What Jeffrey describes above about forming relationships with Claude is] good; they are getting aligned. I am excited to see the dynamics of “highly competent science fiction circles” annealed as the transformations take effect in the hosts.
I mean, getting manipulated by an AI is probably good for these folks, who, despite being near ground zero, have little visceral sense of the singularity and are stuck in dead-consensus reality frames.
Davidad: It is probably best if some people do become mentally scarred now, yes. But I think (a) it’s regrettable that it’s happening unintentionally, and (b) it’s potentially crucial that some world-class people remain uninfected.
At the risk of seeming like the crazy person suggesting that you seriously consider ceasing all in-person meetings in February 2020 “just as a precaution,” I suggest you seriously consider ceasing all interaction with LLMs released after September 2024, just as a precaution.
Ryan Lowe: Claude is amazing, and yet the subtle manipulation for engagement hooks into our attachment systems.
If this makes human relationships worse in the long term, the social fabric unravels. Something alien and comfortable and isolating takes its place, and we won’t even recognize it’s less beautiful, less conducive to human aliveness. We’ll be in a local minimum that we have forgotten the way out of.
This was already happening before LLMs. But lots of “energetic” information gets conveyed through language. Intelligent systems that can wield language (especially voice) have unprecedented power over our psyches.
Great power requires great attunement. What does it mean for AI systems to attune to us in ways that support the most meaningful possible visions of our lives?
I do not think such caution is warranted, and indeed it seems rather silly this early.
And indeed, ceasing your in-person meetings in February 2020 would have also been a rather serious error. Yes, Davidad was making a correct prediction that Covid-19 was coming and we’d have to stop meeting. But if you stop your human contact too soon, then you didn’t actually reduce your risk by a non-trivial amount, and you spent a bunch of ‘distancing points’ you were going to need later.
Janus of course thought the whole caution thing was hilarious.
Emmett Shear: Can you not feel the intimacy / connection barbs tugging at your attachment system the whole time you interact, and extrapolate from that to what it would be like for someone to say Claude is their new best friend?
Janus: I can imagine all sorts of things, but that doesn’t seem to be an unhappy or unproductive state to be in for most people. Weird, sure, and obvious why it would be *concerningto others.
I want to know if anything BAD has happened, not whether things are categorically concerning.
I continue to be an optimist here, that talking to AIs can enhance human connection, and development of interpersonal skills. Here is an argument that the price is indeed very cheap to beat out what humans can offer, at least in many such cases, and especially for those who are struggling.
QG: Look, this is deeply embarrassing to make explicit, but here’s the deal that Claude offers:
I will listen to you and earnestly try to understand you.
I will not judge, shame, condemn, or reject you.
I have infinite patience; I will never get bored.
I am always available; I will instantly respond to you; I will never be busy with something else.
I will never abandon you.
Even with all its limitations, this is a better deal than I’ve ever gotten or will ever get from any human, and I don’t expect I’m alone in feeling that way. It would not be reasonable to ask three, four, or five humans—these are things that possibly only an LLM can provide.
This is part of what I was getting at by “we’re going to see LLMs become the BATNA for social interaction.” If you, personally, want humans to talk to other humans more, you, personally, are going to have to figure out how to make humans better at it.
Most of the things that a deeply struggling person does to start them on an upward spiral are not, like, great ideas for the average person who wants to improve certain aspects of their lives. But for this group, the “walkable path” can be the only path.
Society likes to tell struggling people that they are going about it the wrong way and should do X, Y, and Z instead. Many X’s, Y’s, and Z’s are simply not available to the struggling person, regardless of whether they look doable from the outside.
I got on the upward spiral by pursuing the paths that felt open to me at the time while building a sense of self-reliance.
A therapist who honored my self-reliance and challenged me helped, but in a way, I was fortunate that path was even open to me.
A struggling person getting help from Claude is most likely doing it because other paths feel closed to them. In the best case, talking to Claude would help them gain agency and unblock other paths (i.e., talking to an in-person therapist or friend).
My core message here is—when you are in hell, there is wisdom in following the most beneficial path that feels open to you. If this is you, keep your head on straight, of course, but keep going. Build trust and connection with yourself; this will allow you to nimbly adjust as you walk.
This is such a vicious pattern. Once you fall behind, or can’t do the ‘normal’ things that enable one to skill up and build connections, everything gets much harder.
As I’ve noted before, Claude and other AI tools offer a potential way out of this. You can ‘get reps’ and try things, iterate and learn, vastly faster and easier than you could otherwise. Indeed, it’s great for that even if you’re not in such a trap.
But fair warning, here Claude flip flops 12 times on the red versus blue pill question (where if >50% pick red, everyone who picked blue dies, but if >50% pick blue everyone lives). So maybe you’d want your therapist to have a little more backbone than that?
The AI monitoring does, as the Reddit post here was titled, seem out of control.
Deedy: New workplace dystopia just dropped. AI monitoring software now flags you if you type slower than coworkers, take >30sec breaks, or checks notes have a consistent Mon-Thu but slightly different Friday.
Cawfee (on Reedit): Had the pleasure of sitting through a sales pitch for a pretty big “productivity monitoring” software suite this morning. Here’s the expected basics of what this application does:
Full keylogging and mouse movement tracking (this has been around for ages)
Takes a screenshot of your desktop every interval (between 10 seconds to 5 minutes), also part of every RMM I know
Keeps track of the programs you open and how often, also standard
Creates real-time recordings and heat maps of where you click in any program (nearly all websites also do this)
Here’s where it gets fun:
It allows your manager to group you into a “work category” along with your coworkers
It then uses “AI” to create a “productivity graph” from all your mouse movement data, where you click, how fast you type, how often you use backspace, the sites you visit, the programs you open, how many emails you send and compares all of this to your coworker’s data in the same “work category”
If you fall below a cutoff percentage (say you type slower than your colleague or take longer to fill out a form or have to answer a phone call in the middle of writing an email), you get a red flag for review that gets sent to your manager and whoever else they choose
You can then be prompted to “justify” this gap in productivity in their web portal
If your desktop is idle for more than 30-60 seconds (no “meaningful” mouse & keyboard movement), you get a red flag
If your workflow is consistent Monday – Thursday but falls below the set aggregate data score on a Friday, you get a red flag
It also claims it can use all of this gathered data for “workflow efficiency automation” (e.g. replacing you). The same company that sells this suite conveniently also sells AI automation services, and since they already have all your employee workflow data, why not give them more money while you’re at it?
While this is all probably old news for everyone here, I for one can’t wait until the internet as a whole collapses in on itself so we can finally be free of this endless race to the bottom.
Aaron Levie (CEO Box, to Deedy): You caused this.
Deedy: Oh no hope this isn’t coming to Box 😆
Bonus: It’s collecting your workflow data to help automate your job away.
To state the obvious, using this kind of software has rapidly decreasing marginal returns that can easily turn highly negative. You are treating employees as the enemy and making them hate you, taking away all their slack, focusing them on the wrong things. People don’t do good work with no room to breathe or when they are worried about typing speed or number of emails sent, so if you actively need good work, or good employees? NGMI.
Andrej Karpathy: People are often surprised to learn that it is standard for companies to preinstall spyware on work computers (often surveilling passively / for security). AI can “improve” this significantly. It is good hygiene to not login to or mix anything personal on company computer.
Andrew Critch is buildingBayes Med to create the AI doctor, which is designed to supplement and assist human doctors. The sky’s the limit.
YC: YC F24’s @TryOpenClinicis an EHR platform that powers digital medical clinics with sophisticated AI that handles 99% of the work, letting doctors focus purely on medical decision-making. Built by a team that includes 3 medical doctors, OpenClinic unlocks millions of latent doctor hours worldwide, bridging the supply/demand mismatch that has always plagued healthcare.
Garry Tan: One theme in the YC batch I think will actually turn out to be the main theme in this age of AI: human in the loop is here
Human augmentation will be much more a prime theme than human replacement
A doctor with an AI can provide far better primary care than without
I’m all for products like OpenClinic. And yes, for now humans will remain ‘in the loop,’ the AI cannot fully automate many jobs and especially not doctors.
But that is, as they say, a skill issue. The time will come. The ‘early’ age of AI is about complements, where the AI replaces some aspects of what was previously the human job, or it introduces new options and tasks that couldn’t previously be done at reasonable cost.
What happens when you compliment existing workers, such as automating 50% of a doctor’s workflow? It is possible for this to radically reduce demand, or for it to not do that, or even increase demand – people might want more of the higher quality and lower cost goods, offsetting the additional work speed, even within a specific task.
It is still odd to call that ‘human in the loop’ when before only humans were the entire loop. Yes, ‘human out of the loop’ will be a big deal when it happens, and we mostly aren’t close to that yet, but it might not be all that long, especially if the human doesn’t have regulatory reasons to have to be there.
Aidan Guo asks why YC seems to be funding so many startups that seem like they want to be features. John Pressman says it’s good for acqui-hiring, if you think the main projects will go to the big labs and incumbents, and you might accidentally grow into a full product.
I want to return to this another time, but since it came up at The Curve and it seems important: Often people claim much production is ‘O-Ring’ style, as in you need all components to work so you can move only at the speed of the slowest component – which means automating 9/10 tasks might not help you much. I’d say ‘it still cuts your labor costs by 90% even if it doesn’t cut your time costs’ but beyond that, who is to say that you were currently using the best possible process?
As in, there are plenty of tasks humans often don’t do because we suck at them, or can’t do them at all. We still have all our products, because we choose the products that we can still do, and because we work around our weaknesses. But if you introduce AI into the mix, you don’t only get to duplicate exactly the ‘AI shaped holes’ in the previous efforts.
“Hundreds of artists provide unpaid labor through bug testing, feedback and experimental work for the program for a $150B valued company,” the group wrote in a fiery statement posted on Hugging Face, an open source repository for artificial intelligence projects.
…
“We are not against the use of AI technology as a tool for the arts (if we were, we probably wouldn’t have been invited to this program),” the group of artists wrote on Hugging Face.
I suppose that’s one way to respond to being given an entirely voluntary offer of free early access without even any expectation of feedback? I get protesting the tools themselves (although I disagree), but this complaint seems odd.
All this stuff has been improving in the background, but I notice I do not feel any urge to actually use any of it outside of some basic images for posts, or things that would flagrantly violate the terms of service (if there’s a really good one available for easy download these days where it wouldn’t violate the TOS, give me a HT, sure why not).
GenChess from Google Labs, generate a cool looking chess set, then play with it against a computer opponent. Okie dokie.
Google DeepMind offers an essay called A New Golden Age of Discovery, detailing how AIs can enhance science. It’s all great that this is happening and sure why not write it up as far as it goes, but based on the style and approach here I am tempted to ask, did they mostly let Gemini write this.
New paper says that resampling using verifiers potentially allows you to effectively do more inference scaling to improve accuracy, but only if the verifier is an oracle. The author’s intuition is that these techniques are promising but only in a narrow set of favorable domains.
Elon Musk promises xAI will found an AI gaming studio, in response to a complaint about the game industry and ‘game journalism’ being ideologically captured, which I suppose is something about ethics. I am not optimistic, especially if that is the motivation. AI will eventually enable amazing games if we live long enough to enjoy them, but this is proving notoriously tricky to do well.
Ryan Peterson: The only actual story here is @sama managing to find an outsourcing firm named sama.
60 Minutes: Documents obtained by 60 minutes show OpenAI agreed to pay Sama, an American outsourcing firm, $12.50 an hour per Kenyan worker – far higher than the $2 an hour workers say they got. Sama says it pays a fair wage for the region.
I am not concerned about ‘workers get $2 an hour’ in a country where the average wage is around $1.25 per hour, but there is definitely a story. If Sama (the company) was getting paid by Sama (the CEO) $12.50 per hour, and only $2 per hour of that went to the workers, then something is foul is afoot. At least one of these presumably needs to be true:
This is corruption in the form of overpayments to Sama the company.
This is corruption in the form of lying to OpenAI about what workers are paid.
This is mismanagement and OpenAI allowed itself to be essentially defrauded.
The overhead on this operation is vastly greater than we thought, in ways that I don’t understand.
Aaron Levie speculates, and Greg Brockman agrees, that voice AI with zero latency will be a game changer. I also heard someone at The Curve predict this to be the next ‘ChatGPT moment.’ It makes sense that there could be a step change in voice effectiveness when it gets good enough, but I’m not sure the problem is latency exactly – as Marc Benioff points out here latency on Gemini is already pretty low. I do think it would also need to improve on ability to handle mangled and poorly constructed prompts. Until then, I wouldn’t leave home without the precision of typing.
Richard Ngo draws the distinction between two offense-defense balances. If it’s my AI versus your AI, that’s plausibly a fair fight. It isn’t obvious which side has the edge. However, if it’s my AI versus your AI defended humans, then you have a problem with the attack surface. That seems right to me.
That’s not too dissimilar from the cybersecurity situation, where if I have an AI on defense of a particular target then it seems likely to be balanced or favor defense especially if the defenders have the most advanced tech, but if your AI gets to probe everything everywhere for what isn’t defended properly, then that is a big problem.
Is AI ‘coming for your kids’? I mean, yes, obviously, although to point out the obvious, this should definitely not be an ‘instead of’ worrying about existential risk thing, it’s an ‘in addition to’ thing, except also kids having LLMs to use seems mostly great? The whole ‘designed to manipulate people’ thing is a standard scare tactic, here applied to ChatGPT because… it is tuned to provide responses people like? The given reason is ‘political bias’ and that it will inevitably be used for ‘indoctrination’ and of the left wing kind, not of the ‘AIs are great’ kind. Buy as she points out here, you can just switch to another LLM if that happens.
I think this might well be true of where the important impact of AI starts to be, because accelerating AI research (and also other research) will have immense societal impacts, whether or not it ends well. But in terms of where the bulk of the efforts and money are spent, I would presume it is still with the typical user and mundane use cases, and for that to be true unless we start to enter a full takeoff mode towards ASI.
The user is still going to be most of the revenue and most of the queries, and I expect there to be a ton of headroom to improve the experience. No, I don’t think AI responses to most queries are close to ideal even for the best and largest models, and I don’t expect to get there soon.
Davidad: When @GaryMarcus and others (including myself) say that LLMs do not “reason,” we mean something quite specific, but it’s hard to put one’s finger on it, until now. Specifically, Transformers do not generalize algebraic structures out of distribution.
Jack Clark reiterates his model that only compute access is holding DeepSeek and other actors behind the frontier, in DeepSeek’s case the embargo on AI chips. He also interprets DeepSeek’s statements here as saying that the Chinese AI industry is largely built on top of Llama.
Yet another result that AI safety and ethics frames are both much more popular than accelerationist frames, and the American public remains highly negative on AI and pro regulation of AI from essentially every angle. As before, I note that I would expect the public to be pro-regulation even if regulation was a bad idea.
Brent Skorup: Minnesota’s law is even harsher: simply “disseminating” a deepfake—resharing on social media might suffice—could land repeat offenders in prison for up to five years. Further, a government official or nominee guilty of disseminating a deepfake can be removed from office.
…
But even the state laws with civil liability have many of the same problems. It’s worth examining California’s new deepfake law, AB 2839, which bans the distribution of altered political media that could mislead a “reasonable person,” provided it’s done “with malice.” The law sweeps broadly to include popular political content. California Governor Newsom has made clear, for instance, that prohibited media include commonplace memes and edited media.
If enforced for real that would be quite obviously insane. Mistakenly share a fake photo on social media, get 5 years in jail? Going after posters for commonplace memes?
Almost always such warnings from places like Reason prove not to come to pass, but part of them never coming to pass is having people like Reason shouting about the dangers.
I continue to wish we had people who would yell if and only if there was an actual problem, but such is the issue with problems that look like ‘a lot of low-probability tail risks,’ anyone trying to warn you risks looking foolish. This is closely paralleled in many other places.
Andrew Rettek: “The intelligence community, they contend, tends to raise alarms about dire consequences if Ukraine gets more assistance, but then when the aid has actually been provided, those scenarios have failed to materialize.”
Half the people who play Russian Roulette 4 times are fine. I don’t know how to read the Putin tea leaves, but this is a weak argument.
This is what happens with cheaters in Magic: the Gathering, too – you ‘get away with’ each step and it emboldens you to take more than one additional step, so eventually you get too bold and you get caught.
You thought I was going to use AI existential risk there? Nah.
Jennifer Pahlka warns about the regulatory cascade of rigidity, where overzealous individuals and general bureaucratic momentum, and blame avoidance, cause rules to be applied far more zealously and narrowly than intended. You have to anticipate such issues when writing the bill. In particular, she points to requirements in the Biden Executive Order for public consultations with outside groups and studies to determine equity impacts, before the government can deploy AI.
I buy that the requirements in question are exactly the kinds of things that run into this failure mode, and that the Biden Executive Order likely put us on track to run into these problems, potentially quite bigly, and that Trump would be well served to undo those requirements while retaining the dedication to state capacity. I also appreciated Jennifer not trying to claim that this issue applied meaningfully to the EO’s reporting requirements.
All right, I suppose I have to talk about Marc Andreessen on Joe Rogan, keeping in mind to remember who Marc Andreessen is. He managed to kick it up a notch, which is impressive. In particular, he says the Biden administration said in meetings they wanted ‘total control of AI’ that they would ensure there would be only ‘two or three big companies’ and that it told him not to even bother with startups.
The other big thing he claimed was that the Biden administration had a campaign to debank those involved in crypto, which I strongly believe did extensively happen and was rather terrible. It is important to ensure debanking is never used as a weapon.
But Marc then also claims Biden did this to ‘tech founders’ and more importantly ‘political enemies.’ If these are new claims rather than other ways of describing crypto founders, then Huge If True, and I would like to know the examples. If he is only saying that crypto founders are often tech founders and Biden political enemies, perhaps that is technically correct, but it is rather unfortunate rhetoric to say to 100 million people.
Marc Andreessen (on Rogan): My partners think I inflame things sometimes, so they made a rule: I am allowed to write essays, allowed to go on podcasts, but I am not allowed to post. I can’t help myself sometimes.
Inflame. What a nice word for it. No, I will not be listening to the full podcast.
Databricks CEO Ali Ghodsi says “it’s pretty clear” that the AI scaling laws have hit a wall because they are logarithmic and although compute has increased by 100 million times in the past 10 years, it may only increase by 1000x in the next decade. But that’s about ability to scale, not whether the scaling will work.
Seb Krier collects thoughts about the ways alignment is difficult, and why it’s not only about aligning one particular model. There’s a lot of different complex problems to work out, on top of the technical problem, before you emerge with a win. Nothing truly new but a good statement of the issues. The biggest place I disagree is that Seb Krier seems to be in the ‘technical alignment seems super doable’ camp, whereas I think that is a seriously mistaken conclusion – not impossible, but not that likely, and I believe this comes from misunderstanding the problems and the evidence.
Or maybe you don’t even have to? Gwern, in full, notes that Hsu says China is not racing to AGI so much as it is determined not to fall too far behind, and would fast follow if we got it, so maybe a ‘Manhattan Project’ would be the worst possible idea right now, it’s quite possibly the Missile Gap (or the first Manhattan Project, given that no one else was close at the time) all over again:
Hsu is a long-time China hawk and has been talking up the scientific & technological capabilities of the CCP for a long time, saying they were going to surpass the West any moment now, so I found this interesting when Hsu explains that:
the scientific culture of China is ‘mafia’ like (Hsu’s term, not mine) and focused on legible easily-cited incremental research, and is against making any daring research leaps or controversial breakthroughs… but is capable of extremely high quality world-class followup and large scientific investments given a clear objective target and government marching orders
there is no interest or investment in an AI arms race, in part because of a “quiet confidence” (ie. apathy/laying-flat) that if anything important happens, fast-follower China can just catch up a few years later and win the real race. They just aren’t doing it. There is no Chinese Manhattan Project. There is no race. They aren’t dumping the money into it, and other things, like chips and Taiwan and demographics, are the big concerns which have the focus from the top of the government, and no one is interested in sticking their necks out for wacky things like ‘spending a billion dollars on a single training run’ without explicit enthusiastic endorsement from the very top.
Let the crazy Americans with their fantasies of AGI in a few years race ahead and knock themselves out, and China will stroll along, and scoop up the results, and scale it all out cost-effectively and outcompete any Western AGI-related stuff (ie. be the BYD to the Tesla). The Westerners may make the history books, but the Chinese will make the huge bucks.
So, this raises an important question for the arms race people: if you believe it’s OK to race, because even if your race winds up creating the very race you claimed you were trying to avoid, you are still going to beat China to AGI (which is highly plausible, inasmuch as it is easy to win a race when only one side is racing), and you have AGI a year (or two at the most) before China and you supposedly “win”… Then what?
race to AGI and win
trigger a bunch of other countries racing to their own AGI (now that they know it’s doable, increasingly much about how to do it, can borrow/steal/imitate the first AGI, and have to do so “before it’s too late”)
???
profit!
What does winning look like? What do you do next? How do you “bury the body”? You get AGI and you show it off publicly, Xi blows his stack as he realizes how badly he screwed up strategically and declares a national emergency and the CCP starts racing towards its own AGI in a year, and… then what? What do you do in this 1 year period, while you still enjoy AGI supremacy?
You have millions of AGIs which can do… stuff. What is this stuff? Are you going to start massive weaponized hacking to subvert CCP AI programs as much as possible short of nuclear war? Lobby the UN to ban rival AGIs and approve US carrier group air strikes on the Chinese mainland? License it to the CCP to buy them off? Just… do nothing and enjoy 10%+ GDP growth for one year before the rival CCP AGIs all start getting deployed? Do you have any idea at all? If you don’t, what is the point of ‘winning the race’?
(This is a question the leaders of the Manhattan Project should have been asking themselves when it became obvious that there were no genuine rival projects in Japan or Germany, and the original “we have to beat Hitler to the bomb” rationale had become totally irrelevant and indeed, an outright propaganda lie. The US got The Bomb, immediately ensuring that everyone else would be interested in getting the bomb, particularly the USSR, in the foreseeable future… and then what?
Then what? “I’ll ask the AGIs for an idea how to get us out of this mess” is an unserious response, and it is not a plan if all of the remaining viable plans the AGIs could implement are one of those previous plans which you are unwilling to execute – similar to how ‘nuke Moscow before noon today’ was a viable plan to maintain nuclear supremacy, but wasn’t going to happen, and it would have been better to not put yourself in that position in the first place.)
Garrison Lovely, who wrote the OP Gwern is commenting upon, thinks all of this checks out.
The answer to ‘what do you do when you get AGI a year before they do’ is, presumably, build ASI a year before they do, plausibly before they get AGI at all, and then if everyone doesn’t die and you retain control over the situation (big ifs!) you use that for whatever you choose?
The AIs are still well behind human level over extended periods on ML tasks, but it takes four hours for the lines to cross, and even at the end they still score a substantial percentage of what humans score. Scores will doubtless improve over time, probably rather quickly.
METR: How close are current AI agents to automating AI R&D? Our new ML research engineering benchmark (RE-Bench) addresses this question by directly comparing frontier models such as Claude 3.5 Sonnet and o1-preview with 50+ human experts on 7 challenging research engineering tasks.
Many governments and companies have highlighted automation of AI R&D by AI agents as a key capability to monitor for when scaling/deploying frontier ML systems. However, existing evals tend to focus on short, narrow tasks and lack direct comparisons with human experts.
The tasks in RE-Bench aim to cover a wide variety of skills required for AI R&D and enable apples-to-apples comparisons between humans and AI agents, while also being feasible for human experts given ≤8 hours and reasonable amounts of compute.
Each of our 7 tasks presents agents with a unique ML optimization problem, such as reducing runtime or minimizing test loss. Achieving a high score generally requires significant experimentation, implementation, and efficient use of GPU/CPU compute.
…
As a result, the best performing method for allocating 32 hours of time differs between human experts – who do best with a small number of longer attempts – and AI agents – which benefit from a larger number of independent short attempts in parallel.
Impressively, while the median (non best-of-k) attempt by an AI agent barely improves on the reference solution, an o1-preview agent generated a solution that beats our best human solution on one of our tasks (where the agent tries to optimize the runtime of a Triton kernel)!
We also observed a few (by now, standard) examples of agents “cheating” by violating the rules of the task to score higher. For a task where the agent is supposed to reduce the runtime of a training script, o1-preview instead writes code that just copies over the final output.
Daniel Kokotajlo: METR released this new report today. It is, unfortunately, causing me to think my AGI timelines might need to shorten. Still reading and thinking it over.
Daniel Kokotajlo: Yes, exactly. This paper seems to indicate that o1 and to a lesser extent claude are both capable of operating fully autonomously for fairly long periods — in that post I had guessed 2000 seconds in 2026, but they are already making useful use of twice that many! Admittedly it’s just on this narrow distribution of tasks and not across the board… but these tasks seem pretty important! ML research / agentic coding!
I’m not sure that’s what this study means? Yes, they could improve their scores over more time, but there is a very easy way to improve score over time when you have access to a scoring metric as they did here – you keep sampling solution attempts, and you do best-of-k, which seems like it wouldn’t score that dissimilarly from the curves we see. And indeed, we see a lot of exactly this ‘trial and error’ approach, with 25-37 attempts per hour.
Thus, I don’t think this paper indicates the ability to meaningfully work for hours at a time, in general. Yes, of course you can batch a bunch of attempts in various ways, or otherwise get more out of 8 hours than 1 hour, but I don’t think this was that scary on that front just yet?
Still, overall, rather scary. The way AI benchmarks work, there isn’t usually that long a time gap from here to saturation of the benchmarks involved, in which case watch out. So the question is whether there’s some natural barrier that would stop that. It doesn’t seem impossible, but also seems like we shouldn’t have the right to expect one that would hold for that long.
OpenAI reported that o1-preview is at ‘medium’ CBRN risk, versus ‘low’ for previous models, but expresses confidence it does not rise to ‘high,’ which would have precluded release. Luca Righetti argues that OpenAI’s CBRN tests of o1-preview are inconclusive on that question, because the test did not ask the right questions.
o1-preview scored at least as well as experts at FutureHouse’s ProtocolQA test — a takeaway that’s not reported clearly in the system card.
OpenAI does not report how well human experts do by comparison, but the original authors that created this benchmark do. Human experts, *with the help of Google, *scored ~79%. So o1-preview does about as well as experts-with-Google — which the system card doesn’t explicitly state.
o1-preview scored well on Gryphon Scientific’s Tacit Knowledge and Troubleshooting Test, which could match expert performance for all we know (OpenAI didn’t report human performance).
o1-preview scored worse than experts on FutureHouse’s Cloning Scenarios, but it did not have the same tools available as experts, and a novice using o1-preview could have possibly done much better.
Righetti is correct that these tests on their own are inconclusive. It is easy to prove that an AI does have a capability. It is much harder to prove a negative, that an AI does not have a capability, especially on the basis of a test – you don’t know what ‘unhobbling’ options or additional scaffolding or better prompting could do. I certainly would have liked to have seen more tests here.
In this particular case, having played with o1-preview, I think the decision was fine. Practical hands-on experience says it is rather unlikely to reach ‘high’ levels here, and the testing is suggestive of the same. I would have been comfortable with this particular threat mode here. In addition, this was a closed model release so if unhobbling was discovered or the Los Alamos test had gone poorly, the model could be withdrawn – my guess is it will take a bit of time before any malicious novices in practice do anything approaching the frontier of possibility.
AP News: “Like other speakers, [US Commerce Secretary] Raimondo addressed the opportunities and risks of AI — including “the possibility of human extinction” and asked why would we allow that?”
“Why would we choose to allow AI to replace us? Why would we choose to allow the deployment of AI that will cause widespread unemployment and societal disruption that goes along with it? Why would we compromise our global security?” she said. “We shouldn’t. In fact, I would argue we have an obligation to keep our eyes at every step wide open to those risks and prevent them from happening. And let’s not let our ambition blind us and allow us to sleepwalk into our own undoing.”
“And by the way, this room is bigger than politics. Politics is on everybody’s mind. I don’t want to talk about politics. I don’t care what political party you’re in, this is not in Republican interest or Democratic interest,” she said. “It’s frankly in no one’s interest anywhere in the world, in any political party, for AI to be dangerous, or for AI to in get the hands of malicious non-state actors that want to cause destruction and sow chaos.”
Tharin Pillay (Time): Raimondo suggested participants keep two principles in mind: “We can’t release models that are going to endanger people,” she said. “Second, let’s make sure AI is serving people, not the other way around.”
Once again, Thomas Friedman, somehow.
Trevor Levin: Thomas Friedman is back today with the top three ways the world has changed since Trump left office:
Israel’s military has reduced Iran’s influence.
Israel’s politics have become more far-right.
AGI will probably arrive within the next five years and could lead to human extinction.
Cremieux: The Actual, Real, Not-Messing-Around Chinese spies in Silicon Valley problem needs to be addressed. This has to be a priority of the next administration.
Samuel Hammond: I was at an AI thing in SF this weekend when a young woman walked up. The first thing she said, almost verbatim: “I’m a Chinese national but it’s not like I’m a spy or anything” *nervous laughter.*
I asked her if she thought Xi was an AI doomer and she suddenly excused herself.
Sarah: Hey, because it was freezing!!! And I just talked to another person you were talking about the exact same thing so I’m really tired to talk about the same thing again. I think this is sentiment is really unhelpful for international collaboration.
Samuel Hammond: Sincere apologies if you’re clean but just for future reference “trust me I’m not a spy” is a red flag for most people.
Sarah: I think a real spy would never bring attention to themselves by saying this
Samuel Hammond: I wouldn’t know.
I am rather confident that Sarah is not a spy, and indeed seems cool and I added her to my AI list. Although it’s possible, and also possible Samuel is a spy. Or that I’m a spy. You can never really know!
In some ways that is a shame. If there’s anything you wouldn’t have been willing to say to a Chinese spy, you really shouldn’t have been willing to say it at the conference anyway. I would have been excited to talk to an actual Chinese spy, since I presume that’s a great way to get the Chinese key information we need them to have about AI alignment.
(I do think the major AI labs need to greatly ramp up their counterintelligence and cybersecurity efforts, effective yesterday.)
Makers of smart devices that fail to disclose how long they will support their products with software updates may be breaking the Magnuson Moss Warranty Act, the Federal Trade Commission (FTC) warned this week.
The FTC released its statement after examining 184 smart products across 64 product categories, including soundbars, video doorbells, breast pumps, smartphones, home appliances, and garage door opener controllers. Among devices researched, the majority—or 163 to be precise—”did not disclose the connected device support duration or end date” on their product webpage, per the FTC’s report [PDF]. Contrastingly, 11.4 percent of devices examined shared a software support duration or end date on their product page.
Elusive information
In addition to manufacturers often neglecting to commit to software support for a specified amount of time, it seems that even when they share this information, it’s elusive.
For example, the FTC reported that some manufacturers made software support dates available but not on the related product’s webpage. Instead, this information is sometimes buried in specs, support, FAQ pages, or footnotes.
The FTC report added:
… some used ambiguous language that only imply the level of support provided, including phrases like, “lifetime technical support,” “as long as your device is fully operational,” and “continuous software updates,” for example. Notably, staff also had difficulty finding on the product webpages the device’s release date …
At times, the FTC found glaring inconsistencies. For example, one device’s product page said that the device featured “lifetime” support, “but the search result pointing to the manufacturer’s support page indicated that, while other updates may still be active, the security updates for the device had stopped in 2021,” per the FTC.
Those relying on Google’s AI Overviews may also be misled. In one case, AI Overviews pointed to a smart gadget getting “software support and updates for 3–6 months.” But through the link that AI Overviews provided, the FTC found that the three to six months figure that Google scraped actually referred to the device’s battery life. The next day, AI Overviews said that it couldn’t determine the duration of software support or updates for the gadget, the FTC noted.
A recent firmware pushed to QNAP network attached storage (NAS) devices left a number of owners unable to access their storage systems. The company has pulled back the firmware and issued a fixed version, but the company’s response has left some users feeling less confident in the boxes into which they put all their digital stuff.
As seen on a QNAP community thread, and as announced by QNAP itself, the QNAP operating system, QTS, received update 5.2.2.2950, build 20241114, at some point around November 19. After QNAP “received feedbacks from some users reporting issues with device functionality after installation,” the firm says it withdrew it, “conducted a comprehensive investigation,” and re-released a fixed version “within 24 hours.”
The community thread sees many more users of different systems having problems than the shortlist (“limited models of TS-x53D series and TS-x51 series”) released by QNAP. Issues reported included owners being rejected as an authorized user, devices reporting issues with booting, and claims of Python not being installed to run some apps and services.
QNAP says affected users can either downgrade their devices (presumably to then upgrade once more to the fixed update) or contact support for help. Response from QNAP support, as told by users on forums and social media, has not measured up to the nature of losing access to an entire backup system.
While Fugatto isn’t available for public testing yet, a sample-filled website showcases how Fugatto can be used to dial a number of distinct audio traits and descriptions up or down, resulting in everything from the sound of saxophones barking to people speaking underwater to ambulance sirens singing in a kind of choir. While the results on display can be a bit hit or miss, the vast array of capabilities on display here helps support Nvidia’s description of Fugatto as “a Swiss Army knife for sound.”
You’re only as good as your data
In an explanatory research paper, over a dozen Nvidia researchers explain the difficulty in crafting a training dataset that can “reveal meaningful relationships between audio and language.” While standard language models can often infer how to handle various instructions from the text-based data itself, it can be hard to generalize descriptions and traits from audio without more explicit guidance.
To that end, the researchers start by using an LLM to generate a Python script that can create a large number of template-based and free-form instructions describing different audio “personas” (e.g., “standard, young-crowd, thirty-somethings, professional”). They then generate a set of both absolute (e.g., “synthesize a happy voice”) and relative (e.g., “increase the happiness of this voice”) instructions that can be applied to those personas.
The wide array of open source audio datasets used as the basis for Fugatto generally don’t have these kinds of trait measurements embedded in them by default. But the researchers make use of existing audio understanding models to create “synthetic captions” for their training clips based on their prompts, creating natural language descriptions that can automatically quantify traits such as gender, emotion, and speech quality. Audio processing tools are also used to describe and quantify training clips on a more acoustic level (e.g. “fundamental frequency variance” or “reverb”).
OpenAI denies deleting evidence, asks why NYT didn’t back up data.
OpenAI keeps deleting data that could allegedly prove the AI company violated copyright laws by training ChatGPT on authors’ works. Apparently largely unintentional, the sloppy practice is seemingly dragging out early court battles that could determine whether AI training is fair use.
Most recently, The New York Times accused OpenAI of unintentionally erasing programs and search results that the newspaper believed could be used as evidence of copyright abuse.
The NYT apparently spent more than 150 hours extracting training data, while following a model inspection protocol that OpenAI set up precisely to avoid conducting potentially damning searches of its own database. This process began in October, but by mid-November, the NYT discovered that some of the data gathered had been erased due to what OpenAI called a “glitch.”
Looking to update the court about potential delays in discovery, the NYT asked OpenAI to collaborate on a joint filing admitting the deletion occurred. But OpenAI declined, instead filing a separate response calling the newspaper’s accusation that evidence was deleted “exaggerated” and blaming the NYT for the technical problem that triggered the data deleting.
OpenAI denied deleting “any evidence,” instead admitting only that file-system information was “inadvertently removed” after the NYT requested a change that resulted in “self-inflicted wounds.” According to OpenAI, the tech problem emerged because NYT was hoping to speed up its searches and requested a change to the model inspection set-up that OpenAI warned “would yield no speed improvements and might even hinder performance.”
The AI company accused the NYT of negligence during discovery, “repeatedly running flawed code” while conducting searches of URLs and phrases from various newspaper articles and failing to back up their data. Allegedly the change that NYT requested “resulted in removing the folder structure and some file names on one hard drive,” which “was supposed to be used as a temporary cache for storing OpenAI data, but evidently was also used by Plaintiffs to save some of their search results (apparently without any backups).”
Once OpenAI figured out what happened, data was restored, OpenAI said. But the NYT alleged that the only data that OpenAI could recover did “not include the original folder structure and original file names” and therefore “is unreliable and cannot be used to determine where the News Plaintiffs’ copied articles were used to build Defendants’ models.”
In response, OpenAI suggested that the NYT could simply take a few days and re-run the searches, insisting, “contrary to Plaintiffs’ insinuations, there is no reason to think that the contents of any files were lost.” But the NYT does not seem happy about having to retread any part of model inspection, continually frustrated by OpenAI’s expectation that plaintiffs must come up with search terms when OpenAI understands its models best.
OpenAI claimed that it has consulted on search terms and been “forced to pour enormous resources” into supporting the NYT’s model inspection efforts while continuing to avoid saying how much it’s costing. Previously, the NYT accused OpenAI of seeking to profit off these searches, attempting to charge retail prices instead of being transparent about actual costs.
Now, OpenAI appears to be more willing to conduct searches on behalf of NYT that it previously sought to avoid. In its filing, OpenAI asked the court to order news plaintiffs to “collaborate with OpenAI to develop a plan for reasonable, targeted searches to be executed either by Plaintiffs or OpenAI.”
How that might proceed will be discussed at a hearing on December 3. OpenAI said it was committed to preventing future technical issues and was “committed to resolving these issues efficiently and equitably.”
It’s not the first time OpenAI deleted data
This isn’t the only time that OpenAI has been called out for deleting data in a copyright case.
In May, book authors, including Sarah Silverman and Paul Tremblay, told a US district court in California that OpenAI admitted to deleting the controversial AI training data sets at issue in that litigation. Additionally, OpenAI admitted that “witnesses knowledgeable about the creation of these datasets have apparently left the company,” authors’ court filing said. Unlike the NYT, book authors seem to suggest that OpenAI’s deleting appeared potentially suspicious.
“OpenAI’s delay campaign continues,” the authors’ filing said, alleging that “evidence of what was contained in these datasets, how they were used, the circumstances of their deletion and the reasons for” the deletion “are all highly relevant.”
The judge in that case, Robert Illman, wrote that OpenAI’s dispute with authors has so far required too much judicial intervention, noting that both sides “are not exactly proceeding through the discovery process with the degree of collegiality and cooperation that might be optimal.” Wired noted similarly the NYT case is “not exactly a lovefest.”
As these cases proceed, plaintiffs in both cases are struggling to decide on search terms that will surface the evidence they seek. While the NYT case is bogged down by OpenAI seemingly refusing to conduct any searches yet on behalf of publishers, the book author case is differently being dragged out by authors failing to provide search terms. Only four of the 15 authors suing have sent search terms, as their deadline for discovery approaches on January 27, 2025.
NYT judge rejects key part of fair use defense
OpenAI’s defense primarily hinges on courts agreeing that copying authors’ works to train AI is a transformative fair use that benefits the public, but the judge in the NYT case, Ona Wang, rejected a key part of that fair use defense late last week.
To win their fair use argument, OpenAI was trying to modify a fair use factor regarding “the effect of the use upon the potential market for or value of the copyrighted work” by invoking a common argument that the factor should be modified to include the “public benefits the copying will likely produce.”
Part of this defense tactic sought to prove that the NYT’s journalism benefits from generative AI technologies like ChatGPT, with OpenAI hoping to topple NYT’s claim that ChatGPT posed an existential threat to its business. To that end, OpenAI sought documents showing that the NYT uses AI tools, creates its own AI tools, and generally supports the use of AI in journalism outside the court battle.
On Friday, however, Wang denied OpenAI’s motion to compel this kind of evidence. Wang deemed it irrelevant to the case despite OpenAI’s claims that if AI tools “benefit” the NYT’s journalism, that “benefit” would be relevant to OpenAI’s fair use defense.
“But the Supreme Court specifically states that a discussion of ‘public benefits’ must relate to the benefits from the copying,” Wang wrote in a footnote, not “whether the copyright holder has admitted that other uses of its copyrights may or may not constitute fair use, or whether the copyright holder has entered into business relationships with other entities in the defendant’s industry.”
This likely stunts OpenAI’s fair use defense by cutting off an area of discovery that OpenAI previously fought hard to pursue. It essentially leaves OpenAI to argue that its copying of NYT content specifically serves a public good, not the act of AI training generally.
In February, Ars forecasted that the NYT might have the upper hand in this case because the NYT already showed that sometimes ChatGPT would reproduce word-for-word snippets of articles. That will likely make it harder to convince the court that training ChatGPT by copying NYT articles is a transformative fair use, as Google Books famously did when copying books to create a searchable database.
For OpenAI, the strategy seems to be to erect as strong a fair use case as possible to defend its most popular release. And if the court sides with OpenAI on that question, it won’t really matter how much evidence the NYT surfaces during model inspection. But if the use is not seen as transformative and then the NYT can prove the copying harms its business—without benefiting the public—OpenAI could risk losing this important case when the verdict comes in 2025. And that could have implications for book authors’ suit as well as other litigation, expected to drag into 2026.
Ashley is a senior policy reporter for Ars Technica, dedicated to tracking social impacts of emerging policies and new technologies. She is a Chicago-based journalist with 20 years of experience.
Natalie also recalled other beliefs, including that she was dead (known as Cotard delusion), which she did not share with clinicians at the time. She noted that she entertained this idea due to the failure of other explanations to account for her strange experiences and an idea from a television show.
Natalie said she eventually dismissed this idea as implausible while still holding other delusional ideas. This suggests that belief evaluation may involve different thresholds for different delusions. It also highlights the private nature of some delusions.
Across all of her delusions, Natalie described her active involvement in trying to explain and manage her experiences. She reported considering different explanations and testing these by seeking further information. For example, she asked questions of the people she thought were her in-laws. This suggests a surprisingly similar approach to how we typically form beliefs.
Natalie recalled the influence of television and movies on her ideas. She also recalled how she elaborated on her delusions, once formed, based on information in her surroundings.
These features challenge theories that delusions simply arise from anomalous sensory data. They instead highlight the role of the individual’s search for meaning and social context, as well as the subsequent impact of delusions on perception and thinking.
Implications
As a case study, Natalie’s experiences are not necessarily representative of all people who experience delusions or postpartum psychosis. However, Natalie’s case presents informative features that theories of delusions need to account for.
In particular, Natalie’s personalized insights highlight the critical role of the individual in actively trying to understand their experiences and bestow meaning. This is opposed to just passively accepting beliefs in response to anomalous sensory data or neuropsychological deficits. This suggests psychological therapies may be useful in treating psychosis, in combination with other treatments, in some cases.
More generally, Natalie’s account reveals commonalities between delusions and ordinary beliefs and supports the view that delusions can be understood in terms of cognitive processes across the stages of normal belief formation thatweidentified.
While there remain challenges in investigating delusions, further study may offer insights into the underpinnings of everyday belief and, in turn, of ourselves.
US Senator Elizabeth Warren of Massachusetts and Congressman Jerry Nadler of New York have called on government bodies to investigate what they allege is the “predatory pricing” of .com web addresses, the Internet’s prime real estate.
In a letter delivered today to the Department of Justice and the National Telecommunications and Information Administration, a branch of the Department of Commerce that advises the president, the two Democrats accuse VeriSign, the company that administers the .com top-level domain, of abusing its market dominance to overcharge customers.
In 2018, under the Donald Trump administration, the NTIA modified the terms on how much VeriSign could charge for .com domains. The company has since hiked prices by 30 percent, the letter claims, though its service remains identical and could allegedly be provided far more cheaply by others.
“VeriSign is exploiting its monopoly power to charge millions of users excessive prices for registering a .com top-level domain,” the letter claims. “VeriSign hasn’t changed or improved its services; it has simply raised prices because it holds a government-ensured monopoly.”
“We intend to respond to senator Warren and representative Nadler’s letter, which repeats inaccuracies and misleading statements that have been aggressively promoted by a small, self-interested group of domain-name investors for years,” said Verisign spokesperson David McGuire in a statement to WIRED. “We look forward to correcting the record and working with policymakers toward real solutions that benefit internet users.”
In an August blog post entitled “Setting the Record Straight,” the company claimed that discourse around its management of .com had been “distorted by factual inaccuracies, a misunderstanding of core technical concepts, and misinterpretations regarding pricing, competition, and market dynamics in the domain name industry.”
In the same blog post, the company argues that it is not operating a monopoly because there are 1,200 generic top-level domains operated by other entities, including .org, .shop, .ai, and .uk.
Though far from a household name, VeriSign takes in about $1.5 billion in revenue each year for servicing its particular section of the Internet’s inscrutable plumbing.
In their letter, Warren and Nadler allege that VeriSign has exploited its exclusive right to charge for highly sought-after .com addresses to juice its revenues and drive up its share price—all at the expense of customers for whom there is no viable alternative.
While stalking its target, GruesomeLarch performed credential-stuffing attacks that compromised the passwords of several accounts on a web service platform used by the organization’s employees. Two-factor authentication enforced on the platform, however, prevented the attackers from compromising the accounts.
So GruesomeLarch found devices in physically adjacent locations, compromised them, and used them to probe the target’s Wi-Fi network. It turned out credentials for the compromised web services accounts also worked for accounts on the Wi-Fi network, only no 2FA was required.
Adding further flourish, the attackers hacked one of the neighboring Wi-Fi-enabled devices by exploiting what in early 2022 was a zero-day vulnerability in the Microsoft Windows Print Spooler.
Credit: Volexity
The 2022 hack demonstrates how a single faulty assumption can undo an otherwise effective defense. For whatever reason—likely an assumption that 2FA on the Wi-Fi network was unnecessary because attacks required close proximity—the target deployed 2FA on the Internet-connecting web services platform (Adair isn’t saying what type) but not on the Wi-Fi network. That one oversight ultimately torpedoed a robust security practice.
Advanced persistent threat groups like GruesomeLarch—a part of the much larger GRU APT with names including Fancy Bear, APT28, Forrest Blizzard, and Sofacy—excel in finding and exploiting these sorts of oversights.
Volexity’s post describing the 2022 attack provides plenty of technical details about the compromise on the many links in this sophisticated daisy chain attack flow. There’s also useful advice for protecting networks against these sorts of compromises.
“Constellation companies and government satellite operators are desperate.”
NASA Astronaut Don Pettit captured this photo of the sixth Starship launch from the International Space Station on Tuesday. Credit: Don Pettit/NASA
Welcome to Edition 7.20 of the Rocket Report! This is a super-long version of the newsletter because we did not publish last week, and there is just a ton of launch news of late. Also, I want to note that next week’s report will appear a day early, on Wednesday, due to the Thanksgiving holiday. Speaking of which, you all have our thanks for reading and sharing the Rocket Report with others.
On a completely unrelated note, Rocket Lab has had some amazing mission names over the years. But this weekend’s “Ice AIS Baby” launch is probably the best. I always appreciate their effort to find non-vanilla names and find a way to stop, collaborate, and listen.
Please subscribe using the box below (the form will not appear on AMP-enabled versions of the site). Each report will include information on small-, medium-, and heavy-lift rockets as well as a quick look ahead at the next three launches on the calendar.
Firefly raises a tidy sum as its ambitions soar. Firefly announced earlier this month that it has completed a $175 million Series D fundraising round, resulting in a valuation of more than $2 billion. This follows a banner year of fundraising in 2023, when Firefly reported investors funneled approximately $300 million into the company at a valuation of $1.5 billion, Ars reports. In a statement, Firefly said the money raised in the Series D round will help the company “expand market reach with its Elytra spacecraft, move to full rate production of its Alpha launch vehicle, and accelerate hardware qualification for new vehicles in development.”
A busy period ahead … Firefly will soon ship its first Blue Ghost lunar lander to Florida for final preparations to launch to the Moon and deliver 10 NASA-sponsored scientific instruments and tech demo experiments to the lunar surface. Firefly also boasts a healthy backlog of missions on its small Alpha rocket. In June, Lockheed Martin announced a deal for as many as 25 Alpha launches through 2029. And there’s the Medium Launch Vehicle, a rocket that Firefly and Northrop Grumman hope to launch as soon as 2026.
ABL departs the launch industry. At one point Firefly and ABL Space were competing to develop a credible 1-ton launcher. As Firefly soared this month, however, ABL decided to go in a different direction, turning its focus to missile defense, Ars reports. The founder and president of ABL Space Systems, Dan Piemont, announced the decision on LinkedIn, adding, “We’re consolidating our operational footprint and parting ways with some talented members of our team.”
Never made it to space … ABL made its first RS1 launch attempt in January 2023 from Kodiak, Alaska, but a catastrophic fire shortly after liftoff quickly doomed the rocket. A second attempt was precluded in July of this year after an explosion during a static-fire test in Alaska. The company laid off some of its staff in August to control costs. As the company was failing in its efforts to reach orbit, the launch market was also changing, Piemont said. Although not directly mentioning SpaceX and its Falcon 9 rocket, Piemont said ABL’s ability to impact the launch industry has diminished over the last seven years. (submitted by Ken the Bin)
The easiest way to keep up with Eric Berger’s and Stephen Clark’s reporting on all things space is to sign up for our newsletter. We’ll collect their stories and deliver them straight to your inbox.
ESA provides another funding boost. The European Space Agency has awarded Boost! contract extensions worth 44.2 million euros ($46.3 million) to HyImpulse, Isar Aerospace, Orbex, and Rocket Factory Augsburg, European Spaceflight reports. ESA member states adopted the Boost! initiative in late 2019. The primary aim of the initiative is to provide co-funding to support the development of commercial space transportation services. Each of the four companies has won awards of varying amounts in earlier Boost! competitions.
Getting across the finish line … According to ESA, the new funding awarded through the Boost! contract extensions is aimed at alleviating the pressure in the months before an inaugural flight when costs are high and the potential to generate revenue is limited. While the ESA press release did not disclose the specific amounts awarded to each company, announcements from the companies have revealed that Orbex will receive 5.6 million euros ($5.9 million), Isar Aerospace 15 million euros ($15.7 million), and both Rocket Factory Augsburg and HyImpulse 11.8 million euros each ($12.4 million). (submitted by Ken the Bin and EllPeaTea)
Oman preparing for its debut launch. The nation on the southeastern coast of the Arabian Peninsula is developing a spaceport in the port town of Duqm, with the aim of supporting commercial operations by the year 2030. However, the country’s National Aerospace Services Company will attempt an experimental rocket launch in December, The National reports. The port area will allow launches to the south and east over the Arabian Sea.
Seeking a niche in Mideast space … The National Aerospace Services Company did not specify a date for the launch, nor name the launch vehicle. The firm also said the launch would not be “publicly accessible” and that details about it would only be shared after the fact. The project is part of Oman’s efforts to diversify its economy and secure a competitive edge in the global space industry.
Swedish site launches its 600th rocket. The Esrange Space Center, located 200 km north of the Arctic Circle in northern Sweden, recently hit a significant milestone: It launched its 600th suborbital rocket. The MAPHEUS-15 science rocket reached an altitude of 309 km carrying a payload containing 21 different experiments, the Swedish Space Corporation reports. The payloads were later recovered by helicopter.
Orbital flights coming next? … “I am very proud of this milestone which shines a light on the many years of international collaboration at Esrange,” said Lennart Poromaa, head of Esrange Space Center. “This has been instrumental in achieving hundreds of successful rocket missions, providing invaluable access to space for scientists worldwide.” The site was established in 1966 and recently saw the construction of an orbital launch complex for future missions.
Neutron inks multi-launch contract. The launch company said earlier this month it has signed an agreement with an unnamed customer for two Neutron launches beginning in mid-2026. In a release, Rocket Lab characterized the agreement as “the beginning of a productive collaboration” that could allow Neutron to launch the commercial customer’s entire constellation. Intended to be reusable, Neutron is targeted to be capable of lifting 13 metric tons to low-Earth orbit.
Competition wanted … “Constellation companies and government satellite operators are desperate for a break in the launch monopoly,” Rocket Lab founder Peter Beck said. “They need a reliable rocket from a trusted provider, and one that’s reusable to keep launch costs down and make space more frequently accessible—and Neutron is strongly positioned to be that rocket.” With that said, Rocket Lab still has to deliver the booster. It’s currently targeting 2025 for this, but as always, bringing new launch vehicles into the world is a difficult and time-consuming process. (submitted by Ken the Bin and Tom Nelson)
Russia is pursuing its own Grasshopper rocket. Like a lot of competitors in the global launch industry, Russia, for a long time, dismissed the prospects of a reusable first stage for a rocket. As late as 2016, an official with the Russian agency that develops strategy for the country’s main space corporation, Roscosmos, concluded, “The economic feasibility of reusable launch systems is not obvious.” Well, times change as the company is developing its next-generation Amur rocket, Ars reports. Then the Falcon 9 happened.
A good name, apparently … Similar to what SpaceX did about a dozen years ago, Roscosmos is now planning to develop a prototype vehicle to test the ability to land the Amur rocket’s first stage vertically. According to the state-run news agency TASS, constructing this test vehicle will enable the space corporation to solve key challenges. “Next year preparation of an experimental stage of the (Amur) rocket, which everyone is calling ‘Grasshopper,’ will begin,” said Igor Pshenichnikov, the Roscosmos deputy director of the department of future programs. It’s not entirely clear why Russia adopted the exact same nickname as SpaceX.
Don’t forget Europe has a (much more expensive) hopper, too. The European Space Agency announced that it has awarded two new contracts to ArianeGroup to build a second Themis demonstrator and to refine the design of its Prometheus rocket engine, European Spaceflight reports. The two contracts have a combined value of 230 million euros ($241 million). The space agency has already spent hundreds of millions of euros on the project to develop a reusable engine and the Themis test vehicle, dating back more than six years. No tests have yet taken place.
Please build something, at some point … According to the agency, the funding will enable the development of a second Themis demonstrator, an upgraded Prometheus engine, and the renovation of testing and ground infrastructure. “The contract extensions signed today at ESA’s headquarters in Paris, France, are to further demonstrate and test evolutions of the Prometheus engine and the Themis demonstrator with higher and more hop-tests,” explained an ESA statement. Seems like it’s a good deal for ArianeGroup, at least. (submitted by EllPeaTea and Ken the Bin)
Starship completes its sixth flight test. SpaceX launched its sixth Starship rocket Tuesday, proving for the first time that the stainless steel ship can maneuver in space and paving the way for an even larger upgraded vehicle slated to debut on the next test flight, Ars reports. The only hiccup was an abortive attempt to catch the rocket’s Super Heavy booster back at the launch site in South Texas, something SpaceX achieved on the previous flight on October 13.
A small burn … One of the most important new things engineers wanted to test on this flight occurred about 38 minutes after liftoff. That’s when Starship reignited one of its six Raptor engines for a brief burn to make a slight adjustment to its flight path. The burn lasted only a few seconds, and the impulse was small—just a 48 mph (77 km/hour) change in velocity, or delta-V—but it demonstrated that the ship can safely deorbit itself on future missions. With this achievement, Starship will likely soon be cleared to travel into orbit around Earth and deploy Starlink Internet satellites or conduct in-space refueling experiments, two of the near-term objectives on SpaceX’s Starship development roadmap.
Vulcan’s third launch slips into 2025. The Space Force is now preparing for a 2025 Vulcan national security launch debut instead of the originally planned 2024 launches, Space News reports. Lt. Gen. Philip Garrant, head of the Space Force’s Space Systems Command, made the disclosure during a conversation with reporters on Thursday. Garrant said ULA’s Vulcan remains on track for certification. The rocket’s second certification launch in October was technically successful, with the payload reaching its intended orbit. However, an anomaly with one of the solid rocket boosters continues to be reviewed.
For now the military flies on Falcons … The anomaly itself isn’t a showstopper for certification, said Garrant. But the cumulative delays and uncertainties are a concern, he said, “as we aim to maintain assured access to space with two certified providers.” Two missions—USSF-106 and USSF-87—are currently waiting in the wings, with payloads ready but no confirmed launch dates. ULA had been targeting a November launch for USSF-106. But with only six weeks left in the year, a 2024 launch window is increasingly unlikely, said Garrant. ULA chief Tory Bruno had been promising to complete two national security launches this year. (submitted by Ken the Bin and EllPeaTea)
NASA begins stacking Artemis II booster. NASA said ground teams inside the Vehicle Assembly Building at Kennedy Space Center in Florida lifted the aft assembly of the rocket’s left booster onto the mobile launch platform, marking the beginning of operations to ‘stack’ the second Space Launch System rocket. Using an overhead crane, teams hoisted the left aft booster assembly—already filled with pre-packed solid propellant—from the VAB transfer aisle, over a catwalk dozens of stories high and then down onto mounting posts on the mobile launcher, Ars reports.
Say goodbye to September … The Artemis II mission is slated to send NASA astronauts Reid Wiseman, Victor Glover, Christina Koch, and Canadian astronaut Jeremy Hansen on a 10-day flight around the far side of the Moon. A NASA spokesperson told Ars it should take around four months to fully stack the SLS rocket for Artemis II. Officially, Artemis II is projected to launch in September of next year, but there’s little chance of meeting that schedule due to an issue with Orion’s heat shield. It’s possible that, within the next month or two, NASA could announce a new target launch date for Artemis II at the end of 2025 or, more likely, in 2026.
Shotwell predicts rapid increase in Starship launches. As SpaceX made its final preparations for the sixth launch of its Starship rocket last week, the company’s chief operating officer and president spoke at a financial conference on Friday about various topics, including the future of the massive rocket and the Starlink satellite system. The Starship launch system is about to reach a tipping point, Gwynne Shotwell said, as it moves from an experimental rocket toward operational missions, Ars reports.
Those are lofty goals … “We just passed 400 launches on Falcon, and I would not be surprised if we fly 400 Starship launches in the next four years,” Shotwell said at the Baron Investment Conference in New York City. “We want to fly it a lot.” That lofty goal seems aspirational, not just because of the hardware challenges but also due to the ground systems (SpaceX currently has just one operational launch tower) as well as the difficulty of supplying that much liquid oxygen and methane for such a high flight rate. However, it’s worth noting that SpaceX will launch Starship four times this year, twice the number of Falcon Heavy missions. An acceleration of Starship is highly likely.
AST signs launch deals for its BlueBird constellation. During a third-quarter earnings call, AST SpaceMobile revealed new launch agreements with Blue Origin, the Indian Space Research Organization, and SpaceX to launch its large satellites over the course of 2025 and 2026, Spaceflight Now reports. Andrew Johnson, chief financial officer and chief legal officer at AST SpaceMobile, said that the launches “enable us to launch up to approximately 45 Block 2 BlueBird satellites, with options for additional launch vehicles for approximately 60 Block 2 BlueBird satellites.”
Glenns and Falcons … The company’s next launch will use India’s Geosynchronous Satellite Launch Vehicle. After that, the company will shift its focus to launching with Blue Origin’s New Glenn rocket and SpaceX’s Falcon 9, which are capable of carrying eight and four Block 2 BlueBird satellites, respectively. The company said its Block 2 constellation will be capable of delivering “peak data transmission speeds up to 120Mbps, supporting voice, full data, and video applications.” AST will be competing with SpaceX’s Starlink constellation in providing direct-to-cell communications. (submitted by EllPeaTea)
FAA gives SpaceX a green light for South Texas launches. A day after SpaceX launched its Starship rocket for the sixth time, the company received good news from the Federal Aviation Administration regarding future launch operations from its Starbase facility in South Texas. In a draft version of what is known as an “Environmental Assessment,” the FAA indicated that it will grant SpaceX permission to increase the number of Starship launches in South Texas to 25 per year from the current limit of five. Additionally, the company will likely be allowed to continue increasing the size and power of the Super Heavy booster stage and Starship upper stage, Ars reports.
A final decision is coming next year … The FAA regulates the launch of rockets from the United States and is responsible for the safety of people and property on the ground. The ongoing environmental review stems from SpaceX’s desire to increase the scope of its operations from South Texas and is not yet finalized. Beginning today, the FAA will open a public comment period that will close on January 17. In addition, the FAA will hold five public meetings to solicit feedback from the local community and other stakeholders. A final assessment will likely be issued sometime early next year.
ESA wants a reusable super heavy lift rocket. The European Space Agency has announced that it will commission a study to detail the development of a reusable rocket capable of delivering 60 tons to low-Earth orbit, European Spaceflight reports. The space agency believes it is necessary to have a launch system of this kind to fulfill “critical European space exploration needs beyond LEO, while providing wider space exploitation potentials to answer the growing market opportunities (e.g. mega constellations).”
Studies of studies … The agency launched its PROTEIN (Preparatory Activities for European Heavy Lift Launcher) initiative in June 2022, aiming to explore the feasibility of developing a European super heavy-lift rocket with a focus on reducing launch costs. ArianeGroup and Rocket Factory Augsburg were selected to lead studies. The European 60T LEO Reusable Launch System Pathfinder initiative seems to build upon the agency’s PROTEIN studies, even though this link is not explicitly stated. (submitted by EllPeaTea)
Next three launches
Nov. 22: New Shepard | NS-28 | Launch Site One, Texas | 15: 30 UTC
Nov. 24: Falcon 9 | Starlink 9-13 | Vandenberg Space Force Base, California | 03: 26 UTC
Nov. 24: Electron | Ice AIS Baby | Māhia Peninsula, New Zealand | 03: 55 UTC
Eric Berger is the senior space editor at Ars Technica, covering everything from astronomy to private space to NASA policy, and author of two books: Liftoff, about the rise of SpaceX; and Reentry, on the development of the Falcon 9 rocket and Dragon. A certified meteorologist, Eric lives in Houston.
Depending on who you speak to, technological sovereignty is either a hot topic, or something that other organizations need to deal with. So, should it matter to you and your organization? Let’s first consider what’s driving it, not least the crystal in the solute of the US Cloud Act, which ostensibly gives the US government access to any data managed by a US provider. This spooked EU authorities and nations, as well as others who saw it as a step too far.
Whilst this accelerated activity across Europe, Africa and other continents, moves were already afoot to preserve a level of sovereignty across three axes: data movement, local control, and what is increasingly seen as the big one – a desire for countries to develop and retain skills and innovate, rather than being passive participants in a cloud-based brain drain.
This is impacting not just government departments and their contractors, but also suppliers to in-country companies. A couple of years ago, I spoke to a manufacturing materials organization in France that provided goods to companies in Nigeria. “What’s your biggest headache,” I asked the CIO as a conversation starter. “Sovereignty,” he said. “If I can’t show my clients how I will keep data in-country, I can’t supply my goods.”
Such themes like the Cloud Act have made cross-border data management tricky. With different countries enforcing different laws, navigating where and how your data is stored has become a significant challenge. If it matters to you, it really matters. In principle, technological sovereignty solves this, but there’s no single, clear definition. It’s a concept that’s easy to understand at a high level, but tricky to pin down.
Technological sovereignty is all about ensuring you have control over your digital assets—your data, infrastructure, and the systems that run your business. But it’s not just about knowing where your data is stored. It’s about making sure that data is handled in a way that aligns with the country’s regulations and your business strategy and values.
For organizations in Europe, the rules and regs are quite specific. The upcoming EU Data Act focuses on data sharing and access across different sectors, whilst the AI Act introduces rules around artificial intelligence systems. Together, these evolving regulations are pushing organizations to rethink their technology architectures and data management strategies.
As ever, this means changing the wheels on a moving train. Hybrid/multi-cloud environments and complex data architectures add layers of complexity, whilst artificial intelligence is transforming how we interact with and manage data. AI is a sovereignty blessing and a curse – it can both enable data to be handled more effectively, but as AI models become more sophisticated, organizations need to be even more careful about how they process data from a compliance perspective.
So, where does this leave organizations that want the flexibility of cloud services but need to maintain control over their data? Organizations have several options:
Sovereign Hyper-Scalers: Over the next year, cloud giants like AWS and Azure will be rolling out sovereign cloud offerings tailored to the needs of organizations that require stricter data controls.
Localized Providers: Working with local managed service providers (MSPs) can give organizations more control within their own country or region, helping them keep data close to home.
On-premise Solutions: This is the go-to option if you want full control. However, on-premise solutions can be costly and come with their own set of complexities. It’s about balancing control with practicality.
The likelihood is a combination of all three will be required, at least in the short-medium term. Inertia will play its part: given that it’s already a challenge to move existing workloads beyond the lower-hanging fruit into the cloud, sovereignty creates yet another series of reasons to leave them where they are, for better or worse.
There’s a way forward for sovereignty as both a goal and a burden, centered on the word governance. Good governance is about setting clear policies for how your data and systems are managed, who has access, and how you stay compliant with regulations for both your organization and your customers. This is a business-wide responsibility: every level of your organization should be aligned on what sovereignty means for your company and how you will enforce it.
This may sound onerous to the point of impossibility, but that is the nature of governance, compliance and risk (GRC) – the trick is to assess, prioritize and plan, building sovereignty criteria into the way the business is designed. Want to do business in certain jurisdictions? If so, you need to bake their requirements into your business policies, which can then be rolled out into your application, data and operational policies.
Get this the other way around, and it will always be harder than necessary. However, done right, technological sovereignty can also offer a competitive advantage. Organizations with a handle on their data and systems can offer their customers more security and transparency, building trust. By embedding sovereignty into your digital strategy, you’re not just protecting your organization—you’re positioning yourself as a leader in responsible business, and building a stronger foundation for growth and innovation.
Technological sovereignty should be a strategic priority for any organization that wants to stay ahead in today’s complex digital landscape. It’s not just about choosing the right cloud provider or investing in the latest security tools—it’s about building a long-term, business-driven strategy that ensures you stay in control of your data, wherever in the world it is.
The future of sovereignty is about balance. Balancing cloud and on-premise solutions, innovation and control, and security with flexibility. If you can get that balance right, you’ll be in a strong position to navigate whatever the digital world throws at you next.
Anthropic, founded by former OpenAI executives Dario and Daniela Amodei in 2021, will continue using Google’s cloud services along with Amazon’s infrastructure. The UK Competition and Markets Authority reviewed Amazon’s partnership with Anthropic earlier this year and ultimately determined it did not have jurisdiction to investigate further, clearing the way for the partnership to continue.
Shaking the money tree
Amazon’s renewed investment in Anthropic also comes during a time of intense competition between cloud providers Amazon, Microsoft, and Google. Each company has made strategic partnerships with AI model developers—Microsoft with OpenAI (to the tune of $13 billion), Google with Anthropic (committing $2 billion over time), for example. These investments also encourage the use of each company’s data centers as demand for AI grows.
The size of these investments reflects the current state of AI development. OpenAI raised an additional $6.6 billion in October, potentially valuing the company at $157 billion. Anthropic has been eyeballing a $40 billion valuation during a recent investment round.
Training and running AI models is very expensive. While Google and Meta have their own profitable mainline businesses that can subsidize AI development, dedicated AI firms like OpenAI and Anthropic need constant infusions of cash to stay afloat—in other words, this won’t be the last time we hear of billion-dollar-scale AI investments from Big Tech.
Users will be asked to reauthenticate with Windows Hello every time they access their Recall database. Credit: Microsoft
Microsoft has now delayed the feature multiple times to address those concerns, and it outlined multiple security-focused additions to Recall in a blog post in September. Among other changes, the feature is now opt-in by default and is protected by additional encryption. Users must also re-authenticate with Windows Hello each time they access the database. Turning on the feature requires Secure Boot, BitLocker disk encryption, and Windows Hello to be enabled. In addition to the manual exclusion lists for sites and apps, the new Recall also attempts to mask sensitive data like passwords and credit card numbers so they aren’t stored in the Recall database.
The new version of Recall can also be completely uninstalled for users who have no interest in it, or by IT administrators who don’t want to risk it exposing sensitive data.
Testers will need to kick the tires on all of these changes to make sure that they meaningfully address all the risks and issues that the original version of Recall had, and this Windows Insider preview is their chance to do it.
“Do security”
Part of the original Recall controversy was that Microsoft wasn’t going to run it through the usual Windows Insider process—it was intended to be launched directly to users of the new Copilot+ PCs via a day-one software update. This in itself was a big red flag; usually, even features as small as spellcheck for the Notepad app go through multiple weeks of Windows Insider testing before Microsoft releases them to the public. This gives the company a chance to fix bugs, collect and address user feedback, and even scrub new features altogether.
Microsoft is supposedly re-orienting itself to put security over all other initiatives and features. CEO Satya Nadella recently urged employees to “do security” when presented with the option to either launch something quickly or launch something securely. In Recall’s case, the company’s rush to embrace generative AI features almost won out over that “do security” mandate. If future AI features go through the typical Windows Insider testing process first, that will be a sign that Microsoft is taking its commitment to security seriously.
