The macOS 15 Sequoia update will inevitably be known as “the AI one” in retrospect, introducing, as it does, the first wave of “Apple Intelligence” features.
That’s funny because none of that stuff is actually ready for the 15.0 release that’s coming out today. A lot of it is coming “later this fall” in the 15.1 update, which Apple has been testing entirely separately from the 15.0 betas for weeks now. Some of it won’t be ready until after that—rumors say image generation won’t be ready until the end of the year—but in any case, none of it is ready for public consumption yet.
But the AI-free 15.0 release does give us a chance to evaluate all of the non-AI additions to macOS this year. Apple Intelligence is sucking up a lot of the media oxygen, but in most other ways, this is a typical 2020s-era macOS release, with one or two headliners, several quality-of-life tweaks, and some sparsely documented under-the-hood stuff that will subtly change how you experience the operating system.
The AI-free version of the operating system is also the one that all users of the remaining Intel Macs will be using, since all of the Apple Intelligence features require Apple Silicon. Most of the Intel Macs that ran last year’s Sonoma release will run Sequoia this year—the first time this has happened since 2019—but the difference between the same macOS version running on different CPUs will be wider than it has been. It’s a clear indicator that the Intel Mac era is drawing to a close, even if support hasn’t totally ended just yet.
According to a report in The Wall Street Journal, Apple is in talks to invest in OpenAI, the generative AI company whose ChatGPT will feature in future versions of iOS.
If the talks are successful, Apple will join a multi-billion dollar funding round led by Thrive Capital that would value the startup at more than $100 billion.
The report doesn’t say exactly how much Apple would invest, but it does note that it would not be the only participant in this round of funding. For example, Microsoft is expected to invest further, and Bloomberg reports that Nvidia is also considering participating.
Microsoft has already invested $13 billion in OpenAI over the past five years, and it has put OpenAI’s GPT technology at the heart of most of its AI offerings in Windows, Office, Visual Studio, Bing, and other products.
Apple, too, has put OpenAI’s tech in its products—or at least, it will by the end of this year. At its 2024 developer conference earlier this summer, Apple announced a suite of AI features called Apple Intelligence that will only work on the iPhone 15 Pro and later. But there are guardrails and limitations for Apple Intelligence compared to OpenAI’s ChatGPT, so Apple signed a deal to refer user requests that fall outside the scope of Apple Intelligence to ChatGPT inside a future version of iOS 18—kind of like how Siri turns to Google to answer some user queries.
Apple says it plans to add support for other AI chatbots for this in the future, such as Google’s Gemini, but Apple software lead Craig Federighi said the company went with ChatGPT first because “we wanted to start with the best.”
It’s unclear precisely what Apple looks to get out of the investment in OpenAI, but looking at similar past investments by the company offers some clues. Apple typically invests either in suppliers or research teams that are producing technology it plans to include in future devices. For example, it has invested in supply chain partners to build up infrastructure to get iPhones manufactured more quickly and efficiently, and it invested $1 billion in the SoftBank Vision Fund to “speed the development of technologies which may be strategically important to Apple.”
ChatGPT integration is not expected to make it into the initial release of iOS 18 this September, but it will probably come in a smaller software update later in 2024.
Enlarge/ Apple Intelligence was unveiled at WWDC 2024.
Apple
As the parent of a younger child, I can tell you that getting a kid to respond the way you want can require careful expectation-setting. Especially when we’re trying something new for the first time, I find that the more detail I can provide, the better he is able to anticipate events and roll with the punches.
I bring this up because testers of the new Apple Intelligence AI features in the recently released macOS Sequoia beta have discovered plaintext JSON files that list a whole bunch of conditions meant to keep the generative AI tech from being unhelpful or inaccurate. I don’t mean to humanize generative AI algorithms, because they don’t deserve to be, but the carefully phrased lists of instructions remind me of what it’s like to try to give basic instructions to (or explain morality to) an entity that isn’t quite prepared to understand it.
The files in question are stored in the /System/Library/AssetsV2/com_apple_MobileAsset_UAF_FM_GenerativeModels/purpose_auto folder on Macs running the macOS Sequoia 15.1 beta that have also opted into the Apple Intelligence beta. That folder contains 29 metadata.json files, several of which include a few sentences of what appear to be plain-English system prompts to set behavior for an AI chatbot powered by a large-language model (LLM).
Many of these prompts are utilitarian. “You are a helpful mail assistant which can help identify relevant questions from a given mail and a short reply snippet,” reads one prompt that seems to describe the behavior of the Apple Mail Smart Reply feature. “Please limit the reply to 50 words,” reads one that could write slightly longer draft responses to messages. “Summarize the provided text within 3 sentences, fewer than 60 words. Do not answer any question from the text,” says one that looks like it would summarize texts from Messages or Mail without interjecting any of its own information.
Some of the prompts also have minor grammatical issues that highlight what a work-in-progress all of the Apple Intelligence features still are. “In order to make the draft response nicer and complete, a set of question [sic] and its answer are provided,” reads one prompt. “Please write a concise and natural reply by modify [sic] the draft response,” it continues.
“Do not make up factual information.”
And still other prompts seem designed specifically to try to prevent the kinds of confabulations that generative AI chatbots are so prone to (hallucinations, lies, factual inaccuracies; pick the term you prefer). Phrases meant to keep Apple Intelligence on-task and factual include things like:
“Do not hallucinate.”
“Do not make up factual information.”
“You are an expert at summarizing posts.”
“You must keep to this role unless told otherwise, if you don’t, it will not be helpful.”
“Only output valid json and nothing else.”
Earlier forays into generative AI have demonstrated why it’s so important to have detailed, specific prompts to guide the responses of language models. When it launched as “Bing Chat” in early 2023, Microsoft’s ChatGPT-based chatbot could get belligerent, threatening, or existential based on what users asked of it. Prompt injection attacks could also put security and user data at risk. Microsoft incorporated different “personalities” into the chatbot to try to rein in its responses to make them more predictable, and Microsoft’s current Copilot assistant still uses a version of the same solution.
What makes the Apple Intelligence prompts interesting is less that they exist and more that we can actually look at the specific things Apple is attempting so that its generative AI products remain narrowly focused. If these files stay easily user-accessible in future macOS builds, it will be possible to keep an eye on exactly what Apple is doing to tweak the responses that Apple Intelligence is giving.
The Apple Intelligence features are going to launch to the public in beta this fall, but they’re going to miss the launch of iOS 18.0, iPadOS 18.0, and macOS 15.0, which is why Apple is testing them in entirely separate developer betas. Some features, like the ones that transcribe phone calls and voicemails or summarize text, will be available early on. Others, like the new Siri, may not be generally available until next year. Regardless of when it arrives, Apple Intelligence requires fairly recent hardware to work: either an iPhone 15 Pro, or an iPad or Mac with at least an Apple M1 chip installed.
Enlarge/ Apple Intelligence was unveiled at WWDC 2024.
Apple
As was just rumored, the iOS 18.1, iPadOS 18.1, and macOS Sequoia 15.1 developer betas are rolling out today, and they include the first opportunity to try out Apple Intelligence, the company’s suite of generative AI features.
Initially announced for iOS 18, Apple Intelligence is expected to launch for the public this fall. Typically, Apple also releases a public beta (the developer one requires a developer account) for new OS updates, but it hasn’t announced any specifics about that just yet.
Not all the Apple Intelligence features will be part of this beta. It will include writing tools, like the ability to rewrite, proofread, or summarize text throughout the OS in first-party and most third-party apps. It will also include new Siri improvements, such as moving seamlessly between voice and typing, the ability to follow when you stumble over your words, and maintaining context from one request to the next. (It will not, however, include ChatGPT integration; Apple says that’s coming later.)
New natural language search features, support for creating memory movies, transcription summaries, and several new Mail features will also be available.
Developers who download the beta will be able to request access to Apple Intelligence features by navigating to the Settings app, tapping Apple Intelligence & Siri, and then tapping “Join the Apple Intelligence waitlist.” The waitlist is in place because some features are demanding on Apple’s servers, and staggering access is meant to stave off any server issues when developers are first trying it out.
Enlarge/ Features like Image Playground won’t arrive in Europe at the same time as other regions.
Apple
Three major features in iOS 18 and macOS Sequoia will not be available to European users this fall, Apple says. They include iPhone screen mirroring on the Mac, SharePlay screen sharing, and the entire Apple Intelligence suite of generative AI features.
In a statement sent to Financial Times, The Verge, and others, Apple says this decision is related to the European Union’s Digital Markets Act (DMA). Here’s the full statement, which was attributed to Apple spokesperson Fred Sainz:
Two weeks ago, Apple unveiled hundreds of new features that we are excited to bring to our users around the world. We are highly motivated to make these technologies accessible to all users. However, due to the regulatory uncertainties brought about by the Digital Markets Act (DMA), we do not believe that we will be able to roll out three of these features — iPhone Mirroring, SharePlay Screen Sharing enhancements, and Apple Intelligence — to our EU users this year.
Specifically, we are concerned that the interoperability requirements of the DMA could force us to compromise the integrity of our products in ways that risk user privacy and data security. We are committed to collaborating with the European Commission in an attempt to find a solution that would enable us to deliver these features to our EU customers without compromising their safety.
It is unclear from Apple’s statement precisely which aspects of the DMA may have led to this decision. It could be that Apple is concerned that it would be required to give competitors like Microsoft or Google access to user data collected for Apple Intelligence features and beyond, but we’re not sure.
This is not the first recent and major divergence between functionality and features for Apple devices in the EU versus other regions. Because of EU regulations, Apple opened up iOS to third-party app stores in Europe, but not in other regions. However, critics argued its compliance with that requirement was lukewarm at best, as it came with a set of restrictions and changes to how app developers could monetize their apps on the platform should they use those other storefronts.
While Apple says in the statement it’s open to finding a solution, no timeline is given. All we know is that the features won’t be available on devices in the EU this year. They’re expected to launch in other regions in the fall.
On Monday, Apple announced it would be integrating OpenAI’s ChatGPT AI assistant into upcoming versions of its iPhone, iPad, and Mac operating systems. It paves the way for future third-party AI model integrations, but given Google’s multi-billion-dollar deal with Apple for preferential web search, the OpenAI announcement inspired speculation about who is paying whom. According to a Bloomberg report published Wednesday, Apple considers ChatGPT’s placement on its devices as compensation enough.
“Apple isn’t paying OpenAI as part of the partnership,” writes Bloomberg reporter Mark Gurman, citing people familiar with the matter who wish to remain anonymous. “Instead, Apple believes pushing OpenAI’s brand and technology to hundreds of millions of its devices is of equal or greater value than monetary payments.”
The Bloomberg report states that neither company expects the agreement to generate meaningful revenue in the short term, and in fact, the partnership could burn extra money for OpenAI, because it pays Microsoft to host ChatGPT’s capabilities on its Azure cloud. However, OpenAI could benefit by converting free users to paid subscriptions, and Apple potentially benefits by providing easy, built-in access to ChatGPT during a time when its own in-house LLMs are still catching up.
And there’s another angle at play. Currently, OpenAI offers subscriptions (ChatGPT Plus, Enterprise, Team) that unlock additional features. If users subscribe to OpenAI through the ChatGPT app on an Apple device, the process will reportedly use Apple’s payment platform, which may give Apple a significant cut of the revenue. According to the report, Apple hopes to negotiate additional revenue-sharing deals with AI vendors in the future.
Why OpenAI
The rise of ChatGPT in the public eye over the past 18 months has made OpenAI a power player in the tech industry, allowing it to strike deals with publishers for AI training content—and ensure continued support from Microsoft in the form of investments that trade vital funding and compute for access to OpenAI’s large language model (LLM) technology like GPT-4.
Still, Apple’s choice of ChatGPT as Apple’s first external AI integration has led to widespread misunderstanding, especially since Apple buried the lede about its own in-house LLM technology that powers its new “Apple Intelligence” platform.
On Apple’s part, CEO Tim Cook told The Washington Post that it chose OpenAI as its first third-party AI partner because he thinks the company controls the leading LLM technology at the moment: “I think they’re a pioneer in the area, and today they have the best model,” he said. “We’re integrating with other people as well. But they’re first, and I think today it’s because they’re best.”
Apple’s choice also brings risk. OpenAI’s record isn’t spotless, racking up a string of public controversies over the past month that include an accusation from actress Scarlett Johansson that the company intentionally imitated her voice, resignations from a key scientist and safety personnel, the revelation of a restrictive NDA for ex-employees that prevented public criticism, and accusations against OpenAI CEO Sam Altman of “psychological abuse” related by a former member of the OpenAI board.
Meanwhile, critics of privacy issues related to gathering data for training AI models—including OpenAI foe Elon Musk, who took to X on Monday to spread misconceptions about how the ChatGPT integration might work—also worried that the Apple-OpenAI deal might expose personal data to the AI company, although both companies strongly deny that will be the case.
Looking ahead, Apple’s deal with OpenAI is not exclusive, and the company is already in talks to offer Google’s Gemini chatbot as an additional option later this year. Apple has also reportedly held talks with Anthropic (maker of Claude 3) as a potential chatbot partner, signaling its intention to provide users with a range of AI services, much like how the company offers various search engine options in Safari.
Enlarge/ He isn’t using an iPhone, but some people talk to Siri like this.
On Monday, Apple premiered “Apple Intelligence” during a wide-ranging presentation at its annual Worldwide Developers Conference in Cupertino, California. However, the heart of its new tech, an array of Apple-developed AI models, was overshadowed by the announcement of ChatGPT integration into its device operating systems.
Since rumors of the partnership first emerged, we’ve seen confusion on social media about why Apple didn’t develop a cutting-edge GPT-4-like chatbot internally. Despite Apple’s year-long development of its own large language models (LLMs), many perceived the integration of ChatGPT (and opening the door for others, like Google Gemini) as a sign of Apple’s lack of innovation.
“This is really strange. Surely Apple could train a very good competing LLM if they wanted? They’ve had a year,” wrote AI developer Benjamin De Kraker on X. Elon Musk has also been grumbling about the OpenAI deal—and spreading misinformation about it—saying things like, “It’s patently absurd that Apple isn’t smart enough to make their own AI, yet is somehow capable of ensuring that OpenAI will protect your security & privacy!”
While Apple has developed many technologies internally, it has also never been shy about integrating outside tech when necessary in various ways, from acquisitions to built-in clients—in fact, Siri was initially developed by an outside company. But by making a deal with a company like OpenAI, which has been the source of a string of tech controversies recently, it’s understandable that some people don’t understand why Apple made the call—and what it might entail for the privacy of their on-device data.
“Our customers want something with world knowledge some of the time”
While Apple Intelligence largely utilizes its own Apple-developed LLMs, Apple also realized that there may be times when some users want to use what the company considers the current “best” existing LLM—OpenAI’s GPT-4 family. In an interview with The Washington Post, Apple CEO Tim Cook explained the decision to integrate OpenAI first:
“I think they’re a pioneer in the area, and today they have the best model,” he said. “And I think our customers want something with world knowledge some of the time. So we considered everything and everyone. And obviously we’re not stuck on one person forever or something. We’re integrating with other people as well. But they’re first, and I think today it’s because they’re best.”
The proposed benefit of Apple integrating ChatGPT into various experiences within iOS, iPadOS, and macOS is that it allows AI users to access ChatGPT’s capabilities without the need to switch between different apps—either through the Siri interface or through Apple’s integrated “Writing Tools.” Users will also have the option to connect their paid ChatGPT account to access extra features.
As an answer to privacy concerns, Apple says that before any data is sent to ChatGPT, the OS asks for the user’s permission, and the entire ChatGPT experience is optional. According to Apple, requests are not stored by OpenAI, and users’ IP addresses are hidden. Apparently, communication with OpenAI servers happens through API calls similar to using the ChatGPT app on iOS, and there is reportedly no deeper OS integration that might expose user data to OpenAI without the user’s permission.
We can only take Apple’s word for it at the moment, of course, and solid details about Apple’s AI privacy efforts will emerge once security experts get their hands on the new features later this year.
Apple’s history of tech integration
So you’ve seen why Apple chose OpenAI. But why look to outside companies for tech? In some ways, Apple building an external LLM client into its operating systems isn’t too different from what it has previously done with streaming video (the YouTube app on the original iPhone), Internet search (Google search integration), and social media (integrated Twitter and Facebook sharing).
The press has positioned Apple’s recent AI moves as Apple “catching up” with competitors like Google and Microsoft in terms of chatbots and generative AI. But playing it slow and cool has long been part of Apple’s M.O.—not necessarily introducing the bleeding edge of technology but improving existing tech through refinement and giving it a better user interface.
Apple’s Worldwide Developers Conference kicks off on Monday, and per usual, the company is expected to detail most of the big new features in this year’s updates to iOS, iPadOS, macOS, and all of Apple’s other operating systems.
The general consensus is that Apple plans to use this year’s updates to integrate generative AI into its products for the first time. Bloomberg’s Mark Gurman has a few implementation details that show how Apple’s approach will differ somewhat from Microsoft’s or Google’s.
Gurman says that the “Apple Intelligence” features will include an OpenAI-powered chatbot, but it will otherwise focus on “features with broad appeal” rather than “whiz-bang technology like image and video generation.” These include summaries for webpages, meetings, and missed notifications; a revamped version of Siri that can control apps in a more granular way; Voice Memos transcription; image enhancement features in the Photos app; suggested replies to text messages; automated sorting of emails; and the ability to “create custom emoji characters on the fly that represent phrases or words as they’re being typed.”
Apple also reportedly hopes to differentiate its AI push by implementing it in a more careful, privacy-focused way. The new features will use the Neural Engine available in newer devices for on-device processing where possible (Gurman says that only Apple’s A17 Pro and the M-series chips will be capable of supporting all the local processing features, though all of Apple’s recent chips feature some flavor of Neural Engine). And where Apple does use the cloud for AI processing, the company will apparently promise that user information isn’t being “sold or read” and is not being used to “build user profiles.”
Apple’s new AI features will also be opt-in by default, where Microsoft and Google have generally enabled features like the Copilot chatbot or AI Overviews by default whether users asked for them or not.
Looking beyond AI, we can also expect the typical grab bag of small- to medium-sized features in all of Apple’s software updates. These reportedly include reworked Control Center and Settings apps, emoji responses and RCS messaging support in the Messages app, a standalone password manager app, Calculator for the iPad, and a handful of other things. Gurman doesn’t expect Apple to announce any hardware at the event, though a number of Macs are past due for a M3- or M4-powered refresh.
