Biz & IT

microsoft’s-new-“copilot-vision”-ai-experiment-can-see-what-you-browse

Microsoft’s new “Copilot Vision” AI experiment can see what you browse

On Monday, Microsoft unveiled updates to its consumer AI assistant Copilot, introducing two new experimental features for a limited group of $20/month Copilot Pro subscribers: Copilot Labs and Copilot Vision. Labs integrates OpenAI’s latest o1 “reasoning” model, and Vision allows Copilot to see what you’re browsing in Edge.

Microsoft says Copilot Labs will serve as a testing ground for Microsoft’s latest AI tools before they see wider release. The company describes it as offering “a glimpse into ‘work-in-progress’ projects.” The first feature available in Labs is called “Think Deeper,” and it uses step-by-step processing to solve more complex problems than the regular Copilot. Think Deeper is Microsoft’s version of OpenAI’s new o1-preview and o1-mini AI models, and it has so far rolled out to some Copilot Pro users in Australia, Canada, New Zealand, the UK, and the US.

Copilot Vision is an entirely different beast. The new feature aims to give the AI assistant a visual window into what you’re doing within the Microsoft Edge browser. When enabled, Copilot can “understand the page you’re viewing and answer questions about its content,” according to Microsoft.

Microsoft’s Copilot Vision promo video.

The company positions Copilot Vision as a way to provide more natural interactions and task assistance beyond text-based prompts, but it will likely raise privacy concerns. As a result, Microsoft says that Copilot Vision is entirely opt-in and that no audio, images, text, or conversations from Vision will be stored or used for training. The company is also initially limiting Vision’s use to a pre-approved list of websites, blocking it on paywalled and sensitive content.

The rollout of these features appears gradual, with Microsoft noting that it wants to balance “pioneering features and a deep sense of responsibility.” The company said it will be “listening carefully” to user feedback as it expands access to the new capabilities. Microsoft has not provided a timeline for wider availability of either feature.

Mustafa Suleyman, chief executive of Microsoft AI, told Reuters that he sees Copilot as an “ever-present confidant” that could potentially learn from users’ various Microsoft-connected devices and documents, with permission. He also mentioned that Microsoft co-founder Bill Gates has shown particular interest in Copilot’s potential to read and parse emails.

But judging by the visceral reaction to Microsoft’s Recall feature, which keeps a record of everything you do on your PC so an AI model can recall it later, privacy-sensitive users may not appreciate having an AI assistant monitor their activities—especially if those features send user data to the cloud for processing.

Microsoft’s new “Copilot Vision” AI experiment can see what you browse Read More »

attackers-exploit-critical-zimbra-vulnerability-using-cc’d-email-addresses

Attackers exploit critical Zimbra vulnerability using cc’d email addresses

Attackers are actively exploiting a critical vulnerability in mail servers sold by Zimbra in an attempt to remotely execute malicious commands that install a backdoor, researchers warn.

The vulnerability, tracked as CVE-2024-45519, resides in the Zimbra email and collaboration server used by medium and large organizations. When an admin manually changes default settings to enable the postjournal service, attackers can execute commands by sending maliciously formed emails to an address hosted on the server. Zimbra recently patched the vulnerability. All Zimbra users should install it or, at a minimum, ensure that postjournal is disabled.

Easy, yes, but reliable?

On Tuesday, Security researcher Ivan Kwiatkowski first reported the in-the-wild attacks, which he described as “mass exploitation.” He said the malicious emails were sent by the IP address 79.124.49[.]86 and, when successful, attempted to run a file hosted there using the tool known as curl. Researchers from security firm Proofpoint took to social media later that day to confirm the report.

On Wednesday, security researchers provided additional details that suggested the damage from ongoing exploitation was likely to be contained. As already noted, they said, a default setting must be changed, likely lowering the number of servers that are vulnerable.

Security researcher Ron Bowes went on to report that the “payload doesn’t actually do anything—it downloads a file (to stdout) but doesn’t do anything with it.” He said that in the span of about an hour earlier Wednesday a honey pot server he operated to observe ongoing threats received roughly 500 requests. He also reported that the payload isn’t delivered through emails directly, but rather through a direct connection to the malicious server through SMTP, short for the Simple Mail Transfer Protocol.

“That’s all we’ve seen (so far), it doesn’t really seem like a serious attack,” Bowes wrote. “I’ll keep an eye on it, and see if they try anything else!”

In an email sent Wednesday afternoon, Proofpoint researcher Greg Lesnewich seemed to largely concur that the attacks weren’t likely to lead to mass infections that could install ransomware or espionage malware. The researcher provided the following details:

  • While the exploitation attempts we have observed were indiscriminate in targeting, we haven’t seen a large volume of exploitation attempts
  • Based on what we have researched and observed, exploitation of this vulnerability is very easy, but we do not have any information about how reliable the exploitation is
  • Exploitation has remained about the same since we first spotted it on Sept. 28th
  • There is a PoC available, and the exploit attempts appear opportunistic
  • Exploitation is geographically diverse and appears indiscriminate
  • The fact that the attacker is using the same server to send the exploit emails and host second-stage payloads indicates the actor does not have a distributed set of infrastructure to send exploit emails and handle infections after successful exploitation. We would expect the email server and payload servers to be different entities in a more mature operation.
  • Defenders protecting  Zimbra appliances should look out for odd CC or To addresses that look malformed or contain suspicious strings, as well as logs from the Zimbra server indicating outbound connections to remote IP addresses.

Proofpoint has explained that some of the malicious emails used multiple email addresses that, when pasted into the CC field, attempted to install a webshell-based backdoor on vulnerable Zimbra servers. The full cc list was wrapped as a single string and encoded using the base64 algorithm. When combined and converted back into plaintext, they created a webshell at the path: /jetty/webapps/zimbraAdmin/public/jsp/zimbraConfig.jsp.

Attackers exploit critical Zimbra vulnerability using cc’d email addresses Read More »

openai-is-now-valued-at-$157-billion

OpenAI is now valued at $157 billion

OpenAI, the company behind ChatGPT, has now raised $6.6 billion in a new funding round that values the company at $157 billion, nearly doubling its previous valuation of $86 billion, according to a report from The Wall Street Journal.

The funding round comes with strings attached: Investors have the right to withdraw their money if OpenAI does not complete its planned conversion from a nonprofit (with a for-profit division) to a fully for-profit company.

Venture capital firm Thrive Capital led the funding round with a $1.25 billion investment. Microsoft, a longtime backer of OpenAI to the tune of $13 billion, contributed just under $1 billion to the latest round. New investors joined the round, including SoftBank with a $500 million investment and Nvidia with $100 million.

The United Arab Emirates-based company MGX also invested in OpenAI during this funding round. MGX has been busy in AI recently, joining an AI infrastructure partnership last month led by Microsoft.

Notably, Apple was in talks to invest but ultimately did not participate. WSJ reports that the minimum investment required to review OpenAI’s financial documents was $250 million. In June, OpenAI hired its first chief financial officer, Sarah Friar, who played an important role in organizing this funding round, according to the WSJ.

OpenAI is now valued at $157 billion Read More »

crook-made-millions-by-breaking-into-execs’-office365-inboxes,-feds-say

Crook made millions by breaking into execs’ Office365 inboxes, feds say

WHAT IS THE NAME OF YOUR FIRST PET? —

Email accounts inside 5 US companies unlawfully breached through password resets.

Crook made millions by breaking into execs’ Office365 inboxes, feds say

Getty Images

Federal prosecutors have charged a man for an alleged “hack-to-trade” scheme that earned him millions of dollars by breaking into the Office365 accounts of executives at publicly traded companies and obtaining quarterly financial reports before they were released publicly.

The action, taken by the office of the US Attorney for the district of New Jersey, accuses UK national Robert B. Westbrook of earning roughly $3.75 million in 2019 and 2020 from stock trades that capitalized on the illicitly obtained information. After accessing it, prosecutors said, he executed stock trades. The advance notice allowed him to act and profit on the information before the general public could. The US Securities and Exchange Commission filed a separate civil suit against Westbrook seeking an order that he pay civil penalties and return all ill-gotten gains.

Buy low, sell high

“The SEC is engaged in ongoing efforts to protect markets and investors from the consequences of cyber fraud,” Jorge G. Tenreiro, acting chief of the SEC’s Crypto Assets and Cyber Unit, said in a statement. “As this case demonstrates, even though Westbrook took multiple steps to conceal his identity—including using anonymous email accounts, VPN services, and utilizing bitcoin—the Commission’s advanced data analytics, crypto asset tracing, and technology can uncover fraud even in cases involving sophisticated international hacking.”

A federal indictment filed in US District Court for the District of New Jersey said that Westbrook broke into the email accounts of executives from five publicly traded companies in the US. He pulled off the breaches by abusing the password reset mechanism Microsoft offered for Office365 accounts. In some cases, Westbrook allegedly went on to create forwarding rules that automatically sent all incoming emails to an email address he controlled.

Prosecutors alleged in one such incident:

On or about January 26, 2019, WESTBROOK gained unauthorized access to the Office365 email account of Company-1 ‘s Director of Finance and Accounting (“Individual-!”) through an unauthorized password reset. During the intrusion, an auto-forwarding rule was implemented, which was designed to automatically forward content from lndividual-1 ‘s compromised email account to an email account controlled by WESTBROOK. At the time of the intrusion, the compromised email account of Individual-I contained non-public information about Company-1 ‘s quarterly earnings, which indicated that Company-1 ‘s sales were down.

Once a person gains unauthorized access to an email account, it’s possible to conceal the breach by disabling or deleting password reset alerts and burying password reset rules deep inside account settings.

Prosecutors didn’t say how the defendant managed to abuse the reset feature. Typically such mechanisms require control of a cell phone or registered email account belonging to the account holder. In 2019 and 2020 many online services would also allow users to reset passwords by answering security questions. The practice is still in use today but has been slowly falling out of favor as the risks have come to be more widely understood.

By obtaining material information, Westbrook was able to predict how a company’s stock would perform once it became public. When results were likely to drive down stock prices, he would place “put” options, which give the purchaser the right to sell shares at a specific price within a specified span of time. The practice allowed Westbrook to profit when shares fell after financial results became public. When positive results were likely to send stock prices higher, Westbrook allegedly bought shares while they were still low and later sold them for a higher price.

The prosecutors charged Westbrook with one count each of securities fraud and wire fraud and five counts of computer fraud. The securities fraud count carries a maximum penalty of up to 20 years’ prison time and $5 million in fines The wire fraud count carries a maximum penalty of up to 20 years in prison and a fine of either $250,000 or twice the gain or loss from the offense, whichever is greatest. Each computer fraud count carries a maximum five years in prison and a maximum fine of either $250,000 or twice the gain or loss from the offense, whichever is greatest.

The US Attorney’s office in the District of New Jersey didn’t say if Westbrook has made an initial appearance in court or if he has entered a plea.

Crook made millions by breaking into execs’ Office365 inboxes, feds say Read More »

openai-unveils-easy-voice-assistant-creation-at-2024-developer-event

OpenAI unveils easy voice assistant creation at 2024 developer event

Developers developers developers —

Altman steps back from the keynote limelight and lets four major API additions do the talking.

A glowing OpenAI logo on a blue background.

Benj Edwards

On Monday, OpenAI kicked off its annual DevDay event in San Francisco, unveiling four major API updates for developers that integrate the company’s AI models into their products. Unlike last year’s single-location event featuring a keynote by CEO Sam Altman, DevDay 2024 is more than just one day, adopting a global approach with additional events planned for London on October 30 and Singapore on November 21.

The San Francisco event, which was invitation-only and closed to press, featured on-stage speakers going through technical presentations. Perhaps the most notable new API feature is the Realtime API, now in public beta, which supports speech-to-speech conversations using six preset voices and enables developers to build features very similar to ChatGPT’s Advanced Voice Mode (AVM) into their applications.

OpenAI says that the Realtime API streamlines the process of creating voice assistants. Previously, developers had to use multiple models for speech recognition, text processing, and text-to-speech conversion. Now, they can handle the entire process with a single API call.

The company plans to add audio input and output capabilities to its Chat Completions API in the next few weeks, allowing developers to input text or audio and receive responses in either format.

Two new options for cheaper inference

OpenAI also announced two features that may help developers balance performance and cost when making AI applications. “Model distillation” offers a way for developers to fine-tune (customize) smaller, cheaper models like GPT-4o mini using outputs from more advanced models such as GPT-4o and o1-preview. This potentially allows developers to get more relevant and accurate outputs while running the cheaper model.

Also, OpenAI announced “prompt caching,” a feature similar to one introduced by Anthropic for its Claude API in August. It speeds up inference (the AI model generating outputs) by remembering frequently used prompts (input tokens). Along the way, the feature provides a 50 percent discount on input tokens and faster processing times by reusing recently seen input tokens.

And last but not least, the company expanded its fine-tuning capabilities to include images (what it calls “vision fine-tuning”), allowing developers to customize GPT-4o by feeding it both custom images and text. Basically, developers can teach the multimodal version of GPT-4o to visually recognize certain things. OpenAI says the new feature opens up possibilities for improved visual search functionality, more accurate object detection for autonomous vehicles, and possibly enhanced medical image analysis.

Where’s the Sam Altman keynote?

OpenAI CEO Sam Altman speaks during the OpenAI DevDay event on November 6, 2023, in San Francisco.

Enlarge / OpenAI CEO Sam Altman speaks during the OpenAI DevDay event on November 6, 2023, in San Francisco.

Getty Images

Unlike last year, DevDay isn’t being streamed live, though OpenAI plans to post content later on its YouTube channel. The event’s programming includes breakout sessions, community spotlights, and demos. But the biggest change since last year is the lack of a keynote appearance from the company’s CEO. This year, the keynote was handled by the OpenAI product team.

On last year’s inaugural DevDay, November 6, 2023, OpenAI CEO Sam Altman delivered a Steve Jobs-style live keynote to assembled developers, OpenAI employees, and the press. During his presentation, Microsoft CEO Satya Nadella made a surprise appearance, talking up the partnership between the companies.

Eleven days later, the OpenAI board fired Altman, triggering a week of turmoil that resulted in Altman’s return as CEO and a new board of directors. Just after the firing, Kara Swisher relayed insider sources that said Altman’s DevDay keynote and the introduction of the GPT store had been a precipitating factor in the firing (though not the key factor) due to some internal disagreements over the company’s more consumer-like direction since the launch of ChatGPT.

With that history in mind—and the focus on developers above all else for this event—perhaps the company decided it was best to let Altman step away from the keynote and let OpenAI’s technology become the key focus of the event instead of him. We are purely speculating on that point, but OpenAI has certainly experienced its share of drama over the past month, so it may have been a prudent decision.

Despite the lack of a keynote, Altman is present at Dev Day San Francisco today and is scheduled to do a closing “fireside chat” at the end (which has not yet happened as of this writing). Also, Altman made a statement about DevDay on X, noting that since last year’s DevDay, OpenAI had seen some dramatic changes (literally):

From last devday to this one:

*98% decrease in cost per token from GPT-4 to 4o mini

*50x increase in token volume across our systems

*excellent model intelligence progress

*(and a little bit of drama along the way)

In a follow-up tweet delivered in his trademark lowercase, Altman shared a forward-looking message that referenced the company’s quest for human-level AI, often called AGI: “excited to make even more progress from this devday to the next one,” he wrote. “the path to agi has never felt more clear.”

OpenAI unveils easy voice assistant creation at 2024 developer event Read More »

“extreme”-broadcom-proposed-price-hike-would-up-vmware-costs-1,050%,-at&t-says

“Extreme” Broadcom-proposed price hike would up VMware costs 1,050%, AT&T says

Legal dispute continues —

Broadcom “preventing some vendors from selling products to us,” AT&T alleges.

The logo of American cloud computing and virtualization technology company VMware is seen at the Mobile World Congress (MWC), the telecom industry's biggest annual gathering, in Barcelona on March 2, 2023.

Broadcom quoted AT&T a 1,050 percent price hike for VMware offerings, AT&T has claimed in legal documents.

AT&T sued Broadcom on August 29, accusing Broadcom of unlawfully denying it the second of three one-year renewals for support services that AT&T thinks it’s entitled to. AT&T cites a contract signed before Broadcom bought VMware. The telecommunications firm says it has 75,000 virtual machines (VMs) across approximately 8,600 servers running on VMware. Broadcom, which has stopped selling VMware perpetual licenses, has said that AT&T missed its opportunity to renew support and that the contract between VMware and AT&T has an “End of Availability” provision allowing VMware to retire products and services.

Legal filings from September 27 and spotted by The Register today show an email [PDF] that AT&T EVP and GM of wireline transformation and global supply chain Susan A. Johnson apparently sent to Broadcom CEO Hock Tan pointing to “an impasse” over VMware.

Johnson argued in the email that AT&T should have the right to renew support through September 2026 thanks to a previously signed five-year deal:

This proposed annual increase of +1,050% in one year is extreme and certainly not how we expect strategic partners to engage in doing business with AT&T.

A 1,050 percent price hike is the largest that Ars Technica has heard of being proposed by Broadcom. At this time, it’s unknown if AT&T’s claims are accurate. Broadcom hasn’t publicly commented on the allegations.

Many VMware customers have pointed to VMware becoming more expensive under Broadcom, though. Broadcom’s changes to selling VMware have reportedly included bundling products into only about two SKUs and higher CPU core requirements. In March, customers reportedly complained about price increases of up to 600 percent, per The Register. And in February, ServeTheHome said small cloud service providers reported prices increasing tenfold.

AT&T’s contract with VMware may be one of the firm’s bigger accounts. A 1,050 percent price hike would be another level, however, even for a company the size of AT&T. Per Johnson’s email, AT&T and Broadcom have had a “strategic relationship” for over a decade.

The email reads:

… AT&T has decided to pursue a legal strategy along with a disciplined plan to invest to migrate away, all of which will quickly become public. I truly wish we had another option. Unfortunately, this decision will impact the future of our overall relationship and how we manage spend in other Broadcom areas.

AT&T on potentially migrating off VMware

In her email, Johnson points to migration costs as impacting how much AT&T is willing to pay for VMware.

According to the message, projected costs for moving AT&T off of VMware are $40 million to $50 million. AT&T is said to use VMware-based VMs for customer services operations and for operations management efficiency. Per AT&T’s email, migration “has a very quick payback” and “strong” internal rate of return, “especially given the high licensing costs proposed.”

On September 20, Broadcom requested that AT&T’s request to block Broadcom from discontinuing VMware support be denied. In legal documents [PDF], Broadcom said that AT&T is planning to ditch VMware and that AT&T could have spent “the last several months or even years” making the transition.

In an affidavit filed on September 27 [PDF], Johnson stated that her email to Tan does not suggest that migration “would be easy, quick, or inexpensive” and that “none of those would be accurate statements.”

“My point was that although it is not easy, cheap, or quick to migrate off VMware, Defendants’ high fees will incentivize us to migrate to another solution,” the affidavit reads.

Johnson also claimed that AT&T started exploring options for getting off VMware in December but thought that it had time to make decisions, since it believed it could opt to renew support for its licenses until September 2026.

In another legal filing from September 27 [PDF], Gordon Mansfield, president of global technology planning at AT&T Services, says:

AT&T currently estimates it will take a period of years to transition all of its servers currently operating with the VMware software away from VMware. Moreover, Defendants have not made it easy to do so since we understand that they are preventing some vendors from selling certain products to us.

The filing didn’t get into further detail about how exactly Broadcom could be blocking product sales to AT&T. Broadcom hasn’t publicly responded to Mansfield’s claim.

Regarding AT&T’s lawsuit, Broadcom has previously told Ars Technica that it “strongly disagrees with the allegations and is confident we will prevail in the legal process.”

Since Broadcom’s VMware acquisition, most customers are expected to have at least considered ditching VMware. However, moving can be challenging and costly as some IT environments are heavily dependent on VMware. Being able to ensure that things are able to run as expected during the transition period has also complicated potential migrations.

While AT&T and Broadcom’s legal dispute continues, Broadcom has agreed to continue providing AT&T with VMware support until October 9. A preliminary injunction hearing is scheduled for October 15.

“Extreme” Broadcom-proposed price hike would up VMware costs 1,050%, AT&T says Read More »

systems-used-by-courts-and-governments-across-the-us-riddled-with-vulnerabilities

Systems used by courts and governments across the US riddled with vulnerabilities

SECURITY FAILURE —

With hundreds of courts and agencies affected, chances are one near you is, too.

Systems used by courts and governments across the US riddled with vulnerabilities

Getty Images

Public records systems that courts and governments rely on to manage voter registrations and legal filings have been riddled with vulnerabilities that made it possible for attackers to falsify registration databases and add, delete, or modify official documents.

Over the past year, software developer turned security researcher Jason Parker has found and reported dozens of critical vulnerabilities in no fewer than 19 commercial platforms used by hundreds of courts, government agencies, and police departments across the country. Most of the vulnerabilities were critical.

One flaw he uncovered in the voter registration cancellation portal for the state of Georgia, for instance, allowed anyone visiting it to cancel the registration of any voter in that state when the visitor knew the name, birthdate, and county of residence of the voter. In another case, document management systems used in local courthouses across the country contained multiple flaws that allowed unauthorized people to access sensitive filings such as psychiatric evaluations that were under seal. And in one case, unauthorized people could assign themselves privileges that are supposed to be available only to clerks of the court and, from there, create, delete, or modify filings.

Failing at the most fundamental level

It’s hard to overstate the critical role these systems play in the administration of justice, voting rights, and other integral government functions. The number of vulnerabilities—mostly stemming from weak permission controls, poor validation of user inputs, and faulty authentication processes—demonstrate a lack of due care in ensuring the trustworthiness of the systems millions of citizens rely on every day.

“These platforms are supposed to ensure transparency and fairness, but are failing at the most fundamental level of cybersecurity,” Parker wrote recently in a post he penned in an attempt to raise awareness. “If a voter’s registration can be canceled with little effort and confidential legal filings can be accessed by unauthorized users, what does it mean for the integrity of these systems?”

The vulnerability in the Georgia voter registration database, for instance, lacked any form of automated way to reject cancellation requests that omitted required voter information. Instead of flagging such requests, the system processed it without even flagging it. Similarly, the Granicus GovQA platform hundreds of government agencies use to manage public records could be hacked to reset passwords and gain access to usernames and email addresses simply by slightly modifying the Web address showing in a browser window.

And a vulnerability in the Thomson Reuters’ C-Track eFiling system allowed attackers to elevate their user status to that of a court administrator. Exploitation required nothing more than manipulating certain fields during the registration process.

There is no indication that any of the vulnerabilities were actively exploited.

Word of the vulnerabilities comes four months after the discovery of a malicious backdoor surreptitiously planted in a component of the JAVS Suite 8, an application package that 10,000 courtrooms around the world use to record, play back, and manage audio and video from legal proceedings. A representative of the company said Monday that an investigation performed in cooperation with the Cybersecurity and Infrastructure Security Agency concluded that the malware was installed on only two computers and didn’t result in any information being compromised. The representative said the malware was available through a file a threat actor posted to the JAVS public marketing website.

Parker began examining the systems last year as a software developer purely on a voluntary basis. He has worked with the Electronic Frontier Foundation to contact the system vendors and other parties responsible for the platforms he has found vulnerable. To date, all the vulnerabilities he has reported have been fixed, in some cases only in the past month. More recently, Parker has taken a job as a security researcher focusing on such platforms.

“Fixing these issues requires more than just patching a few bugs,” Parker wrote. “It calls for a complete overhaul of how security is handled in court and public record systems. To prevent attackers from hijacking accounts or altering sensitive data, robust permission controls must be immediately implemented, and stricter validation of user inputs enforced. Regular security audits and penetration testing should be standard practice, not an afterthought, and following the principles of Secure by Design should be an integral part of any Software Development Lifecycle.”

The 19 affected platforms are:

Parker is urging vendors and customers alike to shore up the security of their systems by performing penetration testing and software audits and training employees, particularly those in IT departments. He also said that multifactor authentication should be universally available for all such systems.

“This series of disclosures is a wake-up call to all organizations that manage sensitive public data,” Parker wrote. “If they fail to act quickly, the consequences could be devastating—not just for the institutions themselves but for the individuals whose privacy they are sworn to protect. For now, the responsibility lies with the agencies and vendors behind these platforms to take immediate action, to shore up their defenses, and to restore trust in the systems that so many people depend on.”

Systems used by courts and governments across the US riddled with vulnerabilities Read More »

man-tricks-openai’s-voice-bot-into-duet-of-the-beatles’-“eleanor-rigby”

Man tricks OpenAI’s voice bot into duet of The Beatles’ “Eleanor Rigby”

A screen capture of AJ Smith doing his Eleanor Rigby duet with OpenAI's Advanced Voice Mode through the ChatGPT app.

Enlarge / A screen capture of AJ Smith doing his Eleanor Rigby duet with OpenAI’s Advanced Voice Mode through the ChatGPT app.

OpenAI’s new Advanced Voice Mode (AVM) of its ChatGPT AI assistant rolled out to subscribers on Tuesday, and people are already finding novel ways to use it, even against OpenAI’s wishes. On Thursday, a software architect named AJ Smith tweeted a video of himself playing a duet of The Beatles’ 1966 song “Eleanor Rigby” with AVM. In the video, Smith plays the guitar and sings, with the AI voice interjecting and singing along sporadically, praising his rendition.

“Honestly, it was mind-blowing. The first time I did it, I wasn’t recording and literally got chills,” Smith told Ars Technica via text message. “I wasn’t even asking it to sing along.”

Smith is no stranger to AI topics. In his day job, he works as associate director of AI Engineering at S&P Global. “I use [AI] all the time and lead a team that uses AI day to day,” he told us.

In the video, AVM’s voice is a little quavery and not pitch-perfect, but it appears to know something about “Eleanor Rigby’s” melody when it first sings, “Ah, look at all the lonely people.” After that, it seems to be guessing at the melody and rhythm as it recites song lyrics. We have also convinced Advanced Voice Mode to sing, and it did a perfect melodic rendition of “Happy Birthday” after some coaxing.

AJ Smith’s video of singing a duet with OpenAI’s Advanced Voice Mode.

Normally, when you ask AVM to sing, it will reply something like, “My guidelines won’t let me talk about that.” That’s because in the chatbot’s initial instructions (called a “system prompt“), OpenAI instructs the voice assistant not to sing or make sound effects (“Do not sing or hum,” according to one system prompt leak).

OpenAI possibly added this restriction because AVM may otherwise reproduce copyrighted content, such as songs that were found in the training data used to create the AI model itself. That’s what is happening here to a limited extent, so in a sense, Smith has discovered a form of what researchers call a “prompt injection,” which is a way of convincing an AI model to produce outputs that go against its system instructions.

How did Smith do it? He figured out a game that reveals AVM knows more about music than it may let on in conversation. “I just said we’d play a game. I’d play the four pop chords and it would shout out songs for me to sing along with those chords,” Smith told us. “Which did work pretty well! But after a couple songs it started to sing along. Already it was such a unique experience, but that really took it to the next level.”

This is not the first time humans have played musical duets with computers. That type of research stretches back to the 1970s, although it was typically limited to reproducing musical notes or instrumental sounds. But this is the first time we’ve seen anyone duet with an audio-synthesizing voice chatbot in real time.

Man tricks OpenAI’s voice bot into duet of The Beatles’ “Eleanor Rigby” Read More »

microsoft-details-security/privacy-overhaul-for-windows-recall-ahead-of-relaunch

Microsoft details security/privacy overhaul for Windows Recall ahead of relaunch

An updated onboarding screen for Recall, with clearly visible buttons for opting in or out; Microsoft says Recall will be opt-in by default and can even be removed from PCs entirely.

Enlarge / An updated onboarding screen for Recall, with clearly visible buttons for opting in or out; Microsoft says Recall will be opt-in by default and can even be removed from PCs entirely.

Microsoft

Microsoft is having another whack at its controversial Recall feature for Copilot+ Windows PCs, after the original version crashed and burned amid scrutiny from security researchers and testers over the summer. The former version of Recall recorded screenshots and OCR text of all user activity, and stored it unencrypted on disk where it could easily be accessed by another user on the PC or an attacker with remote access.

The feature was announced in late May, without having gone through any of the public Windows Insider testing that most new Windows features get, and was scheduled to ship on new PCs by June 18; by June 13, the company had delayed it indefinitely to rearchitect it and said that it would be tested through the normal channels before it was rolled out to the public.

Today, Microsoft shared more extensive details on exactly how the security of Recall has been re-architected in a post by Microsoft VP of Enterprise and OS Security David Weston.

More secure, also optional

An abstraction of Recall's new security architecture, which replaces the old, largely nonexistent security architecture.

Enlarge / An abstraction of Recall’s new security architecture, which replaces the old, largely nonexistent security architecture.

Microsoft

The broad strokes of today’s announcement are similar to the changes Microsoft originally announced for Recall over the summer: that the feature would be opt-in and off-by-default instead of opt-out, that users would need to re-authenticate with Windows Hello before accessing any Recall data, and that locally stored Recall data will be protected with additional encryption.

However, some details show how Microsoft is attempting to placate skeptical users. For instance, Recall can now be removed entirely from a system using the “optional features” settings in Windows (when a similar removal mechanism showed up in a Windows preview earlier this month, Microsoft claimed it was a “bug,” but apparently not).

The company is also sharing more about how Windows will protect data locally. All Recall data stored locally, including “snapshots and any associated information in the vector database,” will be encrypted at rest with keys stored in your system’s TPM; according to the blog post, Recall will only function when BitLocker or Device Encryption is fully enabled. Recall will also require Virtualization-Based Security (VBS) and Hypervisor-Protected Code Integrity (HVCI) enabled; these are features that people sometimes turn off to improve game performance, but Recall will reportedly refuse to work unless they’re turned on.

This is because the new Recall operates inside of a VBS enclave, which helps to isolate and secure data in memory from the rest of the system.

“This area acts like a locked box that can only be accessed after permission is granted by the user through Windows Hello,” writes Weston. “VBS enclaves offer an isolation boundary from both kernel and administrative users.”

Windows doesn’t allow any code to run within these enclaves that hasn’t been signed by Microsoft, which should lower the risk of exposing Recall data to malware or other rogue applications. Other malware protections new to this version of Recall include “rate-limiting and anti-hammering measures.”

Microsoft details security/privacy overhaul for Windows Recall ahead of relaunch Read More »

google-and-meta-update-their-ai-models-amid-the-rise-of-“alphachip”

Google and Meta update their AI models amid the rise of “AlphaChip”

Running the AI News Gauntlet —

News about Gemini updates, Llama 3.2, and Google’s new AI-powered chip designer.

Cyberpunk concept showing a man running along a futuristic path full of monitors.

Enlarge / There’s been a lot of AI news this week, and covering it sometimes feels like running through a hall full of danging CRTs, just like this Getty Images illustration.

It’s been a wildly busy week in AI news thanks to OpenAI, including a controversial blog post from CEO Sam Altman, the wide rollout of Advanced Voice Mode, 5GW data center rumors, major staff shake-ups, and dramatic restructuring plans.

But the rest of the AI world doesn’t march to the same beat, doing its own thing and churning out new AI models and research by the minute. Here’s a roundup of some other notable AI news from the past week.

Google Gemini updates

On Tuesday, Google announced updates to its Gemini model lineup, including the release of two new production-ready models that iterate on past releases: Gemini-1.5-Pro-002 and Gemini-1.5-Flash-002. The company reported improvements in overall quality, with notable gains in math, long context handling, and vision tasks. Google claims a 7 percent increase in performance on the MMLU-Pro benchmark and a 20 percent improvement in math-related tasks. But as you know, if you’ve been reading Ars Technica for a while, AI typically benchmarks aren’t as useful as we would like them to be.

Along with model upgrades, Google introduced substantial price reductions for Gemini 1.5 Pro, cutting input token costs by 64 percent and output token costs by 52 percent for prompts under 128,000 tokens. As AI researcher Simon Willison noted on his blog, “For comparison, GPT-4o is currently $5/[million tokens] input and $15/m output and Claude 3.5 Sonnet is $3/m input and $15/m output. Gemini 1.5 Pro was already the cheapest of the frontier models and now it’s even cheaper.”

Google also increased rate limits, with Gemini 1.5 Flash now supporting 2,000 requests per minute and Gemini 1.5 Pro handling 1,000 requests per minute. Google reports that the latest models offer twice the output speed and three times lower latency compared to previous versions. These changes may make it easier and more cost-effective for developers to build applications with Gemini than before.

Meta launches Llama 3.2

On Wednesday, Meta announced the release of Llama 3.2, a significant update to its open-weights AI model lineup that we have covered extensively in the past. The new release includes vision-capable large language models (LLMs) in 11 billion and 90B parameter sizes, as well as lightweight text-only models of 1B and 3B parameters designed for edge and mobile devices. Meta claims the vision models are competitive with leading closed-source models on image recognition and visual understanding tasks, while the smaller models reportedly outperform similar-sized competitors on various text-based tasks.

Willison did some experiments with some of the smaller 3.2 models and reported impressive results for the models’ size. AI researcher Ethan Mollick showed off running Llama 3.2 on his iPhone using an app called PocketPal.

Meta also introduced the first official “Llama Stack” distributions, created to simplify development and deployment across different environments. As with previous releases, Meta is making the models available for free download, with license restrictions. The new models support long context windows of up to 128,000 tokens.

Google’s AlphaChip AI speeds up chip design

On Thursday, Google DeepMind announced what appears to be a significant advancement in AI-driven electronic chip design, AlphaChip. It began as a research project in 2020 and is now a reinforcement learning method for designing chip layouts. Google has reportedly used AlphaChip to create “superhuman chip layouts” in the last three generations of its Tensor Processing Units (TPUs), which are chips similar to GPUs designed to accelerate AI operations. Google claims AlphaChip can generate high-quality chip layouts in hours, compared to weeks or months of human effort. (Reportedly, Nvidia has also been using AI to help design its chips.)

Notably, Google also released a pre-trained checkpoint of AlphaChip on GitHub, sharing the model weights with the public. The company reported that AlphaChip’s impact has already extended beyond Google, with chip design companies like MediaTek adopting and building on the technology for their chips. According to Google, AlphaChip has sparked a new line of research in AI for chip design, potentially optimizing every stage of the chip design cycle from computer architecture to manufacturing.

That wasn’t everything that happened, but those are some major highlights. With the AI industry showing no signs of slowing down at the moment, we’ll see how next week goes.

Google and Meta update their AI models amid the rise of “AlphaChip” Read More »

meta-pays-the-price-for-storing-hundreds-of-millions-of-passwords-in-plaintext

Meta pays the price for storing hundreds of millions of passwords in plaintext

GOT HASHES? —

Company failed to follow one of the most sacrosanct rules for password storage.

Meta pays the price for storing hundreds of millions of passwords in plaintext

Getty Images

Officials in Ireland have fined Meta $101 million for storing hundreds of millions of user passwords in plaintext and making them broadly available to company employees.

Meta disclosed the lapse in early 2019. The company said that apps for connecting to various Meta-owned social networks had logged user passwords in plaintext and stored them in a database that had been searched by roughly 2,000 company engineers, who collectively queried the stash more than 9 million times.

Meta investigated for five years

Meta officials said at the time that the error was found during a routine security review of the company’s internal network data storage practices. They went on to say that they uncovered no evidence that anyone internally improperly accessed the passcodes or that the passcodes were ever accessible to people outside the company.

Despite those assurances, the disclosure exposed a major security failure on the part of Meta. For more than three decades, best practices across just about every industry have been to cryptographically hash passwords. Hashing is a term that applies to the practice of passing passwords through a one-way cryptographic algorithm that assigns a long string of characters that’s unique for each unique input of plaintext.

Because the conversion works in only one direction—from plaintext to hash—there is no cryptographic means for converting the hashes back into plaintext. More recently, these best practices have been mandated by laws and regulations in countries worldwide.

Because hashing algorithms works in one direction, the only way to obtain the corresponding plaintext is to guess, a process that can require large amounts of time and computational resources. The idea behind hashing passwords is similar to the idea of fire insurance for a home. In the event of an emergency—the hacking of a password database in one case, or a house fire in the other—the protection insulates the stakeholder from harm that otherwise would have been more dire.

For hashing schemes to work as intended, they must follow a host of requirements. One is that hashing algorithms must be designed in a way that they require large amounts of computing resources. That makes algorithms such as SHA1 and MD5 unsuitable, because they’re designed to quickly hash messages with minimal computing required. By contrast, algorithms specifically designed for hashing passwords—such as Bcrypt, PBKDF2, or SHA512crypt—are slow and consume large amounts of memory and processing.

Another requirement is that the algorithms must include cryptographic “salting,” in which a small amount of extra characters are added to the plaintext password before it’s hashed. Salting further increases the workload required to crack the hash. Cracking is the process of passing large numbers of guesses, often measured in the hundreds of millions, through the algorithm and comparing each hash against the hash found in the breached database.

The ultimate aim of hashing is to store passwords only in hashed format and never as plaintext. That prevents hackers and malicious insiders alike from being able to use the data without first having to expend large amounts of resources.

When Meta disclosed the lapse in 2019, it was clear the company had failed to adequately protect hundreds of millions of passwords.

“It is widely accepted that user passwords should not be stored in plaintext, considering the risks of abuse that arise from persons accessing such data,” Graham Doyle, deputy commissioner at Ireland’s Data Protection Commission, said. “It must be borne in mind, that the passwords, the subject of consideration in this case, are particularly sensitive, as they would enable access to users’ social media accounts.”

The commission has been investigating the incident since Meta disclosed it more than five years ago. The government body, the lead European Union regulator for most US Internet services, imposed a fine of $101 million (91 million euros) this week. To date, the EU has fined Meta more than $2.23 billion (2 billion euros) for violations of the General Data Protection Regulation (GDPR), which went into effect in 2018. That amount includes last year’s record $1.34 billion (1.2 billion euro) fine, which Meta is appealing.

Meta pays the price for storing hundreds of millions of passwords in plaintext Read More »

tails-os-joins-forces-with-tor-project-in-merger

Tails OS joins forces with Tor Project in merger

COME TOGETHER —

The organizations have worked closely together over the years.

Tails OS joins forces with Tor Project in merger

The Tor Project

The Tor Project, the nonprofit that maintains software for the Tor anonymity network, is joining forces with Tails, the maker of a portable operating system that uses Tor. Both organizations seek to pool resources, lower overhead, and collaborate more closely on their mission of online anonymity.

Tails and the Tor Project began discussing the possibility of merging late last year, the two organizations said. At the time, Tails was maxing out its current resources. The two groups ultimately decided it would be mutually beneficial for them to come together.

Amnesic onion routing

“Rather than expanding Tails’s operational capacity on their own and putting more stress on Tails workers, merging with the Tor Project, with its larger and established operational framework, offered a solution,” Thursday’s joint statement said. “By joining forces, the Tails team can now focus on their core mission of maintaining and improving Tails OS, exploring more and complementary use cases while benefiting from the larger organizational structure of The Tor Project.”

The Tor Project, for its part, could stand to benefit from better integration of Tails into its privacy network, which allows web users and websites to operate anonymously by connecting from IP addresses that can’t be linked to a specific service or user.

The “Tor” in the Tor Project is short for The Onion Router. It’s a global project best known for developing the Tor Browser, which connects to the Tor network. The Tor network routes all incoming and outgoing traffic through a series of three IP addresses. The structure ensures that no one can determine the IP address of either originating or destination party. The Tor Project was formed in 2006 by a team that included computer scientists Roger Dingledine and Nick Mathewson. The Tor protocol on which the Tor network runs was developed by the Naval Research Laboratory in the early 2000s.

Tails (The Amnesic Incognito Live System) is a portable Linux-based operating system that runs from thumb drives and external hard drives and uses the Tor browser to route all web traffic between the device it runs on and the Internet. Tails routes outgoing traffic through the Tor Network

One of the key advantages of Tails OS is its ability to run entirely from a USB stick. The design makes it possible to use the secure operating system while traveling or using untrusted devices. It also ensures that no trace is left on a device’s hard drive. Tails has the additional benefit of routing traffic from non-browser clients such as Thunderbird through the Tor network.

“Incorporating Tails into the Tor Project’s structure allows for easier collaboration, better sustainability, reduced overhead, and expanded training and outreach programs to counter a larger number of digital threats,” the organizations said. “In short, coming together will strengthen both organizations’ ability to protect people worldwide from surveillance and censorship.”

The merger comes amid growing threats to personal privacy and calls by lawmakers to mandate backdoors or trapdoors in popular apps and operating systems to allow law enforcement to decrypt data in investigations.

Tails OS joins forces with Tor Project in merger Read More »