Biz & IT

ukrainian-cells-and-internet-still-out,-1-day-after-suspected-russian-cyberattack

Ukrainian cells and Internet still out, 1 day after suspected Russian cyberattack

Biz & IT, cellular, cyberattack, Internet, russia, Security, Ukraine / /

PLEASE STAND BY —

Hackers tied to Russian military take responsibility for hack on Ukraine’s biggest provider.

–

A service center for

Enlarge / A service center for “Kyivstar”, a Ukrainian telecommunications company, that provides communication services and data transmission based on a broad range of fixed and mobile technologies.

Getty Images

Ukrainian civilians on Wednesday grappled for a second day of widespread cellular phone and Internet outages after a cyberattack, purportedly carried out by Kremlin-supported hackers, hit the country’s biggest mobile phone and Internet provider a day earlier.

Two separate hacking groups with ties to the Russian government took responsibility for Tuesday’s attack striking Kyivstar, which has said it serves 24.3 million mobile subscribers and more than 1.1 million home Internet users. One group, calling itself Killnet, said on Telegram that “an attack was carried out on Ukrainian mobile operators, as well as on some banks,” but didn’t elaborate or provide any evidence. A separate group known as Solntsepek said on the same site that it took “full responsibility for the cyberattack on Kyivstar” and had “destroyed 10,000 computers, more than 4,000 servers, and all cloud storage and backup systems.” The post was accompanied by screenshots purporting to show someone with control over the Kyivstar systems.

In the city of Lviv, street lights remained on after sunrise and had to be disconnected manually, because Internet-dependent automated power switches didn’t work, according to NBC News. Additionally, the outage prevented shops throughout the country from processing credit payments and many ATMs from functioning, the Kyiv Post said.

The outage also disrupted air alert systems that warn residents in multiple cities of incoming missile attacks, a Ukrainian official said on Telegram. The outage forced authorities to rely on backup alarms.

“Cyber ​​specialists of the Security Service of Ukraine and ‘Kyivstar’ specialists, in cooperation with other state bodies, continue to restore the network after yesterday’s hacker attack,” officials with the Security Service of Ukraine said. “According to preliminary calculations, it is planned to restore fixed Internet for households on December 13, as well as start the launch of mobile communication and Internet. The digital infrastructure of ‘Kyivstar’ was critically damaged, so the restoration of all services in compliance with the necessary security protocols takes time.”

Kyivstar suspended mobile and Internet service on Tuesday after experiencing what company CEO Oleksandr Komarov said was an “unprecedented cyberattack” by Russian hackers. The attack represents one of the biggest compromises on a civilian telecommunications provider ever and one of the most disruptive so far in the 21-month Russia-Ukraine war. Kyivstar’s website remained unavailable at the time this post went live on Ars.

According to a report by the New Voice of Ukraine, hackers infiltrated Kyivstar’s infrastructure after first hacking into an internal employee account.

Solntsepek, one of two groups taking responsibility for the attack, has links to “Sandworm,” the name researchers use to track a hacking group that works on behalf of a unit within the Russian military known as the GRU. Sandworm has been tied to some of the most destructive cyberattacks in history, most notably the NotPetya worm, which caused an estimated $10 billion in damage worldwide. Researchers have also attributed Ukrainian power outages in 2015 and 2016 to the group.

Ukrainian cells and Internet still out, 1 day after suspected Russian cyberattack Read More »

dropbox-spooks-users-with-new-ai-features-that-send-data-to-openai-when-used

Dropbox spooks users with new AI features that send data to OpenAI when used

AI, AI privacy, Biz & IT, chatgpt, chatgtp, data privacy, dropbox, large language models, machine learning, openai, privacy / /

adventures in data consent —

AI feature turned on by default worries users; Dropbox responds to concerns.

– Updated

Photo of a man looking into a box.

On Wednesday, news quickly spread on social media about a new enabled-by-default Dropbox setting that shares Dropbox data with OpenAI for an experimental AI-powered search feature, but Dropbox says data is only shared if the feature is actively being used. Dropbox says that user data shared with third-party AI partners isn’t used to train AI models and is deleted within 30 days.

Even with assurances of data privacy laid out by Dropbox on an AI privacy FAQ page, the discovery that the setting had been enabled by default upset some Dropbox users. The setting was first noticed by writer Winifred Burton, who shared information about the Third-party AI setting through Bluesky on Tuesday, and frequent AI critic Karla Ortiz shared more information about it on X.

Wednesday afternoon, Drew Houston, the CEO of Dropbox, apologized for customer confusion in a post on X and wrote, “The third-party AI toggle in the settings menu enables or disables access to DBX AI features and functionality. Neither this nor any other setting automatically or passively sends any Dropbox customer data to a third-party AI service.“

Critics say that communication about the change could have been clearer. AI researcher Simon Willison wrote, “Great example here of how careful companies need to be in clearly communicating what’s going on with AI access to personal data.”

A screenshot of Dropbox's third-party AI feature switch.

Enlarge / A screenshot of Dropbox’s third-party AI feature switch.

Benj Edwards

So why would Dropbox ever send user data to OpenAI anyway? In July, the company announced an AI-powered feature called Dash that allows AI models to perform universal searches across platforms like Google Workspace and Microsoft Outlook.

According to the Dropbox privacy FAQ, the third-party AI opt-out setting is part of the “Dropbox AI alpha,” which is a conversational interface for exploring file contents that involves chatting with a ChatGPT-style bot using an “Ask something about this file” feature. To make it work, an AI language model similar to the one that powers ChatGPT (like GPT-4) needs access to your files.

According to the FAQ, the third-party AI toggle in your account settings is turned on by default if “you or your team” are participating in the Dropbox AI alpha. Still, multiple Ars Technica staff who had no knowledge of the Dropbox AI alpha found the setting enabled by default when they checked.

In a statement to Ars Technica, a Dropbox representative said, “The third-party AI toggle is only turned on to give all eligible customers the opportunity to view our new AI features and functionality, like Dropbox AI. It does not enable customers to use these features without notice. Any features that use third-party AI offer disclosure of third-party use, and link to settings that they can manage. Only after a customer sees the third-party AI transparency banner and chooses to proceed with asking a question about a file, will that file be sent to a third-party to generate answers. Our customers are still in control of when and how they use these features.”

Right now, the only third-party AI provider for Dropbox is OpenAI, writes Dropbox in the FAQ. “Open AI is an artificial intelligence research organization that develops cutting-edge language models and advanced AI technologies. Your data is never used to train their internal models, and is deleted from OpenAI’s servers within 30 days.” It also says, “Only the content relevant to an explicit request or command is sent to our third-party AI partners to generate an answer, summary, or transcript.”

Disabling the feature is easy if you prefer not to use Dropbox AI features. Log into your Dropbox account on a desktop web browser, then click your profile photo > Settings > Third-party AI. This link may take you to that page more quickly. On that page, click the switch beside “Use artificial intelligence (AI) from third-party partners so you can work faster in Dropbox” to toggle it into the “Off” position.

This story was updated on December 13, 2023, at 5: 35 pm ET with clarifications about when and how Dropbox shares data with OpenAI, as well as statements from Dropbox reps and its CEO.

Dropbox spooks users with new AI features that send data to OpenAI when used Read More »

broadcom-ends-vmware-perpetual-license-sales,-testing-customers-and-partners

Broadcom ends VMware perpetual license sales, testing customers and partners

Biz & IT, Broadcom, Software, Tech, virtualization, vmware / /

saas —

Already-purchased licenses can still be used but will eventually lose support.

–

The logo of American cloud computing and virtualization technology company VMware is seen at the Mobile World Congress (MWC), the telecom industry's biggest annual gathering, in Barcelona on March 2, 2023.

Broadcom has moved forward with plans to transition VMware, a virtualization and cloud computing company, into a subscription-based business. As of December 11, it no longer sells perpetual licenses with VMware products. VMware, whose $61 billion acquisition by Broadcom closed in November, also announced on Monday that it will no longer sell support and subscription (SnS) for VMware products with perpetual licenses. Moving forward, VMware will only offer term licenses or subscriptions, according to its VMware blog post.

VMware customers with perpetual licenses and active support contracts can continue using them. VMware “will continue to provide support as defined in contractual commitments,” Krish Prasad, senior vice president and general manager for VMware’s Cloud Foundation Division, wrote. But when customers’ SnS terms end, they won’t have any support.

Broadcom hopes this will force customers into subscriptions, and it’s offering “upgrade pricing incentives” that weren’t detailed in the blog for customers who switch from perpetual licensing to a subscription.

These are the products affected, per Prasad’s blog:

  • VMware Aria Automation
  • VMware Aria Suite
  • VMware Aria Operations
  • VMware Aria Operations for Logs
  • VMware Aria Operations for Networks
  • VMware Aria Universal
  • VMware Cloud Foundation
  • VMware HCX
  • VMware NSX
  • VMware Site Recovery Manager
  • VMware vCloud Suite
  • VMware vSAN
  • VMware vSphere

Subscription-based future

Broadcom is looking to grow VMware’s EBITDA (earnings before interest, taxes, depreciation, and amortization) from about $4.7 billion to about $8.5 billion in three years, largely through shifting the company’s business model to subscriptions, Tom Krause, president of the Broadcom Software Group, said during a December 7 earnings call, per Forbes.

“This shift is the natural next step in our multi-year strategy to make it easier for customers to consume both our existing offerings and new innovations. VMware believes that a subscription model supports our customers with the innovation and flexibility they need as they undertake their digital transformations,” VMware’s blog said.

With changes effective immediately upon announcement, the news might sound abrupt. However, in May, soon after announcing its plans to acquire VMware, Broadcom CEO Hock Tan signaled a “rapid transition” to subscriptions.

At the time, Tan pointed to the importance of maintaining current VMware customers’ happiness, as well as leveraging the VMware sales team already in place. However, after less than a month of the deal’s close, reports point to concern among VMWare customers and partners.

Customer and partner concerns

VMware’s blog said “the industry has already embraced subscription as the standard for cloud consumption.” For years, software and even hardware vendors and investors have been pushing IT solution provider partners and customers toward recurring revenue models. However, VMware built much of its business on the perpetual license model. As noted by The Stack, VMware in February noted that perpetual licensing was the company’s “most renowned model.”

VMware’s blog this week listed “continuous innovation” and “faster time to value” as customer benefits for subscription models but didn’t detail how it came to those conclusions.

“Predictable investments” is also listed, but it’s hard to imagine a more predictable expense than paying for something once and having supported access to it indefinitely (assuming you continue paying any support costs). Now, VMware and its partners will be left convincing customers that their finances can afford a new monthly expense for something they thought was paid for. For Broadcom, though, it’s easier to see the benefits of turning VMware into more of a reliable and recurring revenue stream.

Additionally, Broadcom’s layoffs of at least 2,837 VMware employees have brought uncertainty to the VMware brand. A CRN report in late November pointed to VMware partners hearing customer concern about potential price raises and a lack of support. C.R. Howdyshell, CEO of Advizex, which reportedly made $30 million in VMware-tied revenue in 2022, told the publication that partners and customers were experiencing “significant concern and chaos” around VMware sales. Another channel partner noted to CRN the layoff of a close VMware sales contact.

But Broadcom has made it clear that it wants to “complete the transition of all VMware by Broadcom solutions to subscription licenses,” per Prasad’s blog.

The company hopes to convince skeptical channel partners that they’ll see the way, too. VMware, like many tech companies urging subscription models, pointed to “many partners” having success with subscription models already and “opportunity for partners to engage more strategically with customers and deliver higher-value services that drive customer success.”

However, because there’s no immediate customer benefit to the end of perpetual licenses, those impacted by VMware’s change in business strategy have to assess how much they’re willing to pay to access VMware products moving forward.

Broadcom ends VMware perpetual license sales, testing customers and partners Read More »

a-new-essential-guide-to-electronics-by-naomi-wu-details-a-different-shenzen

A New Essential Guide to Electronics by Naomi Wu details a different Shenzen

andrew "bunnie" huang, andrew huang, Biz & IT, bunnie huang, china, electronics manufacturing, manufacturing, naomi wu, shenzen, Tech / /

Crystal clear, super-bright, and short leads —

Eating, tipping, LGBTQ+ advice, and Mandarin for “Self-Flashing” and “RGB.”

–

Point to translate guide in the New Essential Guide to Electronics in Shenzen

Enlarge / The New Essential Guide to Electronics in Shenzen is made to be pointed at, rapidly, in a crowded environment.

Machinery Enchantress / Crowd Supply

“Hong Kong has better food, Shanghai has better nightlife. But when it comes to making things—no one can beat Shenzen.”

Many things about the Hua Qiang market in Shenzen, China, are different than they were in 2016, when Andrew “bunnie” Huang’s Essential Guide to Electronics in Shenzen was first published. But the importance of the world’s premiere electronics market, and the need for help navigating it, are a constant. That’s why the book is getting an authorized, crowdfunded revision, the New Essential Guide, written by noted maker and Shenzen native Naomi Wu and due to ship in April 2024.

Naomi Wu’s narrated introduction to the New Essential Guide to Electronics in Shenzen.

Huang notes on the crowdfunding page that Wu’s “strengths round out my weaknesses.” Wu speaks Mandarin, lives in Shenzen, and is more familiar with Shenzen, and China, as it is today. Shenzen has grown by more than 2 million people, the central Huaqiangbei Road has been replaced by a car-free boulevard, and the city’s metro system has more than 100 new kilometers with dozens of new stations. As happens anywhere, market vendors have also changed locations, payment and communications systems have modernized, and customs have shifted.

The updated guide’s contents are set to include typical visitor guide items, like “Taxis,” “Tipping,” and, new to this edition, “LGBTQ+ Visitors.” Then there are the more Shenzen-specific guides: “Is It Fake?,” “Do Not Burn Your Contacts,” and “Type It, Don’t Say It.” The original guide had plastic business card pockets, but “They are anachronistic now,” Wu writes; removing them has allowed the 2023 guide to be sold for the same price as the original.

Machinery Enchantress / Crowd Supply

Both the original and updated guide are ring-bound and focus on quick-flipping and “Point to Translate” guides, with clearly defined boxes of English and Mandarin characters for things like “RGB,” “Common anode,” and “LED tape.” “When sourcing components, speed is critical, and it’s quicker to flip through physical pages,” Wu writes. “The market is full of visitors struggling to navigate mobile interfaces in order to make their needs known to busy vendors. It simply doesn’t work as well as walking up and pointing to large, clearly written Chinese of exactly what you want.”

Then there is the other notable thing that’s different about the two guides. Wu, a Chinese national, accomplished hardware maker, and former tech influencer, has gone quiet since the summer of 2023, following interactions with state security actors. The guide’s crowdfunding page notes that “offering an app or download specifically for English-speaking hardware engineers to install on their phones would be… iffy.” Wu adds, “If at some point ‘I’ do offer you such a thing, I’d suggest you not use it.”

Huang, who previously helped sue the government over DRM rules, designed and sold the Chumby, and was one of the first major Xbox hackers, released the original Essential Guide on the rights-friendly Crowd Supply under a Creative Commons license (BY-NC-SA 4.0) that restricted commercial derivatives without explicit permission, which he granted to Wu. The book costs $30, with roughly $8 shipping costs to the US. It is dedicated to Gavin Zhao, whom Huang considered a mentor and who furthered his ambition to print the original guide.

Listing image by Machinery Enchantress/Crowd Supply

A New Essential Guide to Electronics by Naomi Wu details a different Shenzen Read More »

everybody’s-talking-about-mistral,-an-upstart-french-challenger-to-openai

Everybody’s talking about Mistral, an upstart French challenger to OpenAI

AI, Anthropic, Arthur Mensch, Biz & IT, chatgpt, chatgtp, france, GPT-3, GPT-3.5, GPT-4, large language models, LLaMA, Llama 2, machine learning, Meta, Mistral, Mistral AI, Mixtral, Mixtral 8x7B, openai, Paris / /

A challenger appears —

“Mixture of experts” Mixtral 8x7B helps open-weights AI punch above its weight class.

–

An illustrated robot holding a French flag.

Enlarge / An illustration of a robot holding a French flag, figuratively reflecting the rise of AI in France due to Mistral. It’s hard to draw a picture of an LLM, so a robot will have to do.

On Monday, Mistral AI announced a new AI language model called Mixtral 8x7B, a “mixture of experts” (MoE) model with open weights that reportedly truly matches OpenAI’s GPT-3.5 in performance—an achievement that has been claimed by others in the past but is being taken seriously by AI heavyweights such as OpenAI’s Andrej Karpathy and Jim Fan. That means we’re closer to having a ChatGPT-3.5-level AI assistant that can run freely and locally on our devices, given the right implementation.

Mistral, based in Paris and founded by Arthur Mensch, Guillaume Lample, and Timothée Lacroix, has seen a rapid rise in the AI space recently. It has been quickly raising venture capital to become a sort of French anti-OpenAI, championing smaller models with eye-catching performance. Most notably, Mistral’s models run locally with open weights that can be downloaded and used with fewer restrictions than closed AI models from OpenAI, Anthropic, or Google. (In this context “weights” are the computer files that represent a trained neural network.)

Mixtral 8x7B can process a 32K token context window and works in French, German, Spanish, Italian, and English. It works much like ChatGPT in that it can assist with compositional tasks, analyze data, troubleshoot software, and write programs. Mistral claims that it outperforms Meta’s much larger LLaMA 2 70B (70 billion parameter) large language model and that it matches or exceeds OpenAI’s GPT-3.5 on certain benchmarks, as seen in the chart below.

A chart of Mixtral 8x7B performance vs. LLaMA 2 70B and GPT-3.5, provided by Mistral.

Enlarge / A chart of Mixtral 8x7B performance vs. LLaMA 2 70B and GPT-3.5, provided by Mistral.

Mistral

The speed at which open-weights AI models have caught up with OpenAI’s top offering a year ago has taken many by surprise. Pietro Schirano, the founder of EverArt, wrote on X, “Just incredible. I am running Mistral 8x7B instruct at 27 tokens per second, completely locally thanks to @LMStudioAI. A model that scores better than GPT-3.5, locally. Imagine where we will be 1 year from now.”

LexicaArt founder Sharif Shameem tweeted, “The Mixtral MoE model genuinely feels like an inflection point — a true GPT-3.5 level model that can run at 30 tokens/sec on an M1. Imagine all the products now possible when inference is 100% free and your data stays on your device.” To which Andrej Karpathy replied, “Agree. It feels like the capability / reasoning power has made major strides, lagging behind is more the UI/UX of the whole thing, maybe some tool use finetuning, maybe some RAG databases, etc.”

Mixture of experts

So what does mixture of experts mean? As this excellent Hugging Face guide explains, it refers to a machine-learning model architecture where a gate network routes input data to different specialized neural network components, known as “experts,” for processing. The advantage of this is that it enables more efficient and scalable model training and inference, as only a subset of experts are activated for each input, reducing the computational load compared to monolithic models with equivalent parameter counts.

In layperson’s terms, a MoE is like having a team of specialized workers (the “experts”) in a factory, where a smart system (the “gate network”) decides which worker is best suited to handle each specific task. This setup makes the whole process more efficient and faster, as each task is done by an expert in that area, and not every worker needs to be involved in every task, unlike in a traditional factory where every worker might have to do a bit of everything.

OpenAI has been rumored to use a MoE system with GPT-4, accounting for some of its performance. In the case of Mixtral 8x7B, the name implies that the model is a mixture of eight 7 billion-parameter neural networks, but as Karpathy pointed out in a tweet, the name is slightly misleading because, “it is not all 7B params that are being 8x’d, only the FeedForward blocks in the Transformer are 8x’d, everything else stays the same. Hence also why total number of params is not 56B but only 46.7B.”

Mixtral is not the first “open” mixture of experts model, but it is notable for its relatively small size in parameter count and performance. It’s out now, available on Hugging Face and BitTorrent under the Apache 2.0 license. People have been running it locally using an app called LM Studio. Also, Mistral began offering beta access to an API for three levels of Mistral models on Monday.

Everybody’s talking about Mistral, an upstart French challenger to OpenAI Read More »

the-growing-abuse-of-qr-codes-in-malware-and-payment-scams-prompts-ftc-warning

The growing abuse of QR codes in malware and payment scams prompts FTC warning

2-factor authentication, Biz & IT, malware, phishing, qr code, Security / /

SCAN THIS! —

The convenience of QR codes is a double-edged sword. Follow these tips to stay safe.

–

A woman scans a QR code in a café to see the menu online.

Enlarge / A woman scans a QR code in a café to see the menu online.

The US Federal Trade Commission has become the latest organization to warn against the growing use of QR codes in scams that attempt to take control of smartphones, make fraudulent charges, or obtain personal information.

Short for quick response codes, QR codes are two-dimensional bar codes that automatically open a Web browser or app when they’re scanned using a phone camera. Restaurants, parking garages, merchants, and charities display them to make it easy for people to open online menus or to make online payments. QR codes are also used in security-sensitive contexts. YouTube, Apple TV, and dozens of other TV apps, for instance, allow someone to sign into their account by scanning a QR code displayed on the screen. The code opens a page on a browser or app of the phone, where the account password is already stored. Once open, the page authenticates the same account to be opened on the TV app. Two-factor authentication apps provide a similar flow using QR codes when enrolling a new account.

The ubiquity of QR codes and the trust placed in them hasn’t been lost on scammers, however. For more than two years now, parking lot kiosks that allow people to make payments through their phones have been a favorite target. Scammers paste QR codes over the legitimate ones. The scam QR codes lead to look-alike sites that funnel funds to fraudulent accounts rather than the ones controlled by the parking garage.

In other cases, emails that attempt to steal passwords or install malware on user devices use QR codes to lure targets to malicious sites. Because the QR code is embedded into the email as an image, anti-phishing security software isn’t able to detect that the link it leads to is malicious. By comparison, when the same malicious destination is presented as a text link in the email, it stands a much higher likelihood of being flagged by the security software. The ability to bypass such protections has led to a torrent of image-based phishes in recent months.

Last week, the FTC warned consumers to be on the lookout for these types of scams.

“A scammer’s QR code could take you to a spoofed site that looks real but isn’t,” the advisory stated. “And if you log in to the spoofed site, the scammers could steal any information you enter. Or the QR code could install malware that steals your information before you realize it.”

The warning came almost two years after the FBI issued a similar advisory. Guidance issued from both agencies include:

  • After scanning a QR code, ensure that it leads to the official URL of the site or service that provided the code. As is the case with traditional phishing scams, malicious domain names may be almost identical to the intended one, except for a single misplaced letter.
  • Enter login credentials, payment card information, or other sensitive data only after ensuring that the site opened by the QR code passes a close inspection using the criteria above.
  • Before scanning a QR code presented on a menu, parking garage, vendor, or charity, ensure that it hasn’t been tampered with. Carefully look for stickers placed on top of the original code.
  • Be highly suspicious of any QR codes embedded into the body of an email. There are rarely legitimate reasons for benign emails from legitimate sites or services to use a QR code instead of a link.
  • Don’t install stand-alone QR code scanners on a phone without good reason and then only after first carefully scrutinizing the developer. Phones already have a built-in scanner available through the camera app that will be more trustworthy.

An additional word of caution when it comes to QR codes. Codes used to enroll a site into two-factor authentication from Google Authenticator, Authy, or another authenticator app provide the secret seed token that controls the ever-chaning one-time password displayed by these apps. Don’t allow anyone to view such QR codes. Re-enroll the site in the event the QR code is exposed.

The growing abuse of QR codes in malware and payment scams prompts FTC warning Read More »

as-chatgpt-gets-“lazy,”-people-test-“winter-break-hypothesis”-as-the-cause

As ChatGPT gets “lazy,” people test “winter break hypothesis” as the cause

AI, Biz & IT, chatgpt, ChatGPT lazy, chatgtp, GPT-3.5, GPT-4, machine learning, openai, winter break hypothesis / /

only 14 shopping days ’til Christmas —

Unproven hypothesis seeks to explain ChatGPT’s seemingly new reluctance to do hard work.

–

A hand moving a wooden calendar piece that says

In late November, some ChatGPT users began to notice that ChatGPT-4 was becoming more “lazy,” reportedly refusing to do some tasks or returning simplified results. Since then, OpenAI has admitted that it’s an issue, but the company isn’t sure why. The answer may be what some are calling “winter break hypothesis.” While unproven, the fact that AI researchers are taking it seriously shows how weird the world of AI language models has become.

“We’ve heard all your feedback about GPT4 getting lazier!” tweeted the official ChatGPT account on Thursday. “We haven’t updated the model since Nov 11th, and this certainly isn’t intentional. model behavior can be unpredictable, and we’re looking into fixing it.”

On Friday, an X account named Martian openly wondered if LLMs might simulate seasonal depression. Later, Mike Swoopskee tweeted, “What if it learned from its training data that people usually slow down in December and put bigger projects off until the new year, and that’s why it’s been more lazy lately?”

Since the system prompt for ChatGPT feeds the bot the current date, people noted, some began to think there may be something to the idea. Why entertain such a weird supposition? Because research has shown that large language models like GPT-4, which powers the paid version of ChatGPT, respond to human-style encouragement, such as telling a bot to “take a deep breath” before doing a math problem. People have also less formally experimented with telling an LLM that it will receive a tip for doing the work, or if an AI model gets lazy, telling the bot that you have no fingers seems to help lengthen outputs.

  • “Winter break hypothesis” test result screenshots from Rob Lynch on X.

  • “Winter break hypothesis” test result screenshots from Rob Lynch on X.

  • “Winter break hypothesis” test result screenshots from Rob Lynch on X.

On Monday, a developer named Rob Lynch announced on X that he had tested GPT-4 Turbo through the API over the weekend and found shorter completions when the model is fed a December date (4,086 characters) than when fed a May date (4,298 characters). Lynch claimed the results were statistically significant. However, a reply from AI researcher Ian Arawjo said that he could not reproduce the results with statistical significance. (It’s worth noting that reproducing results with LLM can be difficult because of random elements at play that vary outputs over time, so people sample a large number of responses.)

As of this writing, others are busy running tests, and the results are inconclusive. This episode is a window into the quickly unfolding world of LLMs and a peek into an exploration into largely unknown computer science territory. As AI researcher Geoffrey Litt commented in a tweet, “funniest theory ever, I hope this is the actual explanation. Whether or not it’s real, [I] love that it’s hard to rule out.”

A history of laziness

One of the reports that started the recent trend of noting that ChatGPT is getting “lazy” came on November 24 via Reddit, the day after Thanksgiving in the US. There, a user wrote that they asked ChatGPT to fill out a CSV file with multiple entries, but ChatGPT refused, saying, “Due to the extensive nature of the data, the full extraction of all products would be quite lengthy. However, I can provide the file with this single entry as a template, and you can fill in the rest of the data as needed.”

On December 1, OpenAI employee Will Depue confirmed in an X post that OpenAI was aware of reports about laziness and was working on a potential fix. “Not saying we don’t have problems with over-refusals (we definitely do) or other weird things (working on fixing a recent laziness issue), but that’s a product of the iterative process of serving and trying to support sooo many use cases at once,” he wrote.

It’s also possible that ChatGPT was always “lazy” with some responses (since the responses vary randomly), and the recent trend made everyone take note of the instances in which they are happening. For example, in June, someone complained of GPT-4 being lazy on Reddit. (Maybe ChatGPT was on summer vacation?)

Also, people have been complaining about GPT-4 losing capability since it was released. Those claims have been controversial and difficult to verify, making them highly subjective.

As Ethan Mollick joked on X, as people discover new tricks to improve LLM outputs, prompting for large language models is getting weirder and weirder: “It is May. You are very capable. I have no hands, so do everything. Many people will die if this is not done well. You really can do this and are awesome. Take a deep breathe and think this through. My career depends on it. Think step by step.”

As ChatGPT gets “lazy,” people test “winter break hypothesis” as the cause Read More »

elon-musk’s-new-ai-bot,-grok,-causes-stir-by-citing-openai-usage-policy

Elon Musk’s new AI bot, Grok, causes stir by citing OpenAI usage policy

AI, Biz & IT, chatgpt, elon musk, GPT-3, GPT-3.5, GPT-4, grok, large language model, large language models, machine learning, Twitter, X-Com, xAI / /

You are what you eat —

Some experts think xAI used OpenAI model outputs to fine-tune Grok.

–

Illustration of a broken robot exchanging internal gears.

Grok, the AI language model created by Elon Musk’s xAI, went into wide release last week, and people have begun spotting glitches. On Friday, security tester Jax Winterbourne tweeted a screenshot of Grok denying a query with the statement, “I’m afraid I cannot fulfill that request, as it goes against OpenAI’s use case policy.” That made ears perk up online since Grok isn’t made by OpenAI—the company responsible for ChatGPT, which Grok is positioned to compete with.

Interestingly, xAI representatives did not deny that this behavior occurs with its AI model. In reply, xAI employee Igor Babuschkin wrote, “The issue here is that the web is full of ChatGPT outputs, so we accidentally picked up some of them when we trained Grok on a large amount of web data. This was a huge surprise to us when we first noticed it. For what it’s worth, the issue is very rare and now that we’re aware of it we’ll make sure that future versions of Grok don’t have this problem. Don’t worry, no OpenAI code was used to make Grok.”

In reply to Babuschkin, Winterbourne wrote, “Thanks for the response. I will say it’s not very rare, and occurs quite frequently when involving code creation. Nonetheless, I’ll let people who specialize in LLM and AI weigh in on this further. I’m merely an observer.”

A screenshot of Jax Winterbourne's X post about Grok talking like it's an OpenAI product.

Enlarge / A screenshot of Jax Winterbourne’s X post about Grok talking like it’s an OpenAI product.

Jason Winterbourne

However, Babuschkin’s explanation seems unlikely to some experts because large language models typically do not spit out their training data verbatim, which might be expected if Grok picked up some stray mentions of OpenAI policies here or there on the web. Instead, the concept of denying an output based on OpenAI policies would probably need to be trained into it specifically. And there’s a very good reason why this might have happened: Grok was fine-tuned on output data from OpenAI language models.

“I’m a bit suspicious of the claim that Grok picked this up just because the Internet is full of ChatGPT content,” said AI researcher Simon Willison in an interview with Ars Technica. “I’ve seen plenty of open weights models on Hugging Face that exhibit the same behavior—behave as if they were ChatGPT—but inevitably, those have been fine-tuned on datasets that were generated using the OpenAI APIs, or scraped from ChatGPT itself. I think it’s more likely that Grok was instruction-tuned on datasets that included ChatGPT output than it was a complete accident based on web data.”

As large language models (LLMs) from OpenAI have become more capable, it has been increasingly common for some AI projects (especially open source ones) to fine-tune an AI model output using synthetic data—training data generated by other language models. Fine-tuning adjusts the behavior of an AI model toward a specific purpose, such as getting better at coding, after an initial training run. For example, in March, a group of researchers from Stanford University made waves with Alpaca, a version of Meta’s LLaMA 7B model that was fine-tuned for instruction-following using outputs from OpenAI’s GPT-3 model called text-davinci-003.

On the web you can easily find several open source datasets collected by researchers from ChatGPT outputs, and it’s possible that xAI used one of these to fine-tune Grok for some specific goal, such as improving instruction-following ability. The practice is so common that there’s even a WikiHow article titled, “How to Use ChatGPT to Create a Dataset.”

It’s one of the ways AI tools can be used to build more complex AI tools in the future, much like how people began to use microcomputers to design more complex microprocessors than pen-and-paper drafting would allow. However, in the future, xAI might be able to avoid this kind of scenario by more carefully filtering its training data.

Even though borrowing outputs from others might be common in the machine-learning community (despite it usually being against terms of service), the episode particularly fanned the flames of the rivalry between OpenAI and X that extends back to Elon Musk’s criticism of OpenAI in the past. As news spread of Grok possibly borrowing from OpenAI, the official ChatGPT account wrote, “we have a lot in common” and quoted Winterbourne’s X post. As a comeback, Musk wrote, “Well, son, since you scraped all the data from this platform for your training, you ought to know.”

Elon Musk’s new AI bot, Grok, causes stir by citing OpenAI usage policy Read More »

stealthy-linux-rootkit-found-in-the-wild-after-going-undetected-for-2-years

Stealthy Linux rootkit found in the wild after going undetected for 2 years

backdoor, Biz & IT, Linux, malware, rootkits, Security / /
Trojan horse on top of blocks of hexadecimal programming codes. Illustration of the concept of online hacking, computer spyware, malware and ransomware.

Stealthy and multifunctional Linux malware that has been infecting telecommunications companies went largely unnoticed for two years until being documented for the first time by researchers on Thursday.

Researchers from security firm Group-IB have named the remote access trojan “Krasue,” after a nocturnal spirit depicted in Southeast Asian folklore “floating in mid-air, with no torso, just her intestines hanging from below her chin.” The researchers chose the name because evidence to date shows it almost exclusively targets victims in Thailand and “poses a severe risk to critical systems and sensitive data given that it is able to grant attackers remote access to the targeted network.

According to the researchers:

  • Krasue is a Linux Remote Access Trojan that has been active since 20 and predominantly targets organizations in Thailand.
  • Group-IB can confirm that telecommunications companies were targeted by Krasue.
  • The malware contains several embedded rootkits to support different Linux kernel versions.
  • Krasue’s rootkit is drawn from public sources (3 open-source Linux Kernel Module rootkits), as is the case with many Linux rootkits.
  • The rootkit can hook the `kill()` syscall, network-related functions, and file listing operations in order to hide its activities and evade detection.
  • Notably, Krasue uses RTSP (Real-Time Streaming Protocol) messages to serve as a disguised “alive ping,” a tactic rarely seen in the wild.
  • This Linux malware, Group-IB researchers presume, is deployed during the later stages of an attack chain in order to maintain access to a victim host.
  • Krasue is likely to either be deployed as part of a botnet or sold by initial access brokers to other cybercriminals.
  • Group-IB researchers believe that Krasue was created by the same author as the XorDdos Linux Trojan, documented by Microsoft in a March 2022 blog post, or someone who had access to the latter’s source code.

During the initialization phase, the rootkit conceals its own presence. It then proceeds to hook the `kill()` syscall, network-related functions, and file listing operations, thereby obscuring its activities and evading detection.

The researchers have so far been unable to determine precisely how Krasue gets installed. Possible infection vectors include through vulnerability exploitation, credential-stealing or -guessing attacks, or by unwittingly being installed as trojan stashed in an installation file or update masquerading as legitimate software.

The three open source rootkit packages incorporated into Krasue are:

An image showing salient research points of Krasue.

Enlarge / An image showing salient research points of Krasue.

Group-IB

Rootkits are a type of malware that hides directories, files, processes, and other evidence of its presence to the operating system it’s installed on. By hooking legitimate Linux processes, the malware is able to suspend them at select points and interject functions that conceal its presence. Specifically, it hides files and directories beginning with the names “auwd” and “vmware_helper” from directory listings and hides ports 52695 and 52699, where communications to attacker-controlled servers occur. Intercepting the kill() syscall also allows the trojan to survive Linux commands attempting to abort the program and shut it down.

Stealthy Linux rootkit found in the wild after going undetected for 2 years Read More »

eu-agrees-to-landmark-rules-on-artificial-intelligence

EU agrees to landmark rules on artificial intelligence

AI, AI law, Biz & IT, European Union, syndication / /

Get ready for some restrictions, Big Tech —

Legislation lays out restrictive regime for emerging technology.

–

EU Commissioner Thierry Breton talks to media during a press conference in June.

Enlarge / EU Commissioner Thierry Breton talks to media during a press conference in June.

Thierry Monasse | Getty Images

European Union lawmakers have agreed on the terms for landmark legislation to regulate artificial intelligence, pushing ahead with enacting the world’s most restrictive regime on the development of the technology.

Thierry Breton, EU commissioner, confirmed in a post on X that a deal had been reached.

He called it a historic agreement. “The EU becomes the very first continent to set clear rules for the use of AI,” he wrote. “The AIAct is much more than a rulebook—it’s a launchpad for EU start-ups and researchers to lead the global AI race.”

The deal followed years of discussions among member states and politicians on the ways AI should be curbed to have humanity’s interest at the heart of the legislation. It came after marathon discussions that started on Wednesday this week.

Members of the European Parliament have spent years arguing over their position before it was put forward to member states and the European Commission, the executive body of the EU. All three—countries, politicians, and the commission—must agree on the final text before it becomes law.

European companies have expressed their concern that overly restrictive rules on the technology, which is rapidly evolving and gained traction after the popularisation of OpenAI’s ChatGPT, will hamper innovation. Last June, dozens of some of the largest European companies, such as France’s Airbus and Germany’s Siemens, said the rules were looking too tough to nurture innovation and help local industries.

Last month, the UK hosted a summit on AI safety, leading to broad commitments from 28 nations to work together to tackle the existential risks stemming from advanced AI. That event attracted leading tech figures such as OpenAI’s Sam Altman, who has previously been critical of the EU’s plans to regulate the technology.

© 2023 The Financial Times Ltd. All rights reserved. Please do not copy and paste FT articles and redistribute by email or post to the web.

EU agrees to landmark rules on artificial intelligence Read More »