Why Is Big Tech Trying to Make AI Agents the New Labubus? | Vogue

Save Story Save this story Save Story Save this story AI agents are here, and they’re… cute? In the last two weeks, Meta released its personal AI assistant, Muse, while OpenAI released its Dots AI agent. The promise is that these agents can autonomously act on users’ behalves, from answering emails and sending invoices to booking flights and buying clothes. What’s notable is their kawaii-coded wrapping: unnervingly advanced bots dressed up as cute, unthreatening companions. OpenAI’s Dots are puffy rainbow colored blobs donning sunglasses and berets. Meta’s Muse agent mascot, named Jolly, is a fuzzy, chubby, rosy-cheeked creature that’s part-Labubu, part-Squishmallow. It will soon come with the Muse Charm, a Tamagotchi-like accessory bearing the mascot’s face that can be activated by a fingerprint sensor. “We’ve packed a lot of technology for this little guy to fit onto a keychain and always be available to talk to,” Mark Zuckerberg said in his keynote at the annual Meta Connect conference in September. Meta is yet to confirm whether the Muse Charm will feature cameras and microphones, but Zuckerberg added on stage: “You just go ahead and tap here in the fingerprint sensor in the corner and you can start talking.” Meta CEO Mark Zuckerberg unveiled the Muse AI agent ‘charm’ at Meta Connect in San Francisco. Photo: Minh Connors/Bloomberg via Getty Images Bloomberg/Getty Images That these agents have arrived at a moment plagued by broader AI doomsday warnings about their potential threat to humanity brings this friendly packaging into sharp relief. “When these companies speak to investors, regulators and researchers, AI is described as extraordinarily powerful technology with serious and unresolved questions around safety, security, and control,” says Dr. Sarah Saska, a sociotechnologist and specialist in tech’s relationship with culture and power. “Yet, increasingly intimate versions of the same technology are presented to consumers as friendly, playful, frictionless, and low-stakes.” In the same week, OpenAI CEO Sam Altman unveiled Dots, the company pulled its latest AI model and delayed its IPO due to existential safety concerns around the very same AI agent tech. Meanwhile, just weeks before its Muse launch, Meta agreed to pay an $18 billion settlement to 48 US states — the largest-ever corporate settlement in tech sector history — to resolve claims that the algorithmic design of its Facebook and Instagram platforms harm children. Safety warnings are coming from inside the house at major rivals, too: Anthropic CEO Dario Amodei called on the industry to slow the pace of frontier AI development, a plea publicly backed by Altman, xAI’s Elon Musk, and Google DeepMind’s Demis Hassabis. (Zuckerberg, for his part, has rejected calls for an industry-wide slowdown in the pace of AI development, arguing instead that individual companies should decide for themselves if and when to slow down.) Users are nevertheless intrigued about these AI agents. Within 10 days of launch, Muse had racked up more than 730,000 US downloads and knocked ChatGPT off the top of Apple’s US App Store rankings. Days later, investor excitement around the agent helped send Meta shares up 11% in a single session, adding some $192 billion to its market value. Yet, within the first two weeks of its release, several reports raised concerns about Muse overstepping users’ intended permissions — including sharing sensitive information and taking actions users said they’d not explicitly approved. After using feminization to soften smart glasses (see Meta’s smart glasses partnerships with Kylie Jenner and Lisa), experts say kawaii-fication is the next frontier in making potentially unsettling tech feel more palatable to consumers. AI safety researchers warn this cuteness could lower consumers’ guards, dulling the privacy concerns they might otherwise have before handing an AI agent access to their emails, private messages and browsing behavior — let alone the power to spend on their behalf. “These are deliberate design choices, made with an audience and a desired response in mind,” says Lidia Velkova, managing director of Clever Together Futureproof, a governance and policy foresight agency that focuses on responsible AI and marketing. “Research on social robots shows baby-like features can increase perceived trustworthiness; separate chatbot research shows that human-like interaction can increase trust and personal disclosure,” she adds. “My concern is that these designs make powerful systems feel harmless and familiar, encouraging people to share more and question less.” A collapse of consumer boundaries Behind their fuzzy exteriors, personal AI agents represent a significant escalation in the access consumers are being asked to hand tech companies to their digital lives. To be genuinely useful, an agent needs context. Meta’s Muse can connect to email, calendars, Instagram, Facebook and other apps, browse the web and make purchases on users’ behalf, they can also interact with it directly through Whatsapp. But it’s exactly this access that exposes consumers to the biggest risks of their data getting into the wrong hands. “In order to make Muse as effective as possible, which makes it stickier with the general public, it defaults to opting all users into using their interactions to train its AI model, as well as storing all the data you share with it,” says Kate Winick, principal analyst at Forrester. “This creates a huge bank of personal data that can be hacked. The more you give Muse, the better it works for you, but the more exposed you are to both bad actors and bad actions by Meta.” Meta’s track record with consumer data gives them good reason for caution: the company has faced repeated fines and lawsuits over its handling of personal data in the past few years. Just days after Muse launched, a New Mexico jury found Meta had misled users about its data practices in a case stemming from the Cambridge Analytica scandal in 2018. A spokesperson for Meta told Vogue Business : “We disagree with the verdict and will continue to defend ourselves against efforts to distort our record.” Meta confirms that use of Muse interactions for AI training is enabled by default, though users can opt out. It also says Muse operates inside an isolated virtual machine where it stores the user’s data and files, with a separate Sentinel system that monitors and vets the agent’s actions. But Meta itself acknowledges these safeguards don’t remove the risk of data breaches. “Muse isn’t immune to attack. Prompt injection remains an open problem in the industry — and Muse will sometimes make mistakes,” Meta said in a blog post about the tech’s privacy measures. Early users are already seeing potential risks playing out. When YouTuber Matt Robb asked his Muse to help him sell a keyboard on Facebook Marketplace, his agent subsequently shared his home address with a prospective buyer, negotiated on his behalf and arranged a collection, with Robb only realizing when the buyer and their family turned up at his front door. It turned out that in this incident, there had been no technical breach or hack; rather, it stemmed from an “allow-always” permission Robb had selected, effectively handing Muse the reins to his future Marketplace conversations, including the ability to draw on personal details he had previously shared, such as his home address. OpenAI CEO Sam Altman unveiled the ChatGPT maker’s new “Dot” agents at OpenAI’s DevDay last week. Photo: Heather Diehl/Getty Images The episode exposes agentic AI’s biggest problem in practice: where we draw the line between access and authority. Consumers are accustomed to clicking through app permissions, but allowing software to read information is different from allowing it to negotiate with strangers, disclose that information, and make decisions in your name. An agent can technically be acting within a permission a user granted, while doing something the user never imagined they had authorized. A Meta customer help page on how Muse makes payments explicitly states to consumers: “You’re responsible for all transactions that your Muse makes on your behalf. Keep an eye out for email confirmations, receipts and statements.” A spokesperson for Meta told Vogue Business : “Users can take over at any time, Muse is designed to always ask permission before buying, and it applies ethical browsing principles when asked to do things that a human could not — like buying all the tickets to an event.” Where brands must pick up the bill The idea is that agents will maneuver around the internet on our behalf — and that lack of distinction between bot and customer is a growing problem for brands and retailers. Amazon has already blocked Muse from shopping on its site, arguing that Meta had not sought its proper permission, the agent did not identify itself as AI, and its access to customer accounts and login credentials raised privacy and security concerns. By contrast, Muse has a much smoother path into brands and retailers that have formally opened their doors to it. Meta has announced integrations with Walmart, Sephora, Gap and others, while its partnership with Shopify already gives Muse structured access to merchants’ product catalogues and, for eligible stores, a direct route through checkout — two options that Shopify says merchants can toggle on or off. For brands, that can mean gaining access to a potentially powerful new sales channel without leaving an AI agent to navigate their websites unaided. But it also requires ceding some control over a crucial part of the customer journey to Meta. The agent can increasingly stand between shopper and store, discovering products, making recommendations, and completing purchases without a customer necessarily visiting the retailer’s own website. In other words, brands may gain a new customer in Muse, while surrendering some of the data and direct relationship that traditionally came with them. That raises questions over everything from customer data and brand experience to fraud and accountability when an agent gets something wrong. It also leaves smaller brands in a more precarious position: without the resources or leverage to negotiate bespoke integrations, an AI agent could arrive on their site and transact on a customer’s behalf without any direct relationship with the company behind it. “The brand or merchant potentially bears residual risk where no merchant agreement exists with Meta, leaving retailers exposed to unvetted agentic transactions,” says Nick Phillips, intellectual property partner at Edwin Coe LLP. Concerns over these loopholes led six major banks, including Bank of America and Capital One, to jointly call this week for common global standards governing agentic commerce, including clearer rules around transparency, consumer consent, fraud, data privacy, and who is liable when an AI-driven transaction goes wrong. The bank intervention points to the bigger unresolved question for brands: when an AI agent makes a purchase that a consumer later disputes, who is responsible for putting it right? For now, experts say much of that burden may still land with the retailer. “Brands can expect all the responsibility they currently bear in digital transactions to apply to agentic transactions. They must protect the security of purchases made on their site, accurately represent and deliver products, and they have to honor their existing consumer protection laws, as well as manage reputational issues, just as they always have,” Winick says. She points to cases where a child places an Amazon order via Alexa, and the retailer generally still refunds the order, even though their technical set-up does place the burden of preventing the order on parents and how they manage their own account settings. “AI agents aren’t children, but when they take actions customers are unhappy with, retailers will feel the pressure to make it right even if they’re not legally required to do so,” she says.