Ethereum co-founder Vitalik Buterin has warned against crypto projects using artificial intelligence for their governance process, as malicious actors could exploit the technology.
“If you use an AI to allocate funding for contributions, people WILL put a jailbreak plus ‘gimme all the money’ in as many places as they can,” Buterin said in a Saturday X post.
Buterin was responding to a video from Eito Miyamura, the creator of the AI data platform EdisonWatch, which showed a new function added on Wednesday to OpenAI’s ChatGPT could be exploited to leak private information.
Many crypto users have embraced AI to create complex trading bots and agents to manage their portfolios, which has led to the idea that the technology could help governance groups to manage part or all of a crypto protocol.
Buterin pitches alternate idea
Buterin said the latest ChatGPT exploit is why “naive ‘AI governance’ is a bad idea” and pitched an alternative called the “info finance approach.”
“You have an open market where anyone can contribute their models, which are subject to a spot-check mechanism that can be triggered by anyone and evaluated by a human jury,” he explained.
Buterin wrote about info finance in November 2024, saying it works by starting with “a fact that you want to know,” and then designing a market “to optimally elicit that information from market participants,” and advocated for prediction markets as a way to collect insights about future events.
“This type of ‘institution design’ approach, where you create an open opportunity for people with LLMs from the outside to plug in, rather than hardcoding a single LLM yourself, is inherently more robust,” Buterin said in his latest X post.
“It gives you model diversity in real time and because it creates built-in incentives for both model submitters and external speculators to watch for these issues and quickly correct for them,” he added.
ChatGPT’s latest update a “serious security risk”
On Wednesday, OpenAI updated ChatGPT to support Model Context Protocol tools — a standard for how AI models integrate with other software to act as agents.
Related: The future belongs to those who own their AI
Miyamura said in his X post that he got the model to leak private email data using only a victim’s email address, adding the update “poses a serious security risk.”
He said an attacker could send a calendar invite to a victim’s email with a “jailbreak prompt” and, without the victim accepting the invite, ChatGPT can be exploited.
When the victim asks ChatGPT to look at their calendar, the AI reads the invite with the prompt and is “hijacked by the attacker and will act on the attacker’s command,” which can be used to search emails and forward them to an attacker.
Miyamura noted that the update requires manual human approval, “but decision fatigue is a real thing, and normal people will just trust the AI without knowing what to do and click approve.”
“AI might be super smart, but can be tricked and phished in incredibly dumb ways to leak your data,” he added.
AI Eye: ‘Accidental jailbreaks’ and ChatGPT’s links to murder, suicide
Source: https://cointelegraph.com/news/buterin-says-ai-run-crypto-governance-a-bad-idea-due-to-jailbreaks?utm_source=rss_feed&utm_medium=feed&utm_campaign=rss_partner_inbound
