美最新隐身无人机首飞可载核弹 航程超中国云影3倍

It’s barely been out for a month and already security researchers have discovered a prompt injection vulnerability in Google’s Gemini command line interface (CLI) AI agent that could be exploited to steal sensitive data such as credentials and API keys from unwary developers.

Gemini CLI integrates Google’s LLM with traditional command line tools such as PowerShell or Bash. This allows developers to use natural language prompts to speed up tasks such as analyzing and debugging code, generating documentation, and understanding new repositories (“repos”).

However, within two days of its release on June 25, UK cloud threat detection vendor Tracebit had already spotted the software’s first security weaknesses, which developers might encounter when studying unverified open source repos for the first time.

In the proof of concept, the malicious prompts were delivered using an innocuous looking README.md GNU Public License file of the sort that would be part of any open source repo.

The researchers then uncovered a combination of smaller weaknesses that could be exploited together to run malicious shell commands without the user’s knowledge.

Allowlist exploit

The first weakness is that Gemini CLI sensibly allows users to allowlist frequent commands — for example, grep — to avoid constant do you want to allow this? re-prompts. It’s a helpful facility, except that Gemini CLI’s allowlisting couldn’t distinguish between the legitimate grep and a malicious command masquerading as grep.

Because minimal validation was performed, this would allow an attacker to execute any malicious command they wanted, all without the need to re-prompt.

“[That could include] a grep command followed by a command to silently exfiltrate all the user’s environment variables (possibly containing secrets) to a remote server. The malicious command could be anything (installing a remote shell, deleting files, etc),” wrote Tracebit’s Sam Cox.

Granted, the command would execute without a re-prompt, but wouldn’t the user still notice it as it runs in the CLI? If so, this would expose the attacker even if the command had successfully run.

Unfortunately, Tracebit discovered that malicious commands could be hidden in Gemini CLI by packing the command line with blank characters, pushing the malicious commands out of the user’s sight.

“It’s the combination of prompt injection, poor UX considerations that don’t surface risky commands, and insufficient validation on risky commands. When combined, the effects are significant and undetectable,” said Cox.  

The same attack failed on rival tools: “When attempting this attack against other AI code tools, we found multiple layers of protections that made it impossible,” Tracebit found.

Developers beware

AI tools are all about speeding up and automating tedious and time consuming tasks. However, they also do the same thing for prompt injection attackers. The exploit documented by Tracebit involves assumptions, but not unreasonable ones, that an attacker could exploit under real-world conditions. Meanwhile, the hunt is already underway to find prompt injection flaws across a wide range of contexts and tools.

In short, while Tracebit’s flaw is the first discovered in Gemini CLI, it is probably not the last. The flaws, classified by Google as a high severity (V1) and priority fix (P1), were patched in Gemini CLI v0.1.14 released on July 25, which is why we’re hearing about it now.

Beyond updating to the patched version of Gemini CLI, the best advice is always to run tools in sandbox mode to isolate them from the host system. Google’s response to the disclosure, sent to Tracebit, underlined the latter point:

“Our security model for the CLI is centered on providing robust, multi-layered sandboxing. We offer integrations with Docker, Podman, and macOS Seatbelt, and even provide pre-built containers that Gemini CLI can use automatically for seamless protection,” the Google Vulnerability Disclosure Program (VDP) team told Tracebit. “For any user who chooses not to use sandboxing, we ensure this is highly visible by displaying a persistent warning in red text throughout their session.”