Improper Output Handling occurs when outputs generated by Large Language Models (LLMs) are not properly validated, sanitised, or managed before being passed to downstream components or systems. This vulnerability arises because LLM-generated content is influenced by input prompts, effectively giving users indirect access to certain functionality. Improper Output Handling focuses on ensuring the safety of outputs before further processing. Exploitation of this vulnerability can lead to security issues such as cross-site scripting (XSS), cross-site request forgery (CSRF) in web browsers, server-side request forgery (SSRF), privilege escalation, or remote code execution on backend systems.

Remediation

• Ensure all outputs are validated against expected formats and data types before being processed by downstream systems.
• Apply sanitisation techniques to remove or escape any malicious content, particularly when outputs are used in web or database contexts.
• Limit the ability of prompts to trigger sensitive or unintended system functionality through output handling.
• For web-based applications, escape LLM outputs to prevent injection attacks such as XSS or CSRF.
• Tailor sanitisation and validation processes based on the specific context in which the outputs will be used (e.g., HTML, JSON, SQL).
• Run LLMs in isolated environments to minimise the risk of backend system compromise through malicious outputs.