Prompt injection happens when untrusted input is interpreted as instruction for large language model (LLM) systems. This is misused to inject malformed instructions and bypass the AI system restrictions, policies and disclose sensitive data.

The root cause of the problem is mixture of data and system instructions.

Remediation

• Utilise token preventative feature that comes with some API (see https://platform.openai.com/docs/api-reference/chat/create#chat/create-stop[OpenAI API stop parameter].
• Provide a more restrictive system instructions.

NOTE: At the time of writing the remediation for prompt injection is an open problem and there is no best practice security recommendation.