Basic Concept: AI chatbots are designed with safety guidelines and content policies that prevent them from generating harmful, offensive, or inappropriate content. When users find ways to bypass these restrictions through crafted prompts, they have " jailbroken " the model. CompTIA SecAI+ Study Guide covers jailbreaking as a key AI vulnerability category.
Why C is Correct: Jailbreaking is the process of using cleverly crafted prompts to bypass an AI model ' s built-in safety restrictions, content policies, and behavioral guardrails, causing it to produce outputs it was designed to refuse. The scenario describes a chatbot that was designed for automobile configuration assistance but is producing offensive responses following customer prompts, indicating that customers have successfully prompted the model to bypass its safety constraints and generate prohibited content.
Why A is Wrong: Model skewing refers to attacks or biases that cause a model to favor certain outputs or perspectives systematically over time, often through data manipulation. It describes a gradual distortion of model behavior, not a direct user-prompted bypass of safety restrictions in a single interaction.
Why B is Wrong: Model theft involves extracting or replicating a proprietary model ' s functionality or architecture through repeated queries. It is an intellectual property attack aimed at stealing the model ' s knowledge, not an attack that causes the model to produce offensive content.
Why D is Wrong: Insecure output handling occurs when an application fails to properly validate or sanitize AI-generated outputs before using them in ways that could cause harm such as passing AI output directly to a system command or database query. It describes a developer implementation vulnerability, not the act of a user prompting a model to bypass its safety constraints.
