CWE-1039 自动识别机制在检测或处理对抗性输入扰动时能力不足

Automated Recognition Mechanism with Inadequate Detection or Handling of Adversarial Input Perturbations

结构: Simple

Abstraction: Class

状态: Incomplete

被利用可能性: unkown

基本描述

The product uses an automated mechanism such as machine learning to recognize complex data inputs (e.g. image or audio) as a particular concept or category, but it does not properly detect or handle inputs that have been modified or constructed in a way that causes the mechanism to detect a different, incorrect concept.

扩展描述

When techniques such as machine learning are used to automatically classify input streams, and those classifications are used for security-critical decisions, then any mistake in classification can introduce a vulnerability that allows attackers to cause the product to make the wrong security decision. If the automated mechanism is not developed or "trained" with enough input data, then attackers may be able to craft malicious input that intentionally triggers the incorrect classification.

Targeted technologies include, but are not necessarily limited to:

For example, an attacker might modify road signs or road surface markings to trick autonomous vehicles into misreading the sign/marking and performing a dangerous action.

相关缺陷

• cwe_Nature: ChildOf cwe_CWE_ID: 693 cwe_View_ID: 1000 cwe_Ordinal: Primary

• cwe_Nature: ChildOf cwe_CWE_ID: 697 cwe_View_ID: 1000

适用平台

Language: {'cwe_Class': 'Language-Independent', 'cwe_Prevalence': 'Undetermined'}

常见的影响

Notes

引用

• REF-16 Intriguing properties of neural networks

• REF-17 Attacking Machine Learning with Adversarial Examples

• REF-15 Magic AI: These are the Optical Illusions that Trick, Fool, and Flummox Computers

• REF-13 CommanderSong: A Systematic Approach for Practical Adversarial Voice Recognition

• REF-14 Audio Adversarial Examples: Targeted Attacks on Speech-to-Text