Sanitize Training Data

Training data should be sanitized prior to use to remove inappropriate content and protect against data poisoning attacks. Inappropriate content may include toxic language, harmful stereotypes, or malformed documents, while "poisoned" content refers to data intentionally crafted to manipulate or degrade model performance.

Sanitization should be tailored to the specific AI system's use case and may involve both automated tools and manual reviews. Various off-the-shelf tools and models exist for filtering inappropriate or low-quality content. However, detecting poisoned data is a more complex task, and no standardized method exists.

Common approaches include:

- Using outlier detection to flag potentially malicious entries for manual review.
- Limiting the number of data points from each specific source (i.e. user) to ensure that they are not intentionally corrupting the data.
- Training a secondary model to distinguish good vs poisoned datapoints and use it as a custom filter.
- Implementing an advanced detector using the Adversarial Robustness Toolbox.

- Prevents Model Degradation from Poisoning
- Poisoned training data can lead to incorrect or manipulated behavior in AI models. Sanitation techniques reduce the risk of training on corrupted examples.
- Improves Model Reliability and Fairness
- Removing inappropriate content improves the integrity of the dataset, leading to more balanced, respectful, and unbiased model behavior.
- Supports Compliance and Safety Standards
- Sanitizing data helps organizations meet content quality and ethical standards set by laws, industry best practices, and internal governance protocols.

- Pre-Training Filters and Logs
- Documentation showing use of automated tools or APIs (e.g. Perspective API, Detoxify) for content screening prior to model training.
- Poisoning Detection Audit Reports
- Reports showing detection tools like the Adversarial Robustness Toolbox were applied and how flagged data was reviewed or removed.
- Manual Review Annotations
- Evidence of human annotations or QA flags for data identified as inappropriate, misleading, or adversarial.
- Source Diversity Metrics
- Documentation of contribution limits by user or domain, ensuring no single source overwhelms the dataset.