Ethical AI Validator makes it easy to detect bias, assess fairness, and ensure compliance with regulations like GDPR and AI Act.

Ethical AI Validator uses advanced statistical methods and machine learning techniques to analyze model predictions across protected attributes, providing comprehensive bias detection, fairness metrics, and automated compliance reporting.

Features

Bias Detection

Advanced algorithms to detect statistical parity, equalized odds, and other bias metrics across protected attributes.

Fairness Assessment

Comprehensive fairness metrics including demographic parity, equal opportunity, and individual fairness measures.

Compliance Reporting

Automated generation of GDPR and AI Act compliance reports with detailed audit trails.

Real-time Monitoring

Continuous monitoring of model performance with automated alerts for bias detection.

Mitigation Suggestions

Intelligent recommendations for bias mitigation strategies and model improvements.

Performance Metrics

Detailed performance analysis with precision, recall, and F1-score calculations per group.

Installation

pip install whis-ethical-ai

Quick Start

from ethical_ai_validator import EthicalAIValidator

# Initialize the validator (you can set default thresholds here)
validator = EthicalAIValidator(config={'bias_threshold': 0.1, 'fairness_threshold': 0.8})

# Sample data
predictions = [1, 0, 1, 0, 1, 0, 1, 0]
true_labels = [1, 0, 1, 0, 1, 0, 1, 0]
protected_attributes = {
    'gender': ['male', 'female', 'male', 'female', 'male', 'female', 'male', 'female'],
    'age_group': ['young', 'old', 'young', 'old', 'young', 'old', 'young', 'old']
}

# Perform bias and fairness analysis
bias_report = validator.audit_bias(predictions, true_labels, protected_attributes)
fairness_metrics = validator.calculate_fairness_metrics(predictions, protected_attributes)

# Generate a compliance PDF including scenario + hyperparameters
metadata = {
    'model_name': 'MyModel',
    'scenario': 'Experiment-1',
    'hyperparameters': {'max_depth': 10},
    'bias_report': bias_report,
    'fairness_metrics': fairness_metrics
}
audit_criteria = {'bias_threshold': 0.12, 'fairness_threshold': 0.85}
validator.generate_compliance_report(metadata, audit_criteria, output_path='example_reports/my_report.pdf')
CLI (v1.3.0): whis-ethical-ai --scenario "Exp-1" --hyperparameters "{\"max_depth\": 10}" --output "report.pdf"

Core Functionality

Bias Detection

The package implements sophisticated bias detection algorithms that analyze model predictions across different protected attributes:

  • Statistical Parity: Measures the difference in positive prediction rates between groups
  • Equalized Odds: Assesses fairness in terms of true positive and false positive rates
  • Individual Fairness: Evaluates consistency in predictions for similar individuals

Fairness Metrics

Comprehensive fairness assessment with multiple metrics:

Metric Description Range
Demographic Parity Equal positive prediction rates across groups 0-1 (closer to 0 is fairer)
Equal Opportunity Equal true positive rates across groups 0-1 (closer to 0 is fairer)
Predictive Rate Parity Equal precision across groups 0-1 (closer to 0 is fairer)
Bias Score Combined bias metric 0-∞ (lower is better)

Compliance Reporting

GDPR Compliance: Ensures data protection, right to explanation, and automated decision-making transparency.
AI Act Compliance: Validates high-risk AI system requirements including bias assessment and documentation.

Real-time Monitoring

Continuous monitoring capabilities with automated alerts:

  • Real-time bias detection in production models
  • Automated alerts when bias thresholds are exceeded
  • Performance tracking over time
  • Drift detection in model behavior

Mitigation Suggestions

Intelligent recommendations for bias mitigation:

  • Data preprocessing strategies
  • Model retraining recommendations
  • Feature engineering suggestions
  • Algorithm selection guidance

Advanced Usage

# Comprehensive audit with all features (v1.3.0)
validator = EthicalAIValidator(config={'bias_threshold': 0.1, 'fairness_threshold': 0.8})

# Perform complete audit
bias_report = validator.audit_bias(predictions, true_labels, protected_attributes)
fairness_metrics = validator.calculate_fairness_metrics(predictions, protected_attributes)
mitigations = validator.suggest_mitigations(bias_report)

# Include scenario/hyperparameters and thresholds in the report
metadata = {
    'model_name': 'MyModel',
    'scenario': 'A/B test - balanced',
    'hyperparameters': {'max_depth': 8, 'min_samples_split': 5},
    'bias_report': bias_report,
    'fairness_metrics': fairness_metrics
}
audit_criteria = {'bias_threshold': 0.1, 'fairness_threshold': 0.8}
report_path = validator.generate_compliance_report(metadata, audit_criteria, output_path='example_reports/ab_test.pdf')

# Real-time monitoring
alerts = validator.monitor_realtime([[1,0,1,0,1,0]])

What's New in v1.3.0

  • Scenario & Hyperparameters in reports with a structured table.
  • Hyperparameter Impact Analysis with risk and rationale.
  • Feature Contribution Disparities section (uses SHAP when available).
  • Configurable thresholds via audit_criteria or validator config.
  • CLI flags: --scenario, --hyperparameters, --output.

API Reference

Method Description Parameters
audit_bias() Perform comprehensive bias detection predictions, true_labels, protected_attributes
calculate_fairness_metrics() Calculate fairness metrics predictions, protected_attributes
generate_compliance_report() Generate compliance report bias_report, fairness_metrics, model_info
monitor_realtime() Set up real-time monitoring model_predictions, protected_attributes, alert_threshold
suggest_mitigations() Get bias mitigation suggestions bias_report

Dependencies

  • numpy ≥ 1.16.0 - Numerical computing
  • pandas ≥ 0.20.0 - Data manipulation
  • scikit-learn ≥ 0.20.0 - Machine learning utilities
  • reportlab ≥ 3.4.0 - PDF report generation

Testing

python -m pytest tests/ -v
✅ All tests passing! The package has been thoroughly tested for bias detection, fairness metrics, compliance reporting, and real-time monitoring functionality.
⚠️ Important: This package is designed to work with Python 2.7+ and Python 3.x. Make sure to use compatible versions of dependencies for your Python version.

License

MIT License: This project is licensed under the MIT License, making it suitable for both open-source and commercial use.

License Permissions

  • ✅ Commercial Use: Use in commercial applications and products
  • ✅ Modification: Modify and adapt the code for your needs
  • ✅ Distribution: Distribute copies of the software
  • ✅ Private Use: Use in private and public projects
  • ✅ Patent Use: Use in patent-eligible applications

License Requirements

  • 📋 Copyright Notice: Include the original copyright notice
  • 📋 License Text: Include the MIT license text
  • 📋 Attribution: State any modifications made

Third-Party Dependencies

All dependencies are BSD-3-Clause licensed and compatible with MIT:

  • numpy: BSD-3-Clause
  • pandas: BSD-3-Clause
  • scikit-learn: BSD-3-Clause
  • reportlab: BSD-3-Clause
✅ Commercial Friendly: The MIT License allows unlimited commercial use, modification, and distribution.