Privacy Policy

AI Evaluator is an LLM-as-a-Judge evaluation platform operated by the AI Evaluator team. Our website is aievaluator.dev.

For privacy questions, contact us at support@aievaluator.dev.

When you create an account:

• Email address (from your OAuth provider: Google, GitHub, GitLab, or Bitbucket)
• Name (from your OAuth provider)
• OAuth provider name

When you use the platform:

• Evaluation data: questions, agent responses, scores, metrics
• Dataset contents you upload
• API key metadata (names, usage timestamps)
• BYOK provider names (keys themselves are encrypted and we cannot read them)
• Organization and team member data (emails, roles)

Automatically:

• IP address (stored in audit logs for enterprise compliance)
• User agent (browser/OS info)
• Usage statistics (evaluation counts, token consumption)

• To provide the evaluation service (your agent evaluations, scores, reports)
• To authenticate you and maintain your session
• To enforce plan limits and rate limits
• To generate audit logs for enterprise compliance
• To improve the service (aggregated, anonymized usage stats)

We do NOT:

• Sell your data to third parties
• Share your data with advertisers
• Use your evaluation data to train AI models

All data is stored on servers in Frankfurt, Germany (Hetzner), within the European Union. Data is encrypted in transit (HTTPS/TLS 1.3) and sensitive values are encrypted at rest (AES-256).

• Evaluation results: retained until you delete them or close your account
• Datasets: retained until you delete them
• Audit logs: retained indefinitely (immutable, for compliance)
• Usage logs: retained for the life of your account

• Access: request a copy of all data we hold about you
• Rectification: correct inaccurate data
• Erasure: delete your account and all associated data
• Portability: export your data in a machine-readable format
• Object: object to processing of your data

To exercise any of these rights, email support@aievaluator.dev. We respond within 30 days.

We use minimal cookies:

• Essential: session cookie for authentication (no consent required)
• CookieScript: remembers your cookie preferences

We do not use tracking cookies, analytics cookies, or advertising cookies.

• Stripe: payment processing (when you upgrade to a paid plan)
• OAuth providers: Google, GitHub, GitLab, Bitbucket — for login only
• LLM providers: DeepSeek, OpenAI, Anthropic — your evaluation prompts are sent to these APIs as part of the judge evaluation process
• CookieScript: cookie consent management

If you require a DPA for GDPR compliance, contact us at support@aievaluator.dev. We provide a standard DPA that covers:

• Nature and purpose of data processing
• Data subject categories and types of personal data
• Technical and organizational security measures
• Sub-processor list
• Data breach notification procedures
• Data deletion upon contract termination

We will notify you of material changes via email or through the platform. Continued use after changes constitutes acceptance.