Expert Talent Pool, Evals and Benchmarks (Associate)
Platinum ? Clera · remote
ApplyJob description
ABOUT THE ROLE
Join a team that designs and runs human-led evaluations of AI models. You will help measure model capabilities by creating rigorous questions, rubrics, and comparative assessments, contributing to better understanding of AI performance.
WHAT YOU'LL DO
• Write questions and tasks that test specific model capabilities.
• Develop rubrics that define strong, partial, and weak answers.
• Create reference answers and edge-case scenarios to challenge models.
• Grade model outputs against rubrics and explain each score clearly.
• Analyze incorrect answers, group error patterns, and document findings.
• Compare multiple models using the same prompts.
• Validate benchmarks and flag questions that may no longer provide reliable measurements.
• Support small studies of how people evaluate or interact with model outputs.
WHAT WE'RE LOOKING FOR
• Zero to five years of professional experience; prior AI-data work is not required.
• Experience or training in QA, test engineering, assessment design, psychometrics, UX research, data analysis, ML evaluation, research support, or a subject area involving rigorous question writing.
• Experience writing test cases, exam questions, rubrics, or evaluation tasks, and reviewing work consistently against written criteria.
• Ability to use spreadsheets or Python to organize and analyze results, and to identify and document error causes.
• Experience with evaluation tools such as Inspect, lm-eval-harness, HELM, or promptfoo, model comparisons, or user and rating studies is useful.
• Clear, professional written communication.
COMPENSATION & BENEFITS
Compensation is $30 to $60 per hour, depending on project, field, and experience. Payment is for hours worked.
LOCATION
Fully remote. Projects typically run from one week to several months.
Pro gives you a designed CV, CV tailoring, matches and an interview coach, with the company and apply link on every job.
Get 3 months of Pro: €69or €29/month · €89/year
Remote jobs available today
34,939 live remote jobs · 5,189 added in the last 7 days · 2,938 employers hiring · 34,834 passed every scam check · 5 markets: USA, UK, Canada, Australia, Europe
Live from our job database, updated 2026-11-21. Every job shows where the employer can hire from. How we check every job
Tailor your CV free for remote jobs like this
Similar remote jobs
- Expert Talent Pool, Generation and Writing Associate · Clera
- Head of Growth · Clera
- Expert Talent Pool, Safety Testing Associate · Clera
- Piping and Instrumentation Diagram (P&ID) Specialist · Clera
- Mid-Market Sales Leader - EMEA · Bitwarden
- Senior Digital Strategist (w/m/d) · Bertelsmann
- Senior Systems Engineer · General Dynamics UK
- Technical Product Manager, Identification & JS Agent · Fingerprint
- Senior Data Platform Engineer · Fingerprint
- Senior Corporate Enterprise Platform Engineer, Windows · Dropbox
- Senior Client Platform Engineer, Endpoint Platform & Automation · Dropbox
- Manager, Developer Advocacy, EMEA · Temporal Technologies
More remote jobs like this
- Remote jobs in Europe · 10,467 jobs
- Remote jobs at Clera · 49 jobs
- Remote jobs open to the USA · 20,576 jobs
Remote jobs in USA · Remote jobs in UK · Remote jobs in Canada · Remote jobs in Europe · Remote jobs in Australia
Latest remote jobs
- Partner Success Manager · Koddi
- Customer Care Agent - AmTrav (Remote) · Perk
- Overnight Customer Care Agent - AmTrav (Remote) · Perk
- Sales Account Manager, Molecular Biology and Protein · GenScript
- Sales Account Manager, DACH · GenScript
- Principal Systems Engineer · General Dynamics UK
- Recruiting Coordinator · Genesis AI
- Outcomes Associate · General Assembly
- Strategic Partnerships and Sales Enablement Manager · Gen Digital
- Strategic Partnerships Manager · Gen Digital