67 jobs for "Calibration Specialist"

Responsibilities

About the Team
Generative AI and large language models are reshaping the core capabilities of global content platforms - and high-quality training and evaluation data is one of the key factors that determines a model's ceiling.

We are the AI Data Service & Operations team behind TikTok's international products, responsible for producing multilingual, multimodal data assets across both safety and non-safety domains. The standards we set and the data we deliver power two critical fronts: TikTok's global content ecosystem governance strategy on one side, and the training and evaluation pipelines for AI/large language models on the other. In short, we produce the fuel that helps models truly understand content, communities, and users around the world.

The AI Evaluation & Enablement Specialist is responsible for evaluating content and AI/model performance through data annotation, analysing evaluation data, and applying AI, analytics and automation capabilities to improve evaluation accuracy, coverage and efficiency. The role also enables BPO quality improvements through calibration, performance insights and training intelligence, while partnering with cross-functional teams to identify opportunities for model, process and operational improvement.

We are looking for talented individuals to join our team. As a graduate, you will get opportunities to pursue bold ideas, tackle complex challenges, and unlock limitless growth. Successful candidates must be able to commit to an onboarding date by the end of the year. Please state your availability and graduation date clearly in your resume. Candidates can apply to a maximum of two positions and will be considered for jobs in the order you apply. The application limit is applicable to our Company and its affiliates' jobs globally. Applications will be reviewed on a rolling basis - we encourage you to apply early.

Key Responsibilities
- Analyze evaluation results to identify error patterns, trends, anomalies, performance gaps and potential risks, translating insights into actionable recommendations.
- Support and maintain evaluation datasets, analytical views and reports to support data-driven performance insights and decision-making.
- Support calibration activities across evaluators and BPO teams to ensure consistency, accuracy and alignment in evaluation decisions, policy application and SOP adoption.
- Identify BPO performance, knowledge and process gaps, and translate findings into targeted training, coaching, enablement and performance improvement initiatives.
- Drive continuous improvement in evaluation quality, scalability and efficiency by partnering with Product Ops, cross-functional teams and BPO stakeholders, while supporting AI experimentation, automation, tool testing and technology-enabled workflow adoption.

What is expected from this role?
- Content & Model Evaluation - accurate and consistent assessment of content and AI/model performance through data annotation/labelling
- Evaluation Intelligence - actionable insights from evaluation data, trends, errors and performance patterns.
- BPO Enablement - improved BPO capability through calibration, training intelligence and targeted performance interventions.
- Evaluation Efficiency - increased coverage and reduced manual effort through AI, automation and workflow optimisation.
- Model & Process Improvement - actionable feedback that contributes to improved model and operational performance.
- Evaluation Governance - consistent application of evaluation standards, methodologies and scorecards.

Qualifications

Minimum Qualification(s)
- Currently pursuing an undergraduate degree
- Excellent English proficiency (listening, speaking, reading, writing) with accurate comprehension of English video and text content; IELTS 7.5 / TOEFL 105 or equivalent
- Familiarity with Excel and other common data-processing tools
- Clear logic and strong structured decomposition, abstraction, synthesis, judgment, and communication skills. Detail-oriented, with an independent view of what constitutes high-quality data.

Preferred Qualification(s)
- Majoring in law, philosophy, economics, journalism, languages, social sciences, or computer science related fields
- Internship or project experience in data quality, content moderation, compliance, content operations, AI data, or annotation
- Deep experience with mainstream foundation-model products such as ChatGPT, Claude, Gemini, or Doubao, and a basic understanding of AI capability boundaries

Responsibilities

Qualifications

Qualifications

Minimum Qualification(s)
- Currently pursuing an undergraduate degree
- Excellent English proficiency (listening, speaking, reading, writing) with accurate comprehension of English video and text content; IELTS 7.5 / TOEFL 105 or equivalent
- Familiarity with Excel and other common data-processing tools
- Clear logic and strong structured decomposition, abstraction, synthesis, judgment, and communication skills. Detail-oriented, with an independent view of what constitutes high-quality data.

Preferred Qualification(s)
- Majoring in law, philosophy, economics, journalism, languages, social sciences, or computer science related fields
- Internship or project experience in data quality, content moderation, compliance, content operations, AI data, or annotation
- Deep experience with mainstream foundation-model products such as ChatGPT, Claude, Gemini, or Doubao, and a basic understanding of AI capability boundaries

About TikTok

TikTok is the leading destination for short-form mobile video and our mission is to inspire creativity and bring joy.

Size: 5,001 - 10,000 employees
Industry: Entertainment & Gaming

View company profile