135 jobs for "Quality Specialist (Starship)"

Responsibilities

Our Business Integrity team has a strong user focus and a dedication to technical excellence. We aim to meet our users' needs with reliable and high-performing platforms and services.

We are seeking a Model Optimization Specialist to join our dynamic team. You will be responsible for designing and optimizing workflows that improve machine model performance in content moderation - training models to make accurate policy application, rejection, and leakage judgements while maintaining low False Decision Rates (FDR) and improving machine coverage. You will work at the intersection of policy, data, and AI to drive measurable quality outcomes across ad content review systems.

Responsibilities
- Model Quality & Workflow Design: Design, manage, and optimize end-to-end workflows to improve machine model performance in content policy enforcement - including signal detection, rejection accuracy, and leakage reduction. Develop training data pipelines, QA processes, and performance tracking systems aligned to model improvement goals.
- Adversarial Testing & Risk Identification: Conduct structured adversarial testing on AI models, features, and content policies to surface vulnerabilities, edge cases, and emerging risk trends. Explore model behaviour across contexts and user journeys to identify failure modes not captured in standard evaluations.
- Root Cause Analysis & Error Optimization: Conduct structured root cause analysis (RCA) on model errors - including overkills, leakages, and misclassification - and translate findings into actionable model improvement recommendations. Partner with Algo and product teams to close root causes through memory insertion, threshold adjustments, rewrite rules, or policy iteration.
- Data Analysis & Reporting: Analyze model training and operational performance data to generate actionable insights through reports and presentations. Use data to surface trends, support business decisions, and inform future model training directions or policy adjustments.
- Cross-Functional Stakeholder Partnership: Partner with policy, product, business, and operations teams to validate mitigation strategies, align on quality metrics, and ensure root cause closure. Communicate quality insights, risk trends, and model performance updates clearly to leadership and cross-functional stakeholders.
- Guidelines & Knowledge Management : Develop and maintain technical guidelines, SOPs, and casebooks to ensure consistent, high-quality decision-making across QA and annotation workflows. Drive alignment between policy intent and operational execution.

Qualifications

Minimum Qualification(s)
- Minimum 1 year of experience in Quality Audit, Quality Governance, or Content Moderation Policy & SOP management.
- Prior experience with AI/ML model quality assessment, data annotation, or model evaluation (LLM/VLM preferred).
- Strong data sensitivity and analytical skills - able to identify patterns, surface trends, and present findings with clear, actionable data evidence to support business decisions.
- Ability to design structured workflows and translate qualitative policy requirements into machine-actionable signals that drive measurable model improvement.
- Proficiency in data and productivity tools (Excel, Lark Base/Bitable, or equivalent); comfortable working with dashboards, reports, and performance tracking systems.
- Excellent written and verbal communication in English. Genuine curiosity in AI, large language models, and human-AI collaboration - willing to learn, explore, and stay ahead of emerging model behaviours and risks.
- Comfortable balancing independent judgment with cross-functional collaboration in a fast-paced environment, with a strong ownership mentality and ability to manage multiple projects to closure.

Preferred Qualification(s)
- Prior exposure to content risk, trust & safety, or ads integrity workflows.

Responsibilities

Qualifications

Qualifications

Minimum Qualification(s)
- Minimum 1 year of experience in Quality Audit, Quality Governance, or Content Moderation Policy & SOP management.
- Prior experience with AI/ML model quality assessment, data annotation, or model evaluation (LLM/VLM preferred).
- Strong data sensitivity and analytical skills - able to identify patterns, surface trends, and present findings with clear, actionable data evidence to support business decisions.
- Ability to design structured workflows and translate qualitative policy requirements into machine-actionable signals that drive measurable model improvement.
- Proficiency in data and productivity tools (Excel, Lark Base/Bitable, or equivalent); comfortable working with dashboards, reports, and performance tracking systems.
- Excellent written and verbal communication in English. Genuine curiosity in AI, large language models, and human-AI collaboration - willing to learn, explore, and stay ahead of emerging model behaviours and risks.
- Comfortable balancing independent judgment with cross-functional collaboration in a fast-paced environment, with a strong ownership mentality and ability to manage multiple projects to closure.

Preferred Qualification(s)
- Prior exposure to content risk, trust & safety, or ads integrity workflows.

About TikTok

TikTok is the leading destination for short-form mobile video and our mission is to inspire creativity and bring joy.

Size: 5,001 - 10,000 employees
Industry: Entertainment & Gaming

View company profile