2,233 jobs for "Quality Specialist"

AI/ML Evaluation Specialist, Human Data

At Apple - Cupertino

Posted on Oct 17

View job Apply on company site

Director - Specialist Consulting, VCA-X, Visa Consulting & Analytics

At Visa - Bengaluru

Posted on Oct 12

View job Apply on company site

ADAS Design Release Engineer (DRE) for Front Camera Module

At General Motors - Warren

Posted on Oct 21

View job Apply on company site

Manager, Appraisal Review

At Zillow - Bengaluru

Posted on Oct 25

View job Apply on company site

FSQR Specialist

At Cargill - Kalmthout

Posted on Oct 21

View job Apply on company site

GE Vernova Materials Quality Internship - Summer 2027

At GE Vernova - Atlanta

Posted on Oct 22

View job Apply on company site

Transactions Specialist I - Mail Extractor / Weekend Shift

At JPMorgan Chase - Binghamton

Posted on Oct 14

View job Apply on company site

Senior Consultant - AI Red Teamer & Ecosystem Quality

At Infosys - TOKYO

Posted on Oct 28

View job Apply on company site

Senior Director, Head of Specialist & Consulting Delivery VCA-X

At Visa - Bengaluru

Posted on Oct 12

View job Apply on company site

Final Inspection, Senior Technical Specialist

At Southern California Edison (SCE) - Pomona

Posted on Oct 28

View job Apply on company site

Manager, Customer Care Training

At iRhythm Technologies - Remote / Flexible

Posted on Nov 7

View job Apply on company site

Associate Manager, Customer Support - Rotational Shift

At NAVEX - Bengaluru

Posted on Nov 4

View job Apply on company site

Operations Quality Data Analyst

At Jabil - Bray

Posted on Oct 30

View job Apply on company site

Technical Sales Specialist (Early Careers)

At Hitachi Energy

Posted on Nov 6

View job Apply on company site

GTS EMEA Senior Technical Support Specialist

At BD - Kelberg

Posted on Nov 3

View job Apply on company site

INGENIERO SALA DE PRUEBAS

At Hitachi Energy - Dosquebradas

Posted on Nov 3

View job Apply on company site

Welding and Quality Inspector

At Hitachi Energy - Maneja

Posted on Oct 30

View job Apply on company site

Quality Inspector, Data Center (Starlink)

At SpaceX - Memphis

Posted on Oct 12

View job Apply on company site

da Vinci Specialist

At Intuitive - Remote / Flexible

Posted on Nov 10

View job Apply on company site

da Vinci Specialist - Chattanooga, TN

At Intuitive - Remote / Flexible

Posted on Nov 9

View job Apply on company site

Apple

AI/ML Evaluation Specialist, Human Data

Cupertino, CA$144,600 – $263,800Posted 23 days ago

Apply on company site View full job

At Apple, we don't just build products - we build experiences fueled by world-class data. The

Human-centered AI team within Apple Services Engineering is looking for an ML Evaluation Specialist, Human Data to join our Data Quality and Operations division to spearhead complex, multi-stakeholder operations that specialize in data collection, curation, annotation, and human evaluation efforts across Apple Music, App Store, TV+, Podcasts, and Books.

Description

In this role, you will own the operational strategy and continuous improvement of large-scale, multilingual human data programs, from designing onboarding scaffolds that progressively build annotator calibration, to analyzing annotator behavior patterns to identify where automation can offload low-judgment decisions, to enforcing quality frameworks that close the loop between annotator struggle and task redesign. You will identify where human judgment is essential and where it could be better directed, then build the scaffolding, automation, and feedback systems that let annotators focus their cognitive energy where it matters most. Because this work cuts across engineering, data science, research, procurement, and legal, a critical part of the role is serving as the connective tissue between teams who each own a piece of this space, aligning on shared standards, surfacing gaps, and ensuring that insights from the annotation layer inform upstream decisions about task design and tooling. You will bring a point of view on human data best practices and translate it into scalable, human-centered approaches that make generative AI features safer and more reliable.

The ideal candidate brings a rare combination of technical depth and program execution skills. You are comfortable designing and deploying sophisticated data pipelines in the morning, and then seamlessly transitioning to present comprehensive quality rectification strategies to stakeholders in the afternoon. You care deeply about data quality and human alignment, have a creative and systematic approach to finding and fixing problems, and find motivation in wide-ranging work whose impact shows up in everyday Apple experiences.","responsibilities":"Lead the end-to-end execution of human data collection programs for multilingual, multimodal, and multi-turn AI features, from intake and scoping, to delivery and retrospective

Estimate and maintain project timelines, capacity needs, and cost while anticipating and proactively resolving bottlenecks to ensure timely execution

Design and own the measurement framework for human data collection initiatives, defining key indicators such as spend, speed, inter-rater reliability, and volume, and building reporting systems that surface actionable insights to stakeholders

Build and implement human data quality frameworks, including developing statistical process controls and behavioral signals to proactively detect and remediate quality degradation throughout the annotation lifecycle

Apply human-centered AI principles to influence data collection task design, identifying and reducing sources of cognitive burden that impact human performance, consistency, and wellbeing

Systematically analyze data collection workflows to identify inefficiencies and scalability gaps, then design and implement innovative workflows (e.g. agent- and machine-in-the-loop) and build the supporting data pipelines and ETL services to improve data quality and diversity while optimizing cost and lead time

Partner with Legal, Privacy, and New Product Security to design and implement compliant data collection and user study programs, ensuring all human data workflows adhere to Apple's privacy, governance, and regulatory standards

Design and own the data collection lifecycle with external and internal workforces, including building onboarding and calibration programs, performance frameworks, and quality audits that ensure reliable, high-quality deliverables

Act as the connective tissue across engineering, product, legal, security, procurement, and vendor teams, ensuring alignment, clear communication, and follow-through

Preferred Qualifications

Master's degree or higher in Cognitive Science, Linguistics, or a related field that includes an experimental or empirical component

2+ years of experience owning data strategy for frontier AI development and evaluation, with experience in human alignment methodologies and agentic GenAI systems

Experience managing external vendor or workforce partners at scale

Familiarity with AI Safety and Responsible AI principles, including experience applying them to data collection or annotation workflows

Strong organizational skills and execution-oriented mindset; ability to balance attention to detail with big-picture thinking in an environment where program scope and priorities evolve quickly

Excellent written and verbal communication skills; able to translate technical concepts for non-technical stakeholders

Minimum Qualifications

Bachelor's degree or higher in Cognitive Science, Linguistics, or a related field that includes an experimental or empirical component

4+ years of experience defining and leading cross-team human data programs for AI/ML, including annotation operations, quality frameworks, and evaluation strategies, within an NLP/NLU or generative AI environment

Proficiency in programming and data languages (Python, R, SQL) to process, analyze, query large datasets, extract insights, automate tasks, and monitor program performance

Hands-on experience designing and managing 0→1 human-in-the-loop data collection, annotation, and evaluation initiatives, including driving and incorporating agentic workflows to improve quality and scalability

Experience working with diverse data types (e.g., speech, text, multimodal) across multiple languages

Expertise in end-to-end data annotation quality management, including the ability to develop statistical process controls and data quality metrics

Familiarity with privacy-preserving data handling practices and compliance frameworks

Demonstrated success optimizing data pipelines and workflows to improve quality, reduce lead time, and scale operations

Experience working cross-functionally with engineering, data science, legal, privacy, and third-party suppliers

Pay & Benefits

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $144,600 and $263,800, and your base pay will depend on your skills, qualifications, experience, and location.

Apple employees also have the opportunity to become an Apple shareholder through participation in Apple's discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple's Employee Stock Purchase Plan. You'll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses - including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits

Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.

About Apple

Working at Apple means doing more than you ever thought possible and having more impact than you ever imagined.

Size: 10,000+ employees
Industry: Technology

View company profile