Sr. AI/ML QA Automation Engineer
The Apple Photos application is a comprehensive photo and video management solution that seamlessly integrates across the entire Apple ecosystem, enabling users to capture, organize, edit, and share their visual memories with unprecedented ease and intelligence. Working on the Photos team means contributing to one of Apple's most personal and widely-used applications, combining cutting-edge AI, elegant design, and robust engineering to help billions of users around the world preserve and relive their most important life moments.
This is a high-impact role where you'll work at the intersection of AI modeling, agentic workflows, information retrieval, software engineering, evaluation and metrics, and help us push the boundaries of how AI can transform Apple's products.
Description
This role blends traditional software QA skills with advanced evaluation methodologies for modern AI models, including LLMs, multimodal systems, and ML-driven product features. As a member of the team, you will work closely with experienced engineers and machine learning experts to qualify and refine features powered by vision, language, and cross-modal intelligence.
You will be responsible for designing rigorous evaluation strategies for both objective and subjective ML behaviors, creating reliable automated testing pipelines, and developing LLM-driven evaluators that complement human judgement. The ideal candidate is self-directed, creative, and comfortable with ambiguity, with strong technical and interpersonal skills. They have hands-on experience testing ML models directly, defining qualitative scoring rubrics, building reproducible evaluation frameworks, and ensuring that AI behavior is safe, consistent, and aligned with product and on-device constraints.","responsibilities":"Design and implement test strategies for qualifying subjective and ML-driven features
Perform Functional testing of existing/new features on various platforms across concurrent release vectors
Design and execute evaluation plans for multimodal ML models (vision, language, and cross-modal systems)
Develop and maintain robust testing frameworks for machine learning models
Automatic large scale data generation & Evaluation
Collaborate with QA teams to integrate models into existing test frameworks
Consider on-device ML performance constraints (latency, memory, energy) when designing tests and evaluation strategies
Want more jobs like this?
Get jobs in Cupertino, CA delivered to your inbox every week.
Preferred Qualifications
Experience Testing AI Models for accuracy, robustness, fairness, and performance
Experience developing LLM based automated evaluation frameworks
Expertise in Swift and/or Obj-C
Strong programming skills in Python and experience with ML/NLP libraries
3+ years of proven ability in machine learning, including hands-on work with LLMs
Understanding of prompt engineering, and retrieval-augmented generation (RAG)
Knowledge of statistics based evaluation approaches, ML training pipelines and accuracy improvements of ML systems
Minimum Qualifications
BS/MS or equivalent experience in Computer Science or related field
3+ years of experience working in Software Quality Assurance
Strong software engineering skills, including system design, development, testing, debugging, release and maintenance
Expertise with hands on experience in automated software testing, data validation, and ethical AI practices
Familiarity with LLM usage to improve efficiency of their daily work
Ability to evaluate ML models directly (vision or multimodal) and diagnose model failures using quantitative and qualitative methods
Expertise in Python
Proficiency with iOS, macOS, watchOS, tvOS or similar operating systems
Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicant .
Perks and Benefits
Health and Wellness
Parental Benefits
Work Flexibility
Office Life and Perks
Vacation and Time Off
Financial and Retirement
Professional Development
Diversity and Inclusion
Company Videos
Hear directly from employees about what it is like to work at Apple.