118 jobs for "Human Intelligence Manager"

Benchmarking Project Lead, Siri Evaluation

At Apple - Cambridge

Posted on Oct 17

View job Apply on company site

Sr. Manager, Product Management - Auto

At Capital One - Plano

Posted on Nov 4

View job Apply on company site

Data Analytics Manager - Client Legal Services

At Allstate - Remote / Flexible

Posted on Oct 28

View job Apply on company site

Machine Learning Manager, Siri Speech

At Apple - Cupertino

Posted on Oct 17

View job Apply on company site

Senior Manager, Application Engineering (Appian Platform)

At Vanguard - Hyderabad

Posted on Oct 24

View job Apply on company site

Delivery Consultant, Oracle HCM Cloud Functional Core HR

At Deloitte - Gilbert · Lake Mary

Posted on Oct 20

View job Apply on company site

Senior Manager, Product Management - Machine Learning and AI

At Capital One - McLean · Chicago

Posted on Oct 16

View job Apply on company site

Manager, Product Management - AI/ML & Data Platforms

At Capital One - McLean · Chicago

Posted on Oct 11

View job Apply on company site

Senior Machine Learning Engineering Manager, Responsible AI and Safety - AIML/SWE

At Apple - Cupertino

Posted on Oct 17

View job Apply on company site

Senior Manager, Corporate Communications - Research and Insights

At Vanguard - Malvern

Posted on Oct 22

View job Apply on company site

Manager, Fraud Analytics & Operations (FROM BOT)

At Vanguard - Hyderabad

Posted on Oct 22

View job Apply on company site

Market Access Benelux - Manager/Senior Manager

At CSL - Mechelen

Posted on Nov 2

View job Apply on company site

Global Customer Support Manager

At Cornerstone OnDemand - Heredia

Posted on Oct 31

View job Apply on company site

Enablement Hub Lead - India

At Cornerstone OnDemand - Hyderabad

Posted on Oct 24

View job Apply on company site

Manager, Global Customer Support

At Cornerstone OnDemand - Krakow

Posted on Oct 20

View job Apply on company site

Sr Manager Executive Protection

At Mattel - El Segundo

Posted on Oct 15

View job Apply on company site

Senior Software Engineer, Climate

At Flexport - Amsterdam

Posted on Nov 2

View job Apply on company site

Staff, Product Manager - Identity

At Twilio - Remote / Flexible

Posted on Oct 26

View job Apply on company site

Senior Project (Design) Mgr

At Intuitive - Sunnyvale

Posted on Nov 4

View job Apply on company site

Equity Compensation Program Manager

At Roblox - San Mateo

Posted on Oct 26

View job Apply on company site

Apple

Benchmarking Project Lead, Siri Evaluation

Cambridge, United KingdomPosted 22 days ago

Apply on company site View full job

Join the team redefining what a deeply personal and integrated assistant can be. As part of the Siri organization, you will help shape one of the world's most widely used AI assistants, powered by our next-generation of Apple Intelligence, with capabilities like personal context understanding and on-screen awareness, built with privacy from the ground up. Your work will have direct, meaningful impact for users across iOS, iPadOS, macOS, watchOS, and visionOS. This is a rare opportunity to build at the intersection of cutting-edge AI and human-centered design, shipping technology that is centered around users and their needs.

Evaluation is at the heart of how we build our product. As Siri AI becomes more and more powerful and offers ever richer experiences to our users, our evaluations have to keep pace. We are seeking a senior manager to help drive our evaluation efforts. The role will involve managing teams working on evaluation development and data science, and leading high-impact initiatives to bring state-of-the-art agentic evaluation to the whole Siri team.

Description
As part of the work on next generation Siri, we are developing novel measurements of its quality. To ensure that the evaluation systems we are building are reliable, we plan benchmarking their accuracy on a wide range of features, locales, and platforms using humans in the loop.","responsibilities":"Design and execute efficient data collection processes using humans in the loop
Lead annotation efforts for various languages and devices
Plan and manage the budget of annotator resourcing with Finance, Annotation Ops, and International team partners.
Track and improve the quality of the human judgements through revision of annotator training materials, clear annotation questions, annotator training, and efficient reviewing mechanisms
Collaborate with other engineering teams to design and build a tooling ecosystem for managing and browsing rich datasets

Preferred Qualifications
Attunement to computational linguistics, language quality, human in the loop evaluation
Good engineering practices to create sustainable and easy to use data management pipelines
Python experience and other tools for data collection and visualisation

Minimum Qualifications
Agentic Coding proficiency to achieve data-science, data collection and visualisation tasks
Good understanding of metrics, crowd science, data collection, annotation analysis, statistics
Ability to work independently and cross-functionally to integrate in partner team reporting systems and pipelines
Excellent communication skills and the ability to thrive in a highly collaborative work environment

About Apple

Working at Apple means doing more than you ever thought possible and having more impact than you ever imagined.

Size: 10,000+ employees
Industry: Technology

View company profile