Data Scientist, Machine Learning Evaluation

    • San Francisco, CA

Data Scientist, Machine Learning Data

Machine Learning is a critical part of Adobe's Cloud offering. Adobe Clouds enable customers to create and manage digital content, such as assets, composites, 3D, documents etc., and digital experience and transformations.

Creative Cloud is focused on visual imagination and creation, and includes such well-known products as Photoshop CC, Illustrator CC, Lightroom CC, and Adobe Stock. Document Cloud is centered around the creation and management of textual documents, including Acrobat DC and Adobe Sign. Experience Cloud is the set of offerings for the large-scale enterprise, covering marketing, advertising, and analytics.

We are building new machine learning platform, called Adobe Sensei, that powers machine learning and AI across all of these Adobe Cloud product lines. This platform will enable thousands of applied researchers, millions of users, and billions of content pieces. The objective is to make machine learning offerings a world-class, leading-edge, differentiating product in Adobe Cloud ecosystem. We match the pace, innovation and excitement of a startup, backed by the resources and infrastructure of Adobe!

A crucial piece of developing machine learning solutions is the evaluation process. How do we know the quality of our algorithms and models? What is the ground truth we use for judging them? Where do they succeed and how do they fail? These problems are even more subtle in the complex, creative domains of Adobe products - for example, how should we judge the quality of similarity search, where a user finds new images with an image as the query?

Equally core to the ML process is the training set itself. Such training sets often include human judgments, elicted at large scale, which form the target that the ML process predicts.

How can you participate? We're looking for data scientists to define, drive, and coordinate evaluation and training set construction for the Applied Machine Learning team at Adobe. This will involve thinking hard about product quality, the role of machine learning in those products, and the capabilities of both A/B tests and human-judgment tasks to produce training sets for ML as well as quantitative evaluations that measure how well our AI/ML algorithms are doing their jobs

Responsibilities:
• Work closely with product, research, and engineering leaders to understand ML applications and define success
• Design evaluation tasks and guidelines for human judgment to measure the performance of machine-learned and other advanced algorithms.
• Design human judgment tasks that will create training sets for useful machine-learned models.
• Create a winning culture built on collaboration and shared accomplishments- have fun along the way!
Qualifications:
• BS or MS in a technical field (Computer Science, Physics, Math, Statistics) OR in a human-centered field such as Psychology, Linguistics, Human-Computer Interaction.
• Exposure to applied machine learning in an industrial setting
• Experience in validation and evaluation of machine learned models
• Strong quantitative orientation
• Understanding of issues around ground truth and training sets for ML, including human judgment and A/B testing
• Hands-on experience using third-party judgment services such as Crowdflower, Appen, or Amazon Mechanical Turk
At Adobe, you will be immersed in an exceptional work environment that is recognized throughout the world on Best Companies lists. You will also be surrounded by colleagues who are committed to helping each other grow through our unique Check-In approach where ongoing feedback flows freely. If you're looking to make an impact, Adobe's the place for you. Discover what our employees are saying about their career experiences on the Adobe Life blog and explore the meaningful benefits we offer.

At Adobe, you will be immersed in an exceptional work environment that is recognized throughout the world on Best Companies lists. You will also be surrounded by colleagues who are committed to helping each other grow through our unique Check-In approach where ongoing feedback flows freely.

If you're looking to make an impact, Adobe's the place for you. Discover what our employees are saying about their career experiences on the Adobe Life blog and explore the meaningful benefits we offer.

Adobe is an equal opportunity employer. We welcome and encourage diversity in the workplace regardless of race, gender, religion, age, sexual orientation, gender identity, disability or veteran status.

Marketo is a marketing automation platform that provides a wide variety of industries with data-based insights to inform successful customer interactions.

Marketo Company Image


Back to top