Senior Software Engineer, Cluster Interfaces

Cluster Interfaces is responsible for all of the entry points into Quantcast’s big data services. We develop a dashboard through which data pipelines are scheduled and executed.  The dashboard provides a holistic view into the company’s data pipelines, including their dependency trees, capacity needs, and execution times. It also allows users to define data retention policies, monitoring, and alerting. The dashboard is similar to the open source project Azkaban.

The team also develops and operates a SQL-On-MapReduce platform along with a data catalog. Quantcast’s SQL-On-MapReduce platform is very similar to Hive, providing company wide easy access to our core data assets.

Cluster Interfaces is looking for an individual to drive improvements to the systems used to schedule, debug and analyze map/reduce pipelines while facilitating the management of shared cluster resources and capacity planning.

Modernize the front-ends of these internal web-services using state of the art javascript frameworks, implement testing best practices, and develop rich service oriented APIs.  

Responsibilities

  • Consolidate the interfaces to Quantcast’s massive compute cluster
  • Design rich data-visualizations to aid with analysis and resource management tasks
  • Mentor and grow team members
  • Work to understand the needs of customers in order to serve them better
  • Contribute to the technical roadmap of the team

Requirements

  • BS in computer science or equivalent experience
  • Experience leading platform development for big data companies
  • Knowledge of authentication/authorization best practices in web frameworks
  • Mastery of JavaScript, HTML, and CSS
  • Depth in Java
  • Depth in a scripting language (python or ruby)
  • Proven track record of operational excellence



Quantcast is a fast-growing, late-stage, pre-IPO startup headquartered in San Francisco with offices around the world. We are committed to creating an inclusive and diverse environment where everyone can confidently be their authentic self.

Meet Some of Quantcast's Employees

Lindsey M.

Research Analyst

Lindsey looks at performance targeting pipelines for Quantcast to pinpoint inefficiencies. Then, she shares processed insights on customer targeting to make it better down the road.

Rodney C.

Senior Account Manager

Rodney manages a book of accounts in Northern California in order to assist clients. He helps integrate new products and strategies in order to build business.


Back to top