Lead Data Engineer - Historical Metrics Query
- New York, NY
We're on a mission to build the best platform in the world for engineers to understand and scale their systems, applications, and teams. We operate at high scale—trillions of data points per day—providing always-on alerting, metrics visualization, logs, and application tracing for tens of thousands of companies. Our engineering culture values pragmatism, honesty, and simplicity to solve hard problems the right way.
The Historical Metrics Query team allows our customers to query metrics that are older than 24 hours. The team is constantly trying to improve query performance for large data sets that span days, months or even years. And all of this is across multiple cloud environments.
As an Engineering Team Lead, you will manage a team of data engineers, own significant chunks of our architecture, design and build systems at scale, and shape product decisions. You'll work on challenging projects, make an impact, and grow as an engineer and a lead. And since historical metrics are an important part of our product, you’ll work closely with other teams in our Metrics Organization.
- Manage a team of engineers, develop their skills and help advance their careers
- Ensure that the team consistently ships high quality code and architecture
- Build distributed, high-volume data pipelines that power the core Datadog product
- Do it with Spark, Luigi and other open-source technologies
- Work all over the stack, moving fluidly between programming languages: Scala, Java, Python, Go, and more
- Join a tightly knit team solving hard problems the right way
- Own meaningful parts of our service, have an impact, grow with the company
- You have a BS/MS/PhD in a scientific field or equivalent experience
- You have built and operated data pipelines for real customers in production systems
- You are fluent in several programming languages (JVM & otherwise)
- You enjoy wrangling huge amounts of data and exploring new data sets
- You value code simplicity and performance
- You want to work in a fast, high growth startup environment that respects its engineers and customers
- You enjoy people management and have experience with leading others to greatness
- You are deeply familiar with Spark and/or Hadoop
- In addition to data pipelines, you’re also quite good with Kubernetes and cloud technology
- You’ve built applications that run on AWS or GCP
- You’ve built your own data pipelines from scratch, know what goes wrong, and have ideas for how to fix it
Equal Opportunity at Datadog:
Datadog is an Affirmative Action and Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements.
For more information on how we maintain the privacy of the information you submit as part of your application, please refer to our Applicant and Candidate Privacy Notice.
Back to top