Posted: Oct 20, 2021
Role Number: 200180167
Data Engineers think differently. We value engineering rigor and partnership, unit testing and pragmatism, best practices and empathy, all in equal parts. Data engineers understand that driving meaningful impact is more than enabling our own work; true impact comes from helping others to unlock their full potential. We measure our success in the accomplishment and enablement of others. The vision of the AI/ML Data organization is to improve products by using data as the voice of our customers. Data Engineers assure that voice is heard clearly by those that need to hear it most. We build optimized and robust engineering systems, curated data products, and collaborate with both data scientists and engineering partners to remove noise, ensure quality, and streamline discovery.
- 5+ years of data engineering experience
- 3+ years of coding in Python, Java, Scala, or other mainstream programming languages
- 3+ years working with technical partners from multi-functional groups to collect business requirements, build consensus, and handle expectations
- 3+ years building and monitoring robust data pipelines in distributed computing environments
- Experience with designing schema for instrumentation and logging systems, OR experience supporting and partnering with data science is desired
Partner with data scientists, analysts, software engineers, and product managers to understand and collect use cases. Curate a generalized data model based on those use cases that meets the needs of the many. Instrument the proper logging to make sure the data you need is being generated. Build a robust data pipeline to populate your model with mission-critical data. Educate your consumers on how to access your model, assuring transparency in logic definitions. Push your learnings up the stack, making sure that all upstream systems are aware of your needs and their impact. Skills we're looking for: • Python • SQL • Data Modeling • Distributed computing experience like Spark, MapReduce, or Hive (Spark preferred) Desired Experience: • HDFS, S3, or other cloud data storage technologies • Presto, Hive, Solr, or Druid • Data engineering in a cloud environment
Education & Experience
B.S., M.S., or Ph.D. in Computer Science, Computer Engineering, or equivalent practical experience.