Senior Data Infrastructure Engineer
What You'll Work On
- Data pipelining infrastructure: Our data moves around a lot. We need it available for batch and real-time compute, and we need to expose it for use by both our backend platform and data analytics. Message brokers, key-value stores, and job schedulers are some tools our team works with. You’ll help identify improvements for our architecture, designing and implementing solutions to get data around more efficiently.
- Spark: As data engineers, we want to move fast, and we want our code to move fast as well. We have a batch compute platform using Spark, and we’ve recently built out a near-realtime platform using Spark Streaming. We are continuing to increase the performance of our data pipelines while simultaneously increasing the complexity of the jobs that run on top of them. You'll be expected to improve the infrastructure while you implement on top of it.
- Data quality: A model is only good if it is correct and built on recent data. We put a strong emphasis on data quality. We write unit tests to verify the functional correctness of each module and meta-tests to guard against common programming errors. Throughout our data pipelines we run automated sanity checks on live data, alerting if any data is stale or values fall outside of expected ranges.
What You Have
- 7+ years in a data engineering role.
- Knowledge of the Spark/Hadoop ecosystem.
- Experience scaling a platform to handle a growing user base.
- Advanced knowledge of Java. Familiarity with Scala preferred.
- A BS in computer science or a related field. An MS or PhD is a plus.
Meet Some of Wealthfront's Employees
Whether it’s crafting user flows, designing a direct mail piece, or working on website visuals, Aly uses her passion for human and digital interaction to ensure a great user experience.
Back to top