Slack is looking for an experienced engineer to help design and scale our batch and streaming data pipelines to help our engineers, operations team, product managers, and analysts make better decisions with data.
Our backend data infrastructure is built on Kafka, Hadoop, Hive, Presto, Spark, and MySQL. We are looking for engineers that understand that simplicity and reliability are aspects of a system that can’t be tacked on but are carefully calculated with every decision made. We are also very nice people and are happy to help you learn what you need to know to work on Slack.
This position is based in San Francisco and reports to the Director of Data Engineering.
If you were to join Slack, here are the kinds of things you would do over the course of a typical week:
- Work with the backend engineering team to design systems that reliably transport and process billions of events per day
- Help our analysts and engineers build and optimize data pipelines to understand our large and ever-growing user base
- Whiteboard a fix to a scaling problem -- and then make it happen
- Help our skilled support team triage bugs
- Provide support to the operations staff in troubleshooting and solving production issues
- Build tools that help us make better decisions
Here are things that we consider critical to being a Data Engineer:
- You have a curiosity about how things work.
- You’ve built large-scale data pipelines professionally and can craft clean and beautiful code in Java, Scala, and/or SQL.
- You have hands-on experience running, upgrading, and debugging mission-critical streaming pipelines using Kafka and have strong opinions about the lambda and kappa architectures.
- You’ve built batch data pipelines with Hadoop/Spark as well as with relational database engines, and you understand their respective strengths and weaknesses.
- You can jump into situations with few guardrails and make things better.
- You possess strong computer science fundamentals: data structures, algorithms, programming languages, distributed systems, and information retrieval.
- You are a strong communicator. Explaining complex technical concepts to product managers, support, and other engineers is no problem for you.
- You know how to build reliable and safe distributed systems and understand the tradeoffs made when engineering a feature.
- When things break — and they will — you are eager and able to help fix things.
- You are someone that others enjoy working with due to your technical competence and positive attitude.
- Academic background in computer science or mathematics (BSc or MSc)
- Experience with SOLR and Redis
- Experience building simple scripts and web applications using Python, Ruby, or PHP.
- A solid grasp of basic statistics (regression, hypothesis testing)
- Experience in small start-up environments
Back to top