Senior Data Engineer (Streaming and Big Data)
- San Diego, CA
Innovation Lives Here
You go all in no matter what you do, and so do we. You want to use your powers for good, and that's exactly what we do at Lytx®. Our team is made up of hungry, humble, and capable people who are making a difference every day with technology that helps save lives each year.
Are you looking to take your career in Data Engineering to the next level? Do you enjoy building data pipelines that enable critical business insights? Are you an expert in designing and delivering scalable data streaming solutions? Lytx is looking for a Senior Data Engineer to join our growing team!
You'll know you're right for this job if you enjoy designing, developing, deployment and maintaining Big Data pipelines in the Cloud. In addition, you intimately know the Data Engineering landscape and enjoy advising software developers and mentoring more junior employees.
What You'll Do:
- Participate in the full software development lifecycle (requirements, design, code, unit test, deployment, sustaining)
- Provide senior-level contributions to the development teams responsible for implementing mission critical data applications
- Participate in the research, design, and testing of next generation data engine platforms
- Develop and guide long-term strategy for data pipelines and persistent and NoSQL data storage
- Evaluate and recommend database infrastructure and tools including cloud technologies
- Build integrated and automated data pipelines using the Hadoop Ecosystem (Spark, Kafka, Hive, Yarn, Oozie)
- Develop and manage steaming data pipelines used for training and developing machine learning models
- Process documentation and data flow diagramming
- Mentor and coach junior engineers ,software developers, and data analysts
- Perform other related duties as assigned
What You'll Need:
- Bachelor's degree in Information Technology or relevant experience.
- Minimum 5 years of experience in Data Engineering and/or Software Engineering
- Minimum 5 years experience developing data pipelines with Java, Python or Scala (Real time pipelines preferred)
- Strong knowledge of Apache Hadoop ecosystem such as HDFS, MapReduce, Sqoop, Yarn, Hive, and Oozie
- Significant experience with Spark, Kafka, Nifi, and other streaming data tools
- Experience deploying data pipelines in the cloud (AWS preferred)
- Strong experience with Data Lake technologies and techniques
- Solid understanding of T-SQL and ETL programming including SQL Server Integration Services (SSIS)
- Excellent conceptual, analytical, and problem-solving skills
- Experience in a high growth, fast paced environment
- Demonstrated record of delivering quality technology outcomes
- Strong organizational skills and a keen attention to detail
- Self-motivated, engaged, and accountable
Lytx, Inc. is proud to be an equal opportunity/affirmative action employer and maintains a drug-free workplace. We are committed to attracting, retaining and maximizing the performance of a diverse and inclusive workforce. EOE/M/F/Disabled/Vet.
Back to top