Senior Data Engineer
We are in a data science renaissance.
Companies that embrace data science will lead and those who do not will fall behind.
To help IBM's clients lead, we are building an elite team of data science practitioners to help them learn how to succeed with data science. The team will include data engineers, machine learning engineers , operations research / optimization engineers and data journalists.
The team will engage directly in solving real-world data science problems in a wide array of industries around the globe with IBM clients and internally to IBM. The elite team of data professionals will work with other IBMers and client data science teams to solve problems in banking, insurance, health care, manufacturing, oil & gas and automotive industries, to name a few. We will teach our client data science teams how to execute on key responsibilities.
- Design and implement optimal data pipeline architectures
- Assemble large, complex data sets that meet business requirements.
- Identify, design, and implement internal process improvements: including process automation, optimizing data delivery, etc.
- Design optimal ETL infrastructures from variety of data sources.
- Incorporate governance processes and tools into the data landscape.
- Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
- Work with executive, LOB, design and IT stakeholders on data-related technical issues and infrastructure needs.
- Keep data separated and secure across national boundaries through replication and fail-over techniques
- Guide and mentor clients to become self-sufficient practitioners
Required Technical and Professional Expertise :
- At least 8 years experience with relational SQL and NoSQL databases, including DB2, Oracle, Postgres, Cassandra.
- At least 8 years experience with big data tools: Hadoop, Spark, Kafka, etc.
- At least 5 years experience with data pipeline and workflow management tools: InfoSphere, Informatica, etc.
- At least 5 years experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc.
- At least 5 years experience working on Agile teams.
- At least 3 years experience in team leadership, project management.
- At least 2 years experience working with stream processing, message queuing.
Preferred Technical and Professional Experience:
- Strong analytic skills related to working with unstructured datasets.
- Demonstrated examples of root cause analysis on internal and external data and processes that address business questions and improvements.
- Experience with at least 3 of the following: SQL, Java, Python, Scala or Ruby.
- Training in design thinking.
- Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field.
While working across all these industries, you will also get to travel the World as these engagements will require that the team spend several weeks at client sites working on data science problems with a diverse team.
As a member of the team you will have a T-shaped skill set, having a broad knowledge base in Data Science and Industry Solutions in general, but also in-depth expertise in data engineering.
Required Technical and Professional Expertise
- Please refer to job description.
Preferred Tech and Prof Experience
- Please refer to job description.
IBM is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.
Back to top