Senior Data Engineer (Python, Databricks)

DESCRIPTION

Currently, we are looking for a Senior Data Engineer with Python and Databricks expertise to make our team even stronger.

Responsibilities

• Develop, monitor, and operate the most used and most critical curated data pipeline - Sales Order Data (incl. Post-order information, e.g. shipment, return, payment). This pipeline is processing hundreds of millions of records to provide high-quality datasets for analytical and machine learning use-cases
• Consul with analysts, data scientists, and product managers to build and continuously improve "Single Source of Truth" KPI for business steering such as the central Profit Contribution measurement (PC II)
• Redevelop old legacy pipelines to new, advanced and standard versions that are easy to maintain and scalable for future demands
• Leverage and improve a tech stack that includes Python, Databricks, Kubernetes, Spark, and Airflow

• Fluency in Python programming language
• Good hands on experience with Databricks
• Expertise in Apache Spark along with Spark streaming
• Good understanding & hands-on experience with CI/CD
• Rich working experience with Github
• Upper-Intermediate level of English, both spoken and written (B2+)
• Team player (easy & respectful communications, shares responsibilities for the team overall success)
• Communication skills
• Organizational skills

• Delta-Lake
• Expertise in SQL
• Fluency working with AWS landscape
• Ability to build Apache Airflow pipelines
• Presto
• Superset
• Starburst
• Oracle & Exasol

• Competitive compensation depending on experience and skills
• Variety of projects within one company
• Being a part of a project following engineering excellence standards
• Individual career path and professional growth opportunities
• Internal events and communities
• Flexible work hours