What you'll do:
If you desire to be part of something special, to be part of a winning team, to be part of a fun team - winning is fun. We are looking forward to Data Engineer based in Pune, India. In Eaton, making our work exciting, engaging, and meaningful; ensuring safety, health, and wellness; and being a model of inclusion & diversity are already embedded in who we are - it's in our values, part of our vision, and our clearly defined aspirational goals. This exciting role offers an opportunity to:
Eaton Corporation's Center for Intelligent Power has an opening for a Senior Data Engineer As a Senior Data Engineer, you will be responsible for designing, developing, and maintaining our data infrastructure and systems. You will collaborate with cross-functional teams to understand data requirements, implement data pipelines, and ensure the availability, reliability, and scalability of our data solutions. She/He can program in several languages and understands the end-to-end software development cycle including CI/CD and software release.
Want more jobs like this?
Get Data and Analytics jobs in Pune, India delivered to your inbox every week.
Job responsibilities
Design, develop, and maintain scalable data pipelines and data integration processes to extract, transform, and load (ETL) data from various sources into our data warehouse or data lake.
Collaborate with stakeholders to understand data requirements and translate them into efficient and scalable data engineering solutions.
Optimize data models, database schemas, and data processing algorithms to ensure efficient and high-performance data storage and retrieval.
Implement and maintain data quality and data governance processes, including data cleansing, validation, and metadata management.
Work closely with data scientists, analysts, and business intelligence teams to support their data needs and enable data-driven decision-making.
Develop and implement data security and privacy measures to ensure compliance with regulations and industry best practices.
Monitor and troubleshoot data pipelines, identifying and resolving performance or data quality issues in a timely manner.
Stay up to date with emerging technologies and trends in the data engineering field, evaluating and recommending new tools and frameworks to enhance data processing and analytics capabilities.
* Collaborate with infrastructure and operations teams to ensure the availability, reliability, and scalability of data systems and infrastructure.
Qualifications:
- Bachelor Degree in Computer Science or Software Engineering or Information Technology
Skills:
• Apache Spark, Python
• Azure experience (Data Bricks, Docker, Function App)
• Git
• Working knowledge of Airflow
• Knowledge of Kubernetes and Docker
- Experience in Design Thinking or human-centered methods to identify and creatively solve customer needs, through a holistic understanding of customer's problem area
- Knowledgeable in leveraging multiple data transit protocols and technologies (MQTT, Rest API, JDBC, etc)
- Knowledge of Hadoop and MapReduce/Spark or related frameworks
- Experience on Cloud Development Platforms - Azure & AWS and their associated data storage options
- Experience on CI/CD (Continuous Integration/Delivery) i.e. Jenkins, GIT, Travis-CI
- Virtual build environments (Container, VMs and Microservices) and Container orchestration - Docker Swarm, Kubernetes/Red Hat Openshift.
- Relational & non-relational database systems - SQL, Postgres SQL, NoSQL, MongoDB, CosmosDB, DocumentDB
- Data Warehousing & ETL - Write complex queries that are accessible, secure and perform in optimized manner that outputs to different consumers and systems
- ETL on Big Data Technologies - Hive, Impala
- Progamming Knowledge - Python and associated IDE's (Eclipse, IntelliJ, PyCharm etc.)
- Data pipelining, scripting, reporting
- Experience in Azure Tools - Blob, SQL, Data Lake, Hive, Hadoop, Data Factory , Databricks, Azure Functions
- SW Development life-cycle process & tools
- Agile development methodologies and concepts including handson with Jira, bitbucket and confluence.
- Ability to specify and write code that is accessible, secure, and performs in an optimized manner with an ability to output to different types of consumers and systems
- Proven experience working as a Data Engineer, with at least 6+ years of experience in data engineering, data warehousing, or related fields.
- Strong proficiency in SQL and experience with relational databases like MySQL, PostgreSQL, or similar.
- Hands-on experience with big data technologies such as Apache Hadoop, Spark, Airflow or similar frameworks.
- Expertise in data modeling, data integration, and ETL processes.
- Proficiency in programming languages like Python, Java, or Scala, with experience in building data pipelines and automation scripts.
- Familiarity with cloud-based data platforms and services such as AWS, Azure, or Google Cloud Platform.
- Experience with data visualization tools like Tableau, Power BI, or similar.
- Knowledge of data security and privacy principles
- Strong problem-solving and analytical skills, with the ability to troubleshoot and resolve complex data engineering challenges.
- Excellent communication and collaboration skills, with the ability to work effectively with cross-functional teams and stakeholders.
- Experience with agile development methodologies and version control systems is preferred.