Shell Scripting/Python Developer

Job Description: ETL, python and shell scripting
Key Responsibilities:

• Design and develop robust and scalable ETL processes to move and transform large volumes of data from various sources.
  
• Build and maintain data pipelines on Google Cloud Platform (GCP) using services such as Dataflow, Cloud Functions, BigQuery, and Cloud Storage.
  
• Automate workflows and processes using Shell Scripting and Python.
  
• Write and optimize complex SQL queries for data manipulation and analysis in relational (e.g., PostgreSQL, MySQL) and NoSQL databases.
  
• Monitor, debug, and troubleshoot data pipelines to ensure their proper functioning and performance.
  
• Document implemented processes, architectures, and data flows.
  

• Proven experience in ETL development and/or data engineering.
  
• Knowledge of Google Cloud Platform (GCP), especially with data services like BigQuery and Dataflow.
  
• Proficiency in Python for data manipulation and script development (knowledge of libraries like Pandas, Apache Beam is a plus).
  
• Strong skills in Shell Scripting for task automation in Linux/Unix environments.
  
• Extensive experience working with SQL (e.g., PostgreSQL, SQL Server, MySQL) and NoSQL databases. Ability to write complex and optimized queries.
  
• Solid understanding of data modeling and data warehousing principles.
  
• Excellent problem-solving skills and attention to detail.
  

• Google Cloud certification (e.g., Professional Data Engineer).