Technology Lead
Today Sao Paulo, Brazil
About us
https://www.infosys.com
- Size: 10,000+ employees
- Industry: Consulting
Infosys Brazil is looking for professionals with a Senior Databricks Data Engineer | AWS, PySpark & SQL profile to join our team.
Required Qualifications
Preferred Qualifications
Roles and Responsibilities
Additional Information
Employment Type
Work Model
Benefits
Career Opportunities
Language Requirements
Required Qualifications
- Strong experience in Data Engineering and Databricks environments.
- Advanced proficiency in Python, PySpark, and SQL.
- Experience with Databricks and Delta Lake, including jobs, workflows, Medallion Architecture, Delta Live Tables/Lakeflow, and Asset Bundles.
- Strong knowledge of relational and dimensional data modeling.
- Experience developing ETL/ELT, CDC, and batch data pipelines.
- Hands-on experience with AWS services such as Amazon S3, AWS Glue, Amazon EMR, Amazon Athena, AWS Lambda, AWS DMS, and AWS Step Functions.
- Experience with Azure Synapse, including SQL Pools and data pipelines.
- Knowledge of Git and CI/CD practices applied to Data Engineering.
- Experience with data quality, testing, reconciliation, and automated validation processes.
- Ability to work independently and collaborate effectively with technical and business stakeholders.
- Strong communication and interpersonal skills.
Preferred Qualifications
- Experience with Data Warehouse-to-Lakehouse migration programs.
- Knowledge of Unity Catalog, including permissions, data lineage, governance, and data contracts.
- Experience with DataStage, particularly analyzing existing jobs for reverse engineering.
- Knowledge of automated testing, Data Quality gates, and data validation between legacy and modern environments.
- Experience translating business rules from legacy ETL tools into Spark-based solutions.
Roles and Responsibilities
- Design, develop, test, and maintain scalable batch data pipelines using PySpark and Python.
- Build and operate Databricks and Delta Lake solutions based on Bronze, Silver, and Gold architecture layers.
- Develop data transformations and support relational and dimensional data modeling.
- Migrate and rebuild legacy ETL pipelines using Spark, including CDC and batch ingestion from relational databases.
- Support the reverse engineering of Azure Synapse pipelines and SQL Pools.
- Implement automated testing, data quality controls, reconciliation processes, and validation between legacy and new environments.
- Apply Git and CI/CD practices to data pipeline development and deployment.
- Work with AWS data services to build reliable and scalable data solutions.
- Collaborate with technical teams and business stakeholders to ensure successful solution delivery.
Additional Information
Employment Type
- Full-time, permanent position (CLT).
Work Model
- Hybrid.
Benefits
- Transportation allowance.
- Meal voucher.
- Childcare assistance.
- Life insurance.
- Funeral assistance.
- Medical insurance.
- Dental insurance.
Career Opportunities
- Opportunity to work on global projects and international environments.
- Exposure to Software Quality Assurance and project governance practices.
- Collaboration with multicultural and cross-functional teams.
- Professional development within a global technology organization.
Language Requirements
- Advanced English proficiency required for interaction with international teams and stakeholders.
Additional Information
- Posted: today
Perks and Benefits
Health and Wellness
- Health Insurance
- Life Insurance
- HSA
- Short-Term Disability
Parental Benefits
- Birth Parent or Maternity Leave
- Non-Birth Parent or Paternity Leave
- On-site/Nearby childcare
Vacation and Time Off
- Paid Vacation
- Paid Holidays
- Personal/Sick Days
- Sabbatical
Financial and Retirement
- 401(K)
- Relocation Assistance
Office Life and Perks
- Commuter Benefits Program
Professional Development
- Learning and Development Stipend
Diversity And Inclusion
- Employee Resource Groups (ERG)
