Infosys Brazil is looking for professionals with a
Data Engineer | Databricks, AWS, PySpark & SQL profile to join our team.
Required Qualifications- Strong experience in Data Engineering and Databricks environments.
- Hands-on experience with AWS, Databricks, Python, PySpark, and SQL.
- Advanced knowledge of Python and PySpark.
- Strong knowledge of SQL and relational and dimensional data modeling.
- Experience with Databricks and Delta Lake, including jobs, workflows, and Medallion Architecture.
- Experience with ETL/ELT processes and data pipeline migration.
- Knowledge of AWS services for Data Engineering, including S3, Glue, EMR, Athena, Lambda, DMS, and Step Functions.
- Experience with Azure Synapse.
- Knowledge of data quality, testing, and reconciliation processes.
- Experience with Git and CI/CD practices.
- Advanced English communication skills.
- Strong communication and interpersonal skills.
Preferred Qualifications- Knowledge of DataStage for pipeline analysis and reverse engineering.
- Experience with Data Warehouse-to-Lakehouse migration initiatives.
- Knowledge of Unity Catalog, including permissions and data lineage.
- Knowledge of Data Mesh, data products, and federated data governance.
- Experience with observability and monitoring of data platforms.
- Knowledge of FinOps concepts applied to Databricks, including DBUs, clusters, and compute policies.
- Knowledge of Delta Sharing and data contract standards such as ODCS.
- Experience in financial services or credit environments.
Roles and Responsibilities- Develop and maintain scalable batch data pipelines using PySpark and Python.
- Build data transformations and support relational and dimensional data modeling.
- Build and operate Databricks and Delta Lake pipelines using Bronze, Silver, and Gold architecture.
- Migrate and modernize legacy ETL pipelines using Spark, CDC, and batch ingestion.
- Support the reverse engineering of Azure Synapse pipelines and SQL Pools.
- Implement automated testing, data quality controls, and data reconciliation processes.
- Apply Git and CI/CD practices to data pipeline development and deployment.
- Collaborate with technical and business teams to deliver reliable and scalable data solutions.
Additional InformationEmployment Type- Full-time, permanent position (CLT).
Work Model Benefits- Transportation allowance.
- Meal voucher.
- Childcare assistance.
- Life insurance.
- Funeral assistance.
- Medical insurance.
- Dental insurance.
Career Opportunities- Opportunity to work on global projects and international environments.
- Exposure to Software Quality Assurance and project governance practices.
- Collaboration with multicultural and cross-functional teams.
- Professional development within a global technology organization.
Language Requirements- Advanced English proficiency required for interaction with international teams and stakeholders.