Technology Lead

Infosys is seeking an experienced Databricks Administrator with strong expertise in managing, securing, and optimizing Databricks environments on cloud platform: AWS. The ideal candidate should have hands-on experience in cloud database administration, cluster management, workspace governance, performance optimization, CI/CD integrations, and platform reliability.
The role supports our client's enterprise data modernization initiatives by ensuring a robust, scalable, and secure Databricks platform for analytics and engineering teams.

Required Qualifications:

• Candidates authorized to work for any employer in Canada without employer-based visa sponsorship are welcome to apply. Infosys is unable to provide immigration sponsorship for this role at this time.
• Candidate must be located within commuting distance of Mississauga, Ontario or be willing to relocate to the areas.
• Bachelor's degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education.
• At least 4 years of Information Technology experience
• At least 6 years of experience in cloud-based database and platform administration (AWS/Azure).
• At least 3 years hands-on experience administering Databricks environments.

Preferred Qualifications:

• Strong understanding of autoscaling, job orchestration, and pipeline execution.
• Deep knowledge of cloud storage, networking basics, IAM, and security best practices.
• Experience with Delta Lake, Unity Catalog, and workspace governance.
• Proficiency in scripting (Python, Shell, PowerShell) for automation.
• Experience with monitoring tools and log analytics (CloudWatch, Azure Monitor, Databricks Logs).
• Familiarity with DevOps and CI/CD (GitHub Actions, Azure DevOps, Jenkins).
• Databricks certifications (Admin, Associate Data Engineer, or Architect).
• Understanding of data engineering, ETL/ELT workflows, and big data concepts.

Key Responsibilities:

Databricks Platform & Workspace Administration

• Manage and administer Databricks workspaces, clusters, pools, jobs, and user access controls.
• Implement and enforce cluster policies, workspace best practices, and governance standards.
• Oversee Libraries, Repos, and Job scheduling within Databricks.
• Monitor and optimize runtime versions, cluster configurations, and performance.

Operational Support

• Provide L2/L3 support for platform issues, job failures, cluster outages, and scheduled maintenance.
• Collaborate with Data Engineering, Security, Cloud Infra, and Governance teams.
• Support upgrades, migrations, patching, and operational improvements.

DevOps, Automation & CI/CD

• Automate deployment processes using Git-based workflows (Repos/GitHub).
• Develop and maintain scripts for workspace automation.
• Manage CI/CD pipelines for notebooks, jobs, and Delta pipelines.

Performance Monitoring & Optimization

• Troubleshoot cluster performance issues, job failures, and resource utilization.
• Optimize Spark configurations, autoscaling parameters, and workload management.
• Monitor audit logs, job runs, compute usage, and cost trends.

Security, Compliance & Access Management

• Implement role-based access control (RBAC) and workspace governance aligned with corporate security standards.
• Manage secrets, tokens, service principals, and integration with Azure AD / AWS IAM.
• Ensure compliance with data protection, logging, audit requirements, and enterprise governance.

Cloud & Database Administration

• Administer cloud-based databases (AWS/Azure), including provisioning, performance tuning, maintenance, and monitoring.
• Manage S3 / ADLS storage integrations, mount points, permissions, and lifecycle policies.
• Support ingestion frameworks and ETL/ELT workflows using Spark, Delta Lake, and Databricks Asset Bundles (optional).

The job entails sitting as well as working at a computer for extended periods of time. Should be able to communicate by telephone, email or face to face. Travel may be required as per the job requirements

Estimated annual compensation range for the candidate based in the below location will be:
Ontario: $ 92740 to $ 123375