EPAM Systems

Lead Data Software Engineer

3+ months agoMalaga, Spain

Striving for excellence is in our DNA. Since 1993, we have been helping the world's leading companies imagine, design, engineer, and deliver software and digital experiences that change the world. We are more than just specialists, we are experts.

EPAM is committed to providing our global team of more than 41,150 EPAMers with inspiring careers from day one. EPAMers think creatively and lead with passion and honesty. Our people are the source of our success. We value collaboration, work in partnership with our customers, and strive for the highest standards of excellence. In today's market conditions, we're supporting operations for hundreds of clients around the world remotely. No matter where you are located, you'll join a dedicated, diverse community that will help you discover your fullest potential.

We are extending the team with Senior/Lead Big Data engineers. The role supposes participating in full cycle of big data solutions engineering, including project scope definition and estimating (including work with the stakeholders); architecture design, modelling complex relationships in heterogeneous data environments; technical decision making; functionality implementation; refactoring and optimization; providing technical leadership for the teammates, mentoring; and also participation in knowledge sharing and best practices elaboration within our Big Data competency center.
Required Skills

  • Data
What You'll Do
  • Lead, design, and implement innovative analytical solutions using Cloud Native, Big Data, and NoSQL related technologies
  • Design and implement Cloud/On-Premise/Hybrid solutions using best in the class data frameworks
  • Work with product and engineering teams to understand requirements, evaluate new features and architecture to help drive decisions
  • Build collaborative partnerships with architects and key individuals within other functional groups
  • Perform detailed analysis of business problems and technical environments and use this in designing quality technical solution
  • Actively participate in code review and test solutions to ensure it meets best practice specifications
  • Build and foster a high-performance engineering culture, mentor team members, and provide the team with the tools and motivation
  • Write project documentation
  • Coding experience with one of the following programming languages: Python/Java/Scala/Kotlin
  • Experience with Linux OS: configure services and write basic shell scripts, understanding of network fundamentals
  • Good knowledge of SQL and relational algebra
  • Advanced experience in software development with Data technologies (e.g. administration, configuration management, monitoring, debugging and performance tuning)
  • Engineering experience and practice in Data Management, Data Storage, Data Visualization, Disaster Recovery, Integration, Operation, Security
  • Strong experience building data ingestion pipelines (simulating Extract, Transform, Load workload), Data Warehouse or Database architecture
  • Strong experience with data modeling; hands-on development experience with modern Big Data components
  • Cloud: experience in designing, automation, provisioning, deploying and administering scalable, available and fault tolerant systems
  • Good understanding of CI/CD principles and best practices
  • Analytical approach to problem-solving with an ability to work at an abstract level and gain consensus; excellent interpersonal, leadership and communication skills
  • Data-oriented personality and possessing compliance awareness, such as PI, GDPR, HIPAA
  • Motivated, independent, efficient and able to handle several projects; work under pressure with a solid sense for setting priorities
  • Ability to work in a fast-paced (startup like) agile development environment
  • Strong experience in high load and IoT Data Platform architectures and infrastructures
  • Vast experience with Containers and Resource Management systems: Docker and Kubernetes
  • Experience in direct customer communications
  • Experience in technology/team leading of data oriented projects
  • Solid skills in infrastructure troubleshooting
  • Practical experience in performance tuning, optimization, and problem analysis
  • Experienced in different business domains
  • Advanced understanding of distributed computing principles
  • English proficiency
  • Programming Languages: Python/Java/Scala/Kotlin and SQL
  • Cloud-Native stack: Databricks, Azure DataFactory, AWS Glue, AWS EMR, Athena, GCP DataProc, GCP DataFlow
  • Big Data stack: Spark Core, Spark SQL, Spark ML, Kafka, Kafka Connect, Airflow, Nifi, Streamset
  • NoSQL: CosmosDB, DynamoDB, Cassandra, HBase; MongoDB
  • Queues and Stream processing: Kafka Streams; Flink; Spark Streaming
  • Data Visualization: Tableau, PowerBI, Looker
  • Operation: Cluster operation, Cluster planning
  • Search: Elasticsearch/ELK
  • Solid Cloud experience with 2 or more leading cloud providers (AWS/Azure/GCP): Storage; Compute; Networking; Identity and Security; NoSQL; RDBMS and Cubes; Big Data Processing; Queues and Stream Processing; Serverless; Data Analysis and Visualization; ML as a service (SageMaker; Tensorflow)
  • Enterprise Design Patterns (Secure Inversion of Control etc)
  • Development Methods (TDD, BDD, DDD)
  • Version Control Systems (Git)
  • Testing: Component/ Integration Testing, Unit testing (JUnit)
  • Deep understanding of SQL queries, joins, stored procedures, relational schemas, and SQL optimization
  • Experience in various messaging systems, such as Kafka, RabbitMQ, Event Hub, Pub/Sub
  • Rest, Thrift, GRPC
  • Build Systems: Maven, SBT, Ant, Gradle
  • Docker, Kubernetes
We offer
  • Extended opportunity to grow professionally in a cross-cultural environment
  • Access to various on-line courses from leading provider
  • Access to engineering communities on a global scale
  • Unlimited access to LinkedIn learning solutions
  • Social benefits in line with local legislation
  • Health insurance and meal vouchers programs
  • Special discount program for EPAMers with providers across Malaga and in other cities around the world
  • Regular team collaboration events
  • Office in a good location with easy access
  • Referral bonuses
  • Relocation support (for people from other countries)
  • #DA_SP

Client-provided location(s): Málaga, Spain
Job ID: EPAM-53730