Staff SW Engineer

Overview
Come join the CTG Big Data Analytics Engineering team at IDC as a "Senior Software Engineer - Data Platform". We are leveraging big data technologies to gain new insights into our TurboTax customer experience. Some of the technologies leveraged by our team include: Hadoop, Vertica, and Netezza. We foster an open team environment where we value direct interactions and working code above working in a cave.
Responsibilities

  • Good understanding of distributed data computing concepts, Ingestion technologies, persistent storage, No SQL (at least one stack), batch and real-time processing of data (preferably using open source technology like Spark/Storm). Expected to be SME for at least one tech stack.
  • Designing end to end data pipeline given business and ops requirements (ingestion, processing and storage).
  • Fair undersigning of AWS Data Stack. Good understanding of Hadoop/HDFS and Real time processing (micro batch and Streaming)
  • Good understanding of Scripting languages like Python & Shell, SQL (optional), Scala (optional), Java. Understanding of Micro-services architecture will be a big plus.
  • Designing and developing ETL jobs across multiple platforms and tools including Vertica, Netezza and Hadoop.
  • Gathering functional requirements, developing technical specifications, and project & test planning.
  • Work with business users to develop and refine analytical requirements for quantitative data (view-through, clickstream, acquisition, product usage, transactions), qualitative data (survey, market research) and unstructured data (blog, social network).
  • Designing and developing schema definitions and support data warehouse/mart to enable integration of disparate data sources from within Intuit and outside, aggregate it and make it available for analysis.
  • Support large data volumes and accommodate flexible provisioning of new sources.
  • As a key member of the team drive adoption of new technologies, tools, and process improvements to build world class analytical capabilities for web analytics, optimization, experimentation and personalization.
  • Be the technical leader: Mentoring junior engineers, new team members, and applying technical expertise to challenging data and design problems.
  • Resolve defects/bugs during QA testing, pre-production, production, and post-release patches.
  • Work cross-functionally with various Intuit teams: Product Management, Project Management, Data Architects, Data Scientists, Data Analysts, Software Engineers, and other Data Engineers.
  • Contribute to the design and architecture of project across the data landscape.
  • Experience with Agile Development, SCRUM, or Extreme Programming methodologies.
  • Helps to align to overall strategies and reconcile competing priorities across organization.
Qualifications
  • BS/MS in computer science or equivalent work experience.
  • 8 to 12 years' experience in developing DB schemas, creating ETLs, and familiar with MPP/Hadoop systems.
  • Must have mastery of data warehousing technologies including data modeling, ETL and reporting. Ideal candidate to have 5+ years of experience in end-to-end data warehouse implementations and at least 2 projects with 4TB+ data volume.
  • Extensive experience with databases (Vertica, Netezza or oracle).
  • Good knowledge of Operating Systems (Unix or Linux).
  • Good understanding of Data ware House methodologies.
  • Hands on experience in any of the programming languages (Shell scripting, Python, Java, etc).
  • Must have been through several full life cycle Data Warehousing implementations and involved in scalability and performance related design aspects in Database and ETL.
  • Solid communication skills: Demonstrated ability to explain complex technical issues related to technical and non-technical audiences.
  • Demonstrated understanding of the Software design and architecture process.
  • Experience with unit testing and data quality automation checks
  • Should be results oriented, self-motivated, accountable and work under minimal supervision.
  • Excellent written, oral communication and presentation Skills.


Good to have
  • Knowledge of Big Data eco system like Hadoop M/R, Pig and Hive is a strong plus.
  • Experience in design, development and deployment of one or more tools - ETL (Informatica, OWB, ODI), reporting (Business Objects, QlikView, Tableau)



Back to top