Staff SW Engineer
Come join the CTG Big Data Analytics Engineering team at IDC as a "Senior Software Engineer - Data Platform". We are leveraging big data technologies to gain new insights into our TurboTax customer experience. Some of the technologies leveraged by our team include: Hadoop, Vertica, and Netezza. We foster an open team environment where we value direct interactions and working code above working in a cave.
- Good understanding of distributed data computing concepts, Ingestion technologies, persistent storage, No SQL (at least one stack), batch and real-time processing of data (preferably using open source technology like Spark/Storm). Expected to be SME for at least one tech stack.
- Designing end to end data pipeline given business and ops requirements (ingestion, processing and storage).
- Fair undersigning of AWS Data Stack. Good understanding of Hadoop/HDFS and Real time processing (micro batch and Streaming)
- Good understanding of Scripting languages like Python & Shell, SQL (optional), Scala (optional), Java. Understanding of Micro-services architecture will be a big plus.
- Designing and developing ETL jobs across multiple platforms and tools including Vertica, Netezza and Hadoop.
- Gathering functional requirements, developing technical specifications, and project & test planning.
- Work with business users to develop and refine analytical requirements for quantitative data (view-through, clickstream, acquisition, product usage, transactions), qualitative data (survey, market research) and unstructured data (blog, social network).
- Designing and developing schema definitions and support data warehouse/mart to enable integration of disparate data sources from within Intuit and outside, aggregate it and make it available for analysis.
- Support large data volumes and accommodate flexible provisioning of new sources.
- As a key member of the team drive adoption of new technologies, tools, and process improvements to build world class analytical capabilities for web analytics, optimization, experimentation and personalization.
- Be the technical leader: Mentoring junior engineers, new team members, and applying technical expertise to challenging data and design problems.
- Resolve defects/bugs during QA testing, pre-production, production, and post-release patches.
- Work cross-functionally with various Intuit teams: Product Management, Project Management, Data Architects, Data Scientists, Data Analysts, Software Engineers, and other Data Engineers.
- Contribute to the design and architecture of project across the data landscape.
- Experience with Agile Development, SCRUM, or Extreme Programming methodologies.
- Helps to align to overall strategies and reconcile competing priorities across organization.
- BS/MS in computer science or equivalent work experience.
- 8 to 12 years' experience in developing DB schemas, creating ETLs, and familiar with MPP/Hadoop systems.
- Must have mastery of data warehousing technologies including data modeling, ETL and reporting. Ideal candidate to have 5+ years of experience in end-to-end data warehouse implementations and at least 2 projects with 4TB+ data volume.
- Extensive experience with databases (Vertica, Netezza or oracle).
- Good knowledge of Operating Systems (Unix or Linux).
- Good understanding of Data ware House methodologies.
- Hands on experience in any of the programming languages (Shell scripting, Python, Java, etc).
- Must have been through several full life cycle Data Warehousing implementations and involved in scalability and performance related design aspects in Database and ETL.
- Solid communication skills: Demonstrated ability to explain complex technical issues related to technical and non-technical audiences.
- Demonstrated understanding of the Software design and architecture process.
- Experience with unit testing and data quality automation checks
- Should be results oriented, self-motivated, accountable and work under minimal supervision.
- Excellent written, oral communication and presentation Skills.
Good to have
- Knowledge of Big Data eco system like Hadoop M/R, Pig and Hive is a strong plus.
- Experience in design, development and deployment of one or more tools - ETL (Informatica, OWB, ODI), reporting (Business Objects, QlikView, Tableau)
Back to top