Senior Data Engineer

3+ months agoSan Francisco, CA


About Us:
Launched in 2011, Twitch is a global community that comes together each day to create multiplayer entertainment: unique, live, unpredictable experiences created by the interactions of millions. We bring the joy of co-op to everything, from casual gaming to world-class esports to anime marathons, music, and art streams. Twitch also hosts TwitchCon, where we bring everyone together to celebrate, learn, and grow their personal interests and passions. We're always live at Twitch. Stay up to date on all things Twitch on Linkedin, Twitter and on our Blog.

About the Role:
Twitch is searching for a Senior Data Engineer with a motivated mindset and a focus on exceeding expectations. You should understand a want to build a culture of customer-first partnerships and have the desire to evolve the value of Twitch IT's engineering organization. Working with business service teams and others in Corporate Engineering, you will directly affect important company-wide functions. You will help build data warehouses, data pipelines, and promoting data and analytics use across many business services from Marketing to Finance, from Sales to Customer Experience. You will report to our Senior Director of Corporate Engineering.

• Medical, Dental, Vision & Disability Insurance
• 401(k), Maternity & Parental Leave
• Flexible PTO
• Commuter Benefits
• Amazon Employee Discount
• Monthly Contribution & Discounts for Wellness Related Activities & Programs (e.g., gym memberships, off-site massages, etc.),
• Breakfast, Lunch & Dinner Served Daily
• Free Snacks & Beverages
Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.
We are an equal opportunity employer and value diversity at Twitch. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.


You Will:
• Build data pipelines: Manage data pipelines consist of a series of stages through which data flows. Architecting and creating data pipelines will be the primary responsibility of the data engineer.
• Drive Automation through metadata management: You will use modern tools, techniques and architectures to partially or completely automate the most-common, repeatable data preparation to minimize manual and error-prone processes.
• Collaborate across departments: You will need strong partnership skills to work with varied partners. In particular, the data engineer will work in close relationship with data science teams and with business (data) analysts in refining their data requirements.
• Participate in ensuring compliance with data classification and handling guidelines: Working with information stewards in vetting and promoting content created to ensure all data complies with Twitch's data classification and handling guidelines.


You Have:
• At least five years of work experience in data management disciplines including data architecture, data integration, modeling, optimization, and data quality or other areas directly relevant to data engineering tasks.
• At least two years of experience working in teams with collaborators to support a departmental or multi-departmental data management and analytics initiative.
• Experience with advanced analytics tools for Object-oriented/object function scripting using languages such as Go or Python hosted on AWS EC2 or Lambda
• Can design and manage data pipelines for data structures encompassing data transformation, data models, schemas, metadata, and workload management.
• Experience working with and optimizing existing ETL processes and data integration and data preparation flows and moving them in production.
• Experience working with a spectrum of NoSQL, RDBMS and data stores like AWS Redshift, Athena, PostgreSQL and DynamoDB

Job ID: Amazon-1370359