Staff Software Engineer - Python, SQL, Big Data
- Boston, MA
Who we are:
PPD Algorithms and Data Science is a team that partners with Data Science and business teams to help build systems that leverage the latest-and-greatest in machine-learning, while also fulfilling business-specific needs for agility in decision-making. We facilitate and advise on Data Science workflows, while working to make sure business-user needs are met.
The Demand Forecasting STO predicts the future sales for all of Wayfair's catalogs across US, CA, DE, UK and for 18 months into the future. These forecasting products fuel the supply chain and warehouse management systems that aim to increase product availability and fast-delivery options to Wayfair's customers by effectively deploying capital from our 5000+ suppliers.
What you’ll do:
By joining Demand Forecasting Engineering, a team started in January 2019, you will have the opportunity to significantly impact a space that is growing extremely fast. We are building a team to partner with business and Data Science teams to adapt the current forecasting system to be faster at scale, more reliable, and ultimately to produce more accurate forecasts!
- Creating a fully-automated feature-extraction pipeline that pulls features from inputs across pricing, promotions, out-of-stock, sort rank etc. going back to 2013 to fuel our forecasting model
- The automated part of this pipeline currently consists of ~30 tasks in an Airflow DAG
- Tasks range across Hive, SQL Server, Vertica and Python
- Enabling our end-users to rapidly make business adjustments to 2 million+ item-level forecasts for Wayfair US and CA catalogs where our machine-learning outputs do not meet business needs
- Productionizing further features (we have 200 in total) to make them more scalable and automated, integrating them into our existing pipeline
- Parallelizing existing processes, exploring options of K8s clusters, Airflow and Spark
- Rewriting SQL logic into Python, improving speed and testability
- Facilitating changes to the existing pipeline, with new features and/or changes to the current machine-learning model
- Currently we utilize 16 XGBoost models run in parallel. One future avenue is to use a meta-modelling approach, training a neural net to choose between dozens of models. Other options include utilizing GPUs, or running LightGBM on Spark
- New features include getting more future-facing features into the model, right now we have one of these: planned future promotions
What you’ll need:
- SQL development skills
- 7+ years of hands on coding experience preferably in Python
- Ability to lead small/medium sized technical projects with mentorship
- Strong communication skills to navigate a fast-changing environment with multiple stakeholders
- Interest in Data Science and in solving Big Data problems, experience a plus
Wayfair is one of the world’s largest online destinations for the home. Whether you work in our global headquarters in Boston or Berlin, or in our warehouses or offices throughout the world, we’re reinventing the way people shop for their homes. Through our commitment to industry-leading technology and creative problem-solving, we are confident that Wayfair is, and will be, home to the most rewarding work of your career. If you’re looking for rapid growth, constant learning, and dynamic challenges, then you’ll find that amazing career opportunities are knocking.
No matter who you are, Wayfair is a place you can call home. We’re a community of innovators, risk-takers, and trailblazers who celebrate our differences, and know that our unique perspectives make us stronger, smarter, and well-positioned for success. We value and rely on the collective voices of our employees, customers, community, and suppliers to help guide us as we build a better Wayfair – and world – for all. Every voice, every perspective matters. That’s why we’re proud to be an equal opportunity employer. We do not discriminate on the basis of race, color, ethnicity, ancestry, religion, sex, national origin, sexual orientation, age, citizenship status, marital status, disability, gender identity, gender expression, veteran status, or genetic information.
Back to top