Senior Data Engineer

San Francisco Office

Senior Data Engineer

San Francisco CA


BigCommerce named a “Best and Brightest” place to work in San Francisco is looking for a Senior Data Engineer. This is an exciting opportunity to build a real time big data pipeline and SaaS analytics platform using latest cloud technologies for the #1 e-commerce SaaS platform. Our perfect candidate will have a background in Computer Science or a related technical field and possess extensive experience in building distributed scalable data pipelines and working with large data stores. You will be working with cross-functional teams in highly visible role. You are scrappy, focused on results, a self-starter and have demonstrated success in using engineering and analytics to drive the understanding, growth and success of a product.

What you’ll do:

  • Build scalable and reliable near real time data pipeline on cloud (AWS and GCP) that collects, transforms, loads and curates data from various internal and external data sources
  • Build a scalable distributed data store that will be central source of truth
  • Own data quality for the pipelines you build and make them auditable
  • Build self service tools that helps our data consumers to extract, analyze and visualize data faster
  • Evaluate new technologies and build prototypes for continuous improvements in Data Engineering
  • Partner with Infrastructure and Engineering teams to ensure instrumentation, logging and monitoring is in place
  • Contribute to Open Source community

Who you are:

  • Extensive experience in using big data technologies such as Spark, Kafka, Hadoop, HBase and Hive or their equivalents
  • Experience with AWS and/or GCP
  • 5+ years of experience with Java, Scala and Python
  • 5+ years of experience with SQL (MySQL, Redshift, etc)
  • 3+ years of experience in building and monitoring near real time scalable ETL pipelines
  • Experience with shell scripting
  • Excellent written and verbal communication skills
  • BS or MS in Computer Science or related technical field

Nice to have:

  • Experience with Go and PHP will be a plus
  • Experience with tools such as AirFlow for scheduling and monitoring of data workflow will be a plus
  • Experience with managed data processing services of cloud platforms such as GCP and/or AWS will be a plus
  • Experience working with a team distributed over time zones would be a plus

Inside BigCommerce

BigCommerce is a robust, flexible commerce platform that provides established and emerging brands with everything they need to launch, promote, manage and scale a successful online store.BigCommerce merchants earn more than retailers on competitive commerce solutions thanks to advanced features, better site performance and streamlined business processes. Founded in 2009, the company has received more than $100 million in funding from SoftBank Capital, American Express, Telstra Ventures, General Catalyst, Revolution Growth and FLOODGATE. BigCommerce supports thousands of retailers around the world from its offices in San Francisco, Austin and Sydney.

We’re a team of smart, passionate and talented people on a mission to democratize commerce. If you’re looking to work with industry leaders and love what you do, we want to hear from you!

 Perks & Benefits

  • An amazing company culture that doesn’t just talk values, but lives them
  • Open vacation policy for salaried team members and generous accrual vacation policy for hourly team members, + paid sick time
  • Competitive compensation packages and meaningful stock grants for every employee
  • Comprehensive health insurance coverage that starts on day one
  • Fully stocked kitchens and craft beers (don’t worry, we also subsidize gym memberships)
  • Three especially cool offices in Austin, San Francisco and Sydney
  • A free online store to help you live out your entrepreneurial dreams
  • Employee-driven team building activities like paintball, rock climbing, social sports leagues, hackathons, movie nights, river floats, etc.
  • Time off for volunteering and employee-driven charity events

Share job posting