Senior Data Engineer

San Francisco Office

 

Summary

BigCommerce named a “Best and Brightest” place to work in San Francisco is looking for a Senior Data Engineer. This is an exciting opportunity to build a real time big data pipeline and SaaS analytics platform using latest cloud technologies for the #1 e-commerce SaaS platform. Our perfect candidate will have a background in Computer Science or a related technical field and possess extensive experience in building distributed scalable data pipelines and working with large data stores. You will be working with cross-functional teams in highly visible role. You are scrappy, focused on results, a self-starter and have demonstrated success in using engineering and analytics to drive the understanding, growth and success of a product.

 

What you’ll do:

  • Build scalable and reliable near real time data pipeline on cloud (AWS and GCP) that collects, transforms, loads and curates data from various internal and external data sources
  • Build a scalable distributed data store that will be central source of truth
  • Own data quality for the pipelines you build and make them auditable
  • Build self service tools that helps our data consumers to extract, analyze and visualize data faster
  • Evaluate new technologies and build prototypes for continuous improvements in Data Engineering
  • Partner with Infrastructure and Engineering teams to ensure instrumentation, logging and monitoring is in place

Who you are:

  • Extensive experience in using big data technologies such as Kafka, Spark, HBase or their equivalents
  • Experience with AWS or GCP cloud data platform
  • 3+ years of experience with Java/Scala and Python
  • 2+ years of experience in building and monitoring near real time scalable ETL pipelines
  • Experience with shell scripting
  • Excellent written and verbal communication skills
  • BS or MS in Computer Science or related technical field

 

Nice to have:

  • Experience with Redshift
  • Experience with Google BigQuery, BigTable, Cloud Dataflow and PubSub will be plus
  • Experience with Go and PHP will be a plus
  • Experience with tools such as AirFlow for scheduling and monitoring of data workflow will be a plus
  • Experience working with a team distributed over time zones

 

BigCommerce is a robust, flexible commerce platform that provides established and emerging brands with everything they need to launch, promote, manage and scale a successful online store.  BigCommerce merchants earn more than retailers on competitive commerce solutions thanks to advanced features, better site performance and streamlined business processes. Founded in 2009, the company has received more than $100 million in funding from SoftBank Capital, American Express, Telstra Ventures, General Catalyst, Revolution Growth and FLOODGATE. BigCommerce supports thousands of retailers around the world from its offices in San Francisco, Austin and Sydney. We’re a team of smart, passionate and talented people on a mission to democratize commerce. If you’re looking to work with industry leaders, we want to hear from you.

Share job posting