Data Engineer

BS in Computer Science or related field required
● 5+ years of experience in Data Engineering
● Strong knowledge of SQL

● Strong knowledge of either Scala or Python
● Strong computer science fundamentals
● Experience with Airflow and/or Composer
● Experience with GCS and BigQuery
● Strong database fundamentals, such as
○ ACID
○ snowflake schema
○ normalized/denormalized data
○ serialization formats
○ OLTP vs OLAP
○ relational databases vs key-value stores vs document stores
● Strong Data Engineering fundamentals, such as
○ ETL vs ELT
○ Data streaming
○ Data Warehouses

Nice to haves:
● Strong Knowledge with with Spark (preferably Scala)
● Experience debugging and optimizing Spark jobs
● Experience in the advertising industry

Job responsibilities

● Build and modify data pipelines in BigQuery
● Build and modify Spark jobs (in Scala) to perform various tasks
● Schedule data pipelines using Airflow and Composer
● Help automate existing BQ pipelines using Composer
● Help move some Spark pipelines to BQ
● Develop and launch new features to adapt to evolving business needs
● Work in an agile environment
● Participate in our on call rotation for handling production issues outside of business
hours
● Be an active and engaged owner of our data infrastructure
● Be curious and seek to understand all aspects of our business
● Maintain high standards of code quality, and encourage the same by
providing constructive code reviews to collaborators
● Troubleshoot and resolve issues, problems, and errors encountered across various
systems
● Collaborate with Data Science, Product, Research, and Engineering teams to iterate on
the roadmap
● Gather requirements when underspecified