Senior Data Engineer

Architecture design and implementation of next generation data pipelines, analytics, and BI solutions

- Manage AWS resources including RDS, EMR, MWAA, and Lambda

- Build and deliver high quality data architecture and pipelines to support business analysts and data scientists

- Interface with other technology teams to extract, transform, and load data from a wide variety of data sources

- Continually improve ongoing reporting and analysis processes, automating or simplifying self-service support for customers

A day in the life

- Collaborate with Software Engineers, Product Managers, Data Scientists and Business Intelligence Engineers to design, plan and deliver on high priority data initiatives serving internal stakeholders and Pole Star customers.

- Build automated, fault tolerant and scalable data solutions leveraging state of the art technologies including but not limited to Spark, EMR, Python, AirFlow, Glue and S3.

- Look around corners and be creative - Continuously evaluate and improve our strategy, architecture, tooling and codebase to maximize performance, scalability and availability.

BASIC QUALIFICATIONS

- 5+ years of data engineering experience

- Experience with data modeling, warehousing and building ETL pipelines

- Experience programming with at least one modern language, preferably Python

- Experience working on and delivering end to end projects independently

- Knowledge of distributed systems as it pertains to data storage and computing

- 3+ years of analyzing and interpreting data with Postgres, NoSQL etc. experience

- 3+ Years of hands-on experience with big data framework and storage mechanisms like Apache Spark, Iceberg, EMR, Glue, Data Lake, and BI tools like Quicksight

- Infrastructure as Code using Terraform and Github actions

- Geospatial and time series data experience

- Monitoring platforms, especially AWS CloudWatch