Senior Data Engineer

• Collaborate with engineers, product managers, and data scientists to develop end-to-end data pipelines, solutions, and foundational data sets
• Design and own the data architecture for large-scale projects, making informed decisions about design and operational trade-offs
• Build, launch, and optimize sophisticated data pipelines, data models, and visualizations that support diverse use cases across multiple products or domains
• Define and manage Service Level Agreements (SLAs) for all data sets within areas of ownership
• Implement data security models based on privacy requirements, ensure compliance with safeguards, and evolve data governance processes
• Solve complex data integration challenges using optimal ETL/ELT patterns, frameworks, and query techniques for both structured and unstructured data sources
• Optimize pipelines, dashboards, frameworks, and systems to streamline data artifact development
• Mentor team members, providing and receiving actionable feedback to foster skill growth and collaboration

Must Have Criteria

• 5+ years of experience in data engineering, data warehousing, and ETL/ELT, working with large data sets in the cloud
• Strong proficiency in SQL and data modeling, with hands-on experience in Snowflake and dbt
• 5+ years of Python development experience, including building scalable Big Data solutions and ETL ecosystems
• Hands-on experience with RDBMS — MySQL, PostgreSQL, and MS SQL Server
• Experience with Apache Airflow for pipeline orchestration and working knowledge of CI/CD pipelines
• Experience with Spark and PySpark for large-scale data processing
• Strong proficiency in Terraform for infrastructure-as-code

Nice to Have

• Master's degree in Computer Science, Computer Engineering, or a related technical field
• Experience with Automic or similar enterprise job scheduling/integration tools
• Prior experience in a high-growth startup or scaleup environment handling petabyte-scale data
• Familiarity with AWS data services (S3, Glue, Redshift, Lambda) in production environments
  • Experience with Datadog for pipeline observability and monitoring