AWS Cloud Data Engineer
pal
Job Description
- Design, develop, and maintain robust ELT/ETL pipelines using Fivetran, dbt Core, and Apache Airflow on AWS infrastructure.
- Architect and manage data warehouse solutions on Amazon Redshift, optimizing for performance, cost, and scalability.
- Build and maintain data models in dbt Core, ensuring high data quality, reusability, and documentation standards.
- Orchestrate complex data workflows using Apache Airflow (MWAA), implementing DAGs with best-practice error handling and alerting.
- Manage data ingestion pipelines via Fivetran and custom Python-based connectors, integrating diverse data sources into the warehouse.
- Administer and optimize Amazon S3 data lake structures, including partitioning strategies, lifecycle policies, and access controls.
- Write advanced SQL queries and stored procedures in Redshift and PostgreSQL for data transformation and analysis.
- Develop Python scripts and automation utilities to support pipeline reliability, monitoring, and operational efficiency.
- Leverage Claude AI capabilities to accelerate development, automate documentation, and enhance data operations.
- Maintain version-controlled codebases using Git, following branching strategies, PR reviews, and CI/CD best practices.
- Collaborate with data analysts, engineers, and business stakeholders to translate requirements into scalable data solutions.
- Monitor data pipeline health, troubleshoot failures, and implement proactive alerting and observability solutions.
Qualifications:
- Bachelor's or Master's degree in Computer Science, Information Systems, Data Engineering, or a related technical field.
- 4+ years of professional experience in data engineering, analytics engineering, or a similar role.
- Demonstrated track record of building and maintaining production-grade data pipelines at scale.
- Experience working in Agile/Scrum environments with collaborative cross-functional teams.
- Strong communication skills — ability to explain complex technical concepts to non-technical stakeholders.
- AWS certifications (e.g., AWS Certified Data Analytics – Specialty) are a strong plus.
- dbt Certification is a plus.