Data Engineer
icims
Job Description
- Design, develop, operationalize robust and scalable data pipelines with automated data quality checks to support business needs and demonstrate ownership of live data pipelines.
- Proficiency understanding and implementing data life cycles, data lineage, custom metadata, and data governance.
- Proficiency implementing BigQuery SQL procedures, functions, and similar, with actionable logging
- Optimize ETL processes and data workflows to ensure abiding by performance constraints, for efficiency and scalability.
- Comfortably lead designing and building of development to production data pipelines from data ingestion to consumption using GCP services, Python, BigQuery, DBT, SQL, Apache Airflow, Celigo.
- Expertise in JSON, XML, text parsing using Dataflow, Python, or batch jobs.
- Advocates and practices software best practices in leveraging reusable components in implementation.
- Design, develop and maintain robust, and scalable data models and schemas to support analytics and reporting requirements.
- Optimize data processing performance, ensure high availability, scalability of data systems and solutions.
- Implement monitoring and alerting mechanisms to proactively identify and resolve issues.
- Ensure data quality and consistency through rigorous testing and validation processes in development and production tiers.
- Troubleshoot and resolve data-related issues promptly.
- Create, update, and maintain technical documentation of the data processes, pipelines, and models.
- Stay updated with industry trends and technologies to continuously improve our data engineering practices.