Sr Data Engineer
source-right
Job Description
- Develop, fine-tune, automate, and maintain ETL pipelines using Azure Data Factory, Azure Databricks, Python (PySpark), Azure SQL Database, and SQL Server Stored Procedures.
- Develop ETL solutions using data integration tools for interfacing between source application and the Enterprise Data Warehouse
- Document ETL data mappings, data dictionaries, processes, programs, and solutions as per established standards for data governance
- Integrate ETL development with existing projects to maximize object reuse
- Create, execute, and document unit test plans for ETL and data integration processes and programs
- Work with the peers in data team to assess and troubleshoot potential data quality issues at key intake points such as validating control totals at intake.
- Troubleshoot data issues and defects to determine root cause
- Performance tuning of the ETL process and SQL queries, and recommend and implement ETL and query tuning techniques
- Develop and create transformation queries, views, and stored procedures for ETL processes, and process automations