Senior Data Engineer
persistent
Job Description
- Design, develop, and optimize complex SQL queries, stored procedures, CTEs, window functions, and database objects.
- Perform query tuning, execution plan analysis, indexing, and partitioning to improve database performance and scalability.
- Build and maintain scalable data pipelines for data ingestion, transformation, and analytics workloads.
- Design and implement modern data architectures covering ingestion, storage, processing, and compute layers.
- Work with cloud-based data warehouses including Snowflake, BigQuery, Redshift, and Azure SQL.
- Develop and manage cloud-native data solutions across AWS, Azure, and GCP platforms.
- Integrate data from multiple structured, semi-structured, and unstructured data sources.
- Develop and maintain ETL/ELT workflows using tools such as Airflow, Azure Data Factory (ADF), AWS Glue, Databricks, or similar platforms.
- Automate data processes to improve reliability, scalability, operational efficiency, and reduce manual intervention.
- Ensure data quality, governance, security, compliance, and data lineage standards are maintained across the data ecosystem.
- Collaborate with Data Analysts, Data Scientists, Product Managers, and Business Teams to understand business requirements.
- Translate business requirements into scalable and maintainable data engineering solutions.
- Monitor, troubleshoot, and optimize data pipelines, workflows, database performance, and platform health.
- Participate in architecture reviews and contribute to data platform modernization initiatives.
- Mentor junior data engineers and promote engineering best practices, coding standards, and technical excellence.
- Create technical documentation, architecture diagrams, operational runbooks, and deployment guides.