Data Engineering

Data Engineering & ETL/ELT Development
Develop, maintain, and support ETL/ELT pipelines for enterprise data processing.

Develop batch and near real-time data ingestion processes.

Build data transformations and workflows based on technical specifications.

Develop reusable and efficient data-processing components.

Implement error handling, logging, and recovery mechanisms.

Support data models used for analytics and reporting.

Snowflake / Databricks Development & Support
Develop tables, views, schemas, and data models.

Write and optimize complex SQL queries.

Develop and maintain data-loading and unloading processes.

Develop and support stored procedures and scheduled tasks.

Troubleshoot failed jobs and data-processing issues.

Implement role-based access and permissions as defined by technical design.

Monitor and optimize data-processing jobs for performance.

AWS Development & Support
Develop and support AWS Glue jobs for data ingestion and transformation.

Work with Amazon S3 for file-based data ingestion and storage.

Implement and troubleshoot S3-based ingestion processes.

Work with IAM roles and permissions in accordance with established security guidelines.

Monitor AWS jobs and troubleshoot failures using CloudWatch or equivalent monitoring tools.

Develop file-processing and exception-handling mechanisms.

Oracle / ODI / PL/SQL
Develop and support Oracle Data Integrator (ODI) mappings, packages, and interfaces.

Develop and troubleshoot PL/SQL procedures, functions, and scripts.

Support Oracle database-related development and operational activities.

Troubleshoot data integration issues between Oracle and cloud data platforms.

Assist with Oracle database administration activities where required.

Data Quality & Reconciliation
Execute data quality checks and reconciliation between source and target systems.

Investigate data discrepancies and identify the source of data issues.

Perform completeness, accuracy, and consistency validations.

Fix data-processing defects and implement appropriate preventive controls.

Document data-quality issues and their resolution.

Production Support
Monitor data pipelines and scheduled jobs.

Investigate and resolve production incidents within agreed SLAs.

Perform technical troubleshooting and initial root cause analysis.

Analyze application logs, job logs, and database errors.

Support recurring incident analysis and implement permanent fixes.

Escalate complex technical issues to the Technical Lead with appropriate analysis and supporting evidence.

Performance Optimization
Identify inefficient SQL queries and data-processing jobs.

Perform SQL tuning and optimization.

Analyze job execution times and resource utilization.

Implement improvements to reduce processing time and operational failures.

Support performance testing and benchmarking activities.

Testing & Deployment
Develop unit and integration test cases for data pipelines.

Validate data transformations and reconciliation results.

Fix defects identified during testing and production.

Prepare deployment scripts and technical implementation documentation.

Support release and deployment activities across Development, Test, and Production environments.

Participate in post-deployment validation.