Senior Data Engineer
darwinbox
Job Description
This involves understanding the data models, APIs, and protocols of various data sources and developing connectors that can extract, transform, and load data effectively.
• Define data mappings, transformations, and business rules to ensure consistency and accuracy of integrated data.
• Identify opportunities to optimize data integration processes, reduce latency, and enhance data quality.
• Conduct rigorous testing of data connectors and integration processes to identify and resolve any issues or bugs. Ensure data integrity, reliability, and consistency by performing regular data validation and reconciliation tasks.
• Create and maintain technical documentation, including specifications, user guides, and best practices for data integration and connector development.
• Share knowledge and provide training to other team members or stakeholders to promote self-service data integration capabilities.
• Provide technical support and troubleshooting assistance to resolve integration-related issues.
• Adhere to data governance policies, standards, and regulatory requirements while handling sensitive and confidential data. Implement necessary security measures to protect data during integration and transit.