Data Engineer - Architect

Key Responsibilities

• Design, build, and maintain scalable, reliable data pipelines for efficient data collection, transformation, and storage, aligned with business needs and industry best practices.
• Ensure data quality and integrity through robust validation and cleaning processes.
• Optimize data storage and retrieval for efficient access and analysis by data scientists and analysts.
• Work with both Batch and Streaming data and create robust ETL pipelines.
• Develop and maintain data warehouses and data lakes for centralized and organized data access.
• Implement data security measures to protect sensitive information and ensure compliance with privacy regulations.
• Continuously monitor and optimize system performance for reliability and scalability.
• Data Preparation: Clean, transform, and prepare raw data for analysis, ensuring accuracy, completeness, and readiness for modeling.
• Data Integration: Integrate data from diverse sources, creating a unified and comprehensive organizational data view.
• Data Quality: Continuously assess and improve data quality, reliability, and consistency to ensure trustworthy data for analysis.
• Data Acquisition: Identify, evaluate, and integrate new data sources to enhance data infrastructure.
• Analytical Tools Development: Design and develop tools and programs to streamline data processing, analysis, and reporting.
  • Collaboration: Partner with and lead other engineers and architects, providing technical expertise and ensuring seamless integration between data infrastructure and analytical tools