Data Engineer
sudlifesparsh
Job Description
- Design, develop and support enterprise data warehouse solutions across On-Premises and Microsoft Fabric.
- Build scalable ETL/ELT and data-integration frameworks for structured and semi-structured enterprise data.
- Design Lakehouse, Warehouse and Medallion architecture patterns based on solution requirements.
- Develop and maintain ingestion, transformation and orchestration pipelines.
- Build curated data layers for operational MIS, regulatory reporting, management reporting, Power BI, APIs consumption.
- Perform complex data transformation using SQL/T-SQL, Spark/PySpark and Fabric-native technologies.
- Modernize/migrate existing on-premises data warehouse workloads to Microsoft Fabric.
- Implement data validation, reconciliation and data-quality controls.
- Perform performance optimization of SQL, pipelines, data models and Fabric workloads.
- Monitor DWH jobs, pipelines and production data processing, including root-cause analysis of failures.
- Implement appropriate security, access control, governance, deployment and monitoring practices.
2. API Development & Application Support
Key Responsibilities
- Design, develop, enhance and support enterprise APIs and data/application integration services.
- Build RESTful APIs and services for integration between enterprise applications, DWH, analytics platforms and downstream consumer systems.
- Design reusable and scalable integration services.
- Implement authentication, authorization, validation, structured logging and appropriate error handling.
- Support APIs used by customer-facing and internal enterprise applications.
- Perform L2/L3 production support and root-cause analysis for APIs and data-related applications.
- Investigate API failures, performance degradation, integration issues and data discrepancies.
- Maintain API documentation and interface specifications.
- Manage API testing, release, deployment and post-production monitoring.
- Coordinate with application, infrastructure, database, security, vendor and business teams during incident resolution.
- Develop APIs/services for data consumption by digital channels, enterprise applications
3. Power BI Dashboard Development
Key Responsibilities
- Design and develop enterprise-grade Power BI dashboards and analytical reports.
- Convert complex business requirements into intuitive management and operational dashboards.
- Develop KPIs, measures, calculations, drill-through and analytical views.
- Implement advanced DAX calculations.
- Optimize dashboard performance and user experience.
- Develop reports connecting to Microsoft Fabric and enterprise data sources.
- Modernize legacy/static MIS into Power BI solutions where appropriate.
- Implement Row-Level Security and appropriate report access controls.
- Troubleshoot production dashboard and refresh issues.
- Support report deployment, versioning and lifecycle management.
4. Semantic Model Design & Development
Key Responsibilities
- Design enterprise semantic models as the standardized consumption layer between enterprise data and reporting.
- Develop reusable dimensions, measures, calculations, relationships and hierarchies.
- Establish common KPI definitions and business logic across reports.
- Develop scalable star-schema-based semantic models.
- Implement advanced DAX measures and calculation logic.
- Optimize semantic models for performance and scalability.
- Support Direct Lake, Import and DirectQuery scenarios based on appropriate architecture.
- Implement model-l