Senior Data Engineer

Design, build, and maintain scalable data pipelines

• Handle ingestion and processing of large-scale log data

• Monitor and optimize pipeline performance and reliability

• Work on log parsing and pattern extraction

• Collaborate with data scientists on predictive and proactive maintenance use cases

• Develop and optimize queries using KQL (Kusto Query Language)

• Build and support dashboards using Azure Data Explorer

• Ensure data quality, monitoring, and operational stability

• Support integration with downstream systems such as OneAI and MI Log Interpreter

Required Skills

Core Skills

• Strong experience in data engineering (5–6 years)

• Expertise in writing KQL 

• Expertise in building data pipelines and ETL/ELT processes

• Experience with log data processing and analysis

• Python and C# language skills

Azure Technologies

• Azure Functions

• Azure Event Grid

• Azure Data Explorer (Kusto)+KQL

• Azure Data Factory

• Databricks/Spark

• Delta Lake

Data & Engineering Skills

• Pipeline monitoring and optimization

• Performance tuning of large data systems

• Handling high-volume historical datasets

• Experience in distributed data processing

• Documentation according to CS standards

Good to Have

• Experience with predictive maintenance use cases

• Exposure to machine logs / IoT / telemetry data

• Understanding of data science workflows

Soft Skills

• Strong problem-solving ability

• Data Product Ownership mindset (Development + Operations)

• Ability to work in cross-functional teams

• Proactive approach to optimization and operations

Team Structure

Part of a (4 – 6) member cross-functional team including Development and Operations.

Key Success Metrics

• Stable and optimized data pipelines

• Improved processing efficiency

• Reliable log ingestion and analysis

• Smooth integration with AI systems