Custom Software Engineer

Build automated data ingestion pipelines from Azure Data Lake Storage (ADLS) and Azure SQL.
Develop and maintain Azure Data Factory (ADF) pipelines with linked services, triggers, and parameterization.
Containerize ETL workflows using Azure Container Apps for scalability and portability.
Configure scheduling and message-based execution using Azure Service Bus.
Utilize Ansible playbooks for infrastructure provisioning and ensuring deployment consistency.
Maintain Redis cache for managing intermediate LLM (Large Language Model) processing states.
Implement monitoring, alerting, and logging through Azure Application Insights.
Collaborate with analytics and AI teams to align ingestion schemas and infrastructure with downstream workflows.

Technical Experience & Professional Attributes:
Strong knowledge of Azure Data Services (ADF, ADLS, Azure SQL, Azure Service Bus, Application Insights).
Experience with containerization and CI/CD using Azure Container Apps and Ansible.
Solid understanding of ETL design principles, data orchestration, and workflow automation.
Familiarity with caching mechanisms (Redis) and their use in AI/ML pipelines.
Excellent problem-solving, debugging, and analytical skills.
Strong communication and collaboration skills with cross-functional teams.

Educational Qualifications:
Bachelor s degree in Computer Science, Information Technology, or a related field.
(Optional: Add certifications such as Microsoft Certified: Azure Data Engineer Associate if applicable.)

Additional Information:
Available for immediate or planned deployment based on project timelines.
Committed to delivering high-quality, scalable, and efficient data solutions.
Primary location preference: Bangalore.