Data Engineer - Senior Associate

Design, develop, and maintain data pipelines and ETL processes for GenAI projects. 

- Collaborate with data scientists and software engineers to implement machine learning models and algorithms. 

- Optimize data infrastructure and storage solutions to ensure efficient and scalable data processing. 

- Implement event-driven architectures to enable real-time data processing and analysis. 

- Utilize containerization technologies like Kubernetes and Docker for efficient deployment and scalability. 

- Develop and maintain data lakes for storing and managing large volumes of structured and unstructured data. 

- Implement and integrate LLM frameworks (Langchain, Semantic Kernel) for advanced language processing and analysis. 

- Collaborate with cross-functional teams to design and implement solution architectures for GenAI projects. 

- Utilize cloud computing platforms such as Azure or AWS for data processing, storage, and deployment. 

- Monitor and troubleshoot data pipelines and systems to ensure smooth and uninterrupted data flow. 

- Stay up-to-date with the latest advancements in GenAI technologies and recommend innovative solutions to enhance data engineering processes. 

- Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions. 

- Document data engineering processes, methodologies, and best practices. 

- Maintain solution architecture certificates and stay current with industry best practices.