Senior Data Engineer

avathon

Bengaluru, India 5 Years Exp Posted 2h ago

Job Description

  • Building and optimizing data architectures to handle large-scale data from various sources IoT, PLC, SCADA including cloud and on-premise systems.
  • Architecting scalable ETL and data workflows to support advanced analytics and machine learning models in Renewable domains
  • Collaborating with cross-functional teams to deliver end-to-end data solutions that drive business insights.
  • Owning and leading projects that focus on data platform modernization, ensuring high performance, security, and scalability.
  • Working on complex data engineering challenges that push the boundaries of cloud data infrastructure.

You'll have:

  • A Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical field.
  • 5+ years of experience designing and implementing large-scale data platforms, with at least 2 years of experience leading or mentoring data engineers.
  • Hands-on experience with Python (Pandas, Polars) and modern big data tools such as Databricks (Spark), Flink, or Kafka.
  • Strong understanding of data pipeline development, ETL frameworks, and data lakes/warehouses.
  • Extensive experience with cloud data platforms such as AWS (S3, Redshift, Lambda, Glue), GCP (BigQuery, Dataflow, Dataproc), or Snowflake.
  • Familiarity with workflow orchestration tools like Airflow or Prefect.
  • Proven ability to lead data engineering teams and mentor junior engineers.


It would be great if you have:

  • Experience working with IoT or real-time streaming data.
  • Knowledge of data governance frameworks and best practices.
  • Exposure to MLOps workflows and integrating data with machine learning models.
  • Experience in the energy or manufacturing sector or familiarity with time-series data.
  • A strong understanding of data warehousing concepts, including Kimball model design.
  • A deep understanding of data security and compliance requirements, particularly in cloud environments.

 

Similar Openings for You