Data Engineer

• Hands-on technical person with rich experience on the above mentioned skills.
• Communicating with senior management and customers verbally and in writing.
• Build & Monitor data systems and pipelines.
• Develop Data Frames using PySpark
• Conduct complex data analysis and report on results.
• Combine raw information from different sources.
• Explore ways to enhance data quality and reliability.

Required Skills:

• 4-5 years of data engineering experience loading, analyzing and interpreting data.
• Solid experience with AWS Glue, Lambda, Step Functions, Python, PySpark, Redshift, S3, Kinesis.
• Experience with Airflow / any other orchestration frameworks.
• Knowledge of distributed systems as it pertains to data storage and computing
• Experience with data modeling, warehousing and building ETL pipelines
• Experience working on and delivering end to end projects independently
• Experience with non-relational databases / data stores (object storage, document or key-value stores, graph databases, column-family databases)
• 3+ years of developing and operating large-scale data structures for business intelligence analytics (using ETL/ELT processes) experience, such as ETL Glue etc.,
• Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS and reasonable knowledge/experience writing Cloud Formation Templates (CFT) / Terraform.