Data Engineer II

• As a Big Data Engineer, you'll be responsible for designing and building high performance, and scalable data platforms

• You will be leading team of multiple very enthusiastic and skilled engineers to drive the product development and adoption

• You will be required to effectively collaborate with product teams from business group and understand the product roadmap and vision and translate that into engineering artefacts

• You will work with a variety of teams and individuals, including platform engineers, usecase owners, analytical users to understand their needs and come up with innovative solutions

• You will follow the Amex-way of building engineering products that leads to engineering excellence by adopting DevOps principals

Qualifications

• Bachelor's degree in computer science, Engineering, or a related field. Master's degree would be a plus

• Great to have: - GCP professional certification - Data Engineer/Cloud Architect will be preferable

• Strong hands-on experience with GCP services (Big Query, Dataflow, Dataproc, Pub/Sub, Cloud Storage, Composer, etc.).

• Expert in Google Big Query tool for data warehousing needs

• 4+ years of software development experience with hands-on expertise in coding in Java/Python/Scala etc.

• 3+ years of experience in creating low code/no code ETL tool for setting up large scale data transformation on GCP Cloud

• Strong SQL, RDBMS skills. Expert in writing complex SQLs for different databases such as Hive, MySQL, Postgres etc.  Proficiency in working with NoSQL databases as well

• Experience working with Spark, Big Data and Hive

• Experience in Git Management including PR reviews, maintaining code hygiene

• In-depth understanding of data warehousing concepts, dimensional modelling, and data integration techniques

• Experience in optimizing high volume data processing jobs.

• 3+ year's experience in writing APIs and spring boot services.

• Knowledge of High availability and DR setup.

• Hands-on experience on CICD pipelines, Automated test frameworks, DevOps and source code management will be a big plus (XLR, Jenkins, Git, Stash, Jira, Confluence, Splunk etc.)

• Experience working in Agile/Safe framework for development

• Understanding of Generative AI concepts, including LLMs (Large Language Models), prompt engineering, embeddings, and vector databases.

• Experience integrating GenAI capabilities into data platforms (e.g., semantic search, AI-driven analytics, automated data insights).

• Excellent communication and analytical skills

  • Excellent team-player with ability to work with global team