Applications Developer - ETL/Pyspark - Assistant Vice President

citi

Chennai, India 8 Years Exp Posted 3h ago

Job Description

  • ETL Pipeline Development: Building and maintaining Extraction, Transformation, and Loading (ETL) workflows
  • Big Data Processing: Utilize PySpark for data processing, manipulation, and analysis within big data ecosystems.
  • Data Extraction & Ingestion: Extract data from multiple sources including relational databases (SQL), flat files etc.
  • Data Transformation: Create mapping documents and apply business logic to transform large datasets.
  • Performance Tuning: Assist in analyzing and optimizing existing ETL jobs and Spark scripts for better scalability and execution speed.
  • Testing & Documentation: Perform unit testing, validate data accuracy, and maintain technical documentation for all ETL processes.
  • Production Support: Help monitor scheduled daily/weekly jobs and troubleshoot data integration or quality issues

Recommended Qualifications:

  • 8-10 years of relevant experience
  • Experience in systems analysis and programming of software applications
  • Programming Languages: Proficiency in Pyspark and SQL.
  • Big Data Frameworks: Hands-on experience or strong conceptual understanding of PySpark.
  • ETL Tools: Familiarity with Data Integration/Big Data tools and Ab Initio components.
  • Scripting: Basic to intermediate knowledge of Unix/Linux Shell Scripting.
  • Databases: Understanding of Big Data (Hadoop) & RDBMS like Oracle, or PostgreSQL
    • Ability to work under pressure and manage deadlines or unexpected changes in expectations or requirements