DATA ENGINEER - PySpark

Big Data & Data Engineering

• Strong experience in PySpark
• Spark SQL, DataFrames, RDDs ,memory optimization 
• Data Lake Architecture 
• ETL/ELT Framework Development

AWS Services

• AWS Glue
• Amazon Athena
• Amazon Redshift
• AWS Lambda
• Amazon Kinesis (Data Streams / Firehose)
• Amazon S3
• AWS IAM
• AWS CloudWatch
• AWS Secrets Manager

Programming

• Python
• Advanced SQL

Data Warehousing

• Star Schema & Snowflake Schema
• Dimensional Modeling
• SCD Type 1 & Type 2
• Data Mart Design

Preferred Skills

• Apache Iceberg / Delta Lake / Hudi
• AWS Step Functions
• Terraform / CloudFormation
• CI/CD using GitHub Actions, Jenkins, or AWS CodePipeline
• Airflow or equivalent orchestration tools
• Power BI / Tableau
• Healthcare domain knowledge is a plus