Data Engineer
instahyre
Job Description
- Experience: 6 to 10 years of professional experience in data engineering, backend software engineering, or a closely related field.
- Programming Mastery: Strong proficiency in at least one primary data programming language, preferably Python, Scala, or Java.
- Advanced SQL: Deep expertise in writing complex, highly optimized SQL queries and understanding relational database engines.
- Data Modeling: Solid understanding of data warehousing concepts, dimensional modeling, and database design principles.
- ETL/ELT Fundamentals: Proven experience building batch processing pipelines and integrating data from disparate sources.
- Version Control and Best Practices: Experience with Git, code reviews, writing unit tests, and CI/CD principles.
Preferred / Specialist Skills (Optional):
- We do not expect you to know everything. Experience with at least one tool from the categories below is highly preferred:
- Enterprise ETL and Data Integration: Hands-on experience with enterprise-grade integration tools like Informatica, Talend, Pentaho, OR IBM DataStage.
- Cloud Platforms: Experience with AWS (e. g., S3 RDS), GCP (e. g., Cloud Storage), OR Azure.
- Cloud Data Warehouses: Familiarity with Snowflake, Google BigQuery, OR Amazon Redshift.
- Distributed Processing: Hands-on experience with Apache Spark, Databricks, OR Hadoop/Hive.
- Pipeline Orchestration: Knowledge of scheduling and dependency management using Apache Airflow, Dagster, OR Prefect.
- Streaming / Real-Time Data: Experience with message brokers and streaming tools like Apache Kafka, AWS Kinesis, OR GCP Pub/Sub.
- Infrastructure as Code (IaC): Familiarity with deploying infrastructure using Terraform OR AWS CloudFormation.