Data Engineering
ripplehire
Job Description
- Collaborate with cross-functional teams to design, develop, and deploy data tools that support business operations and product initiatives.
- Build and maintain scalable data pipelines and ETL processes using Apache Spark and PySpark.
- Perform analysis and processing of large-scale datasets leveraging AWS cloud services.
- Own product features throughout the software development lifecycle, including design, development, testing, deployment, and production support.
- Evaluate emerging big data technologies and prototype innovative solutions to enhance data processing capabilities and architecture.
- Develop and automate data workflows and pipelines to improve efficiency, reliability, and scalability.
- Implement best practices for CI/CD, code quality, testing, and deployment automation.
- Conduct functional testing and validation of data solutions to ensure accuracy and performance.
- Monitor, troubleshoot, and optimize data processing jobs and workflows.
- Work closely with stakeholders to understand data requirements and translate them into scalable technical solutions.