Job Details
Position:AWS Data Engineer/Senior EngineerJob Description:Responsibilities:
- Collaborate with stakeholders to understand business requirements and data needs, and translate them into scalable and efficient data engineering solutions using AWS Data Services.
- Design, develop, and maintain data pipelines using AWS serverless technologies such as Glue, S3, Lambda, DynamoDB, Athena, and RedShift.
- Implement data modelling techniques to optimize data storage and retrieval processes.
- Develop and deploy data processing and transformation frameworks to support both real-time and batch processing requirements.
- Ensure data pipelines are scalable, reliable, and performant to handle large-scale data sizes.
- Implement data documentation and observability tools and practices to monitor and troubleshoot data pipeline performance issues.
- Adhere to privacy and security development best practices to ensure data integrity and compliance with regulatory requirements.
- Collaborate with the DevOps team to automate deployment processes using AWS Code Pipeline.
- Bachelor's degree in Computer Science, Engineering, or a related field.
- 5+ years of experience working in data modelling and building real-time and batch processing data pipelines for large-scale data sizes.
- Strong proficiency in Python programming language.
- Extensive experience with AWS serverless or managed services such as S3, Glue, EMR, Lambda, DynamoDB, Athena, and RedShift.
- Solid understanding of privacy and security development best practices.
- Excellent problem-solving skills and ability to troubleshoot complex data pipeline issues.
- Strong communication and collaboration skills, with the ability to work effectively in a cross-functional team environment.
- Experience with Agile development methodologies is a plus.
You Might Also Like
Help us maintain the quality of jobs posted on PowerToFly. Let us know if this job is closed.
Mission
We're connecting diverse talent to big career moves. Meeting people who boost your career is hard - yet networking is key to growth and economic empowerment. We’re here to support you - within your current workplace or somewhere new. Upskill, join daily virtual events, apply to roles (it’s free!).
Are you hiring? Join our platform for diversifiying your team