DATA ENGINEER [PYTHON]

Sriven Systems - New York, NY

Full-time - Mid Level

New York, NY

Professional, Scientific, and Technical Services

About the position

The Data Engineer [Python] position is a contract role based in New York, New York, with a duration of 12 months. The primary focus of this role is to leverage advanced programming skills, particularly in Python and potentially GO, to develop and maintain robust data infrastructure on AWS. The ideal candidate will have extensive experience with AWS services and tools, ensuring that data pipelines are efficient, scalable, and secure. This role requires a deep understanding of AWS infrastructure, including but not limited to AWS Lambda, AWS CloudTrail, and various data storage solutions such as Amazon S3 and DynamoDB. In addition to technical skills, the Data Engineer will be expected to work collaboratively within an Agile development environment, utilizing tools such as JIRA and Jenkins for project management and continuous integration/continuous deployment (CI/CD) processes. The candidate should be adept at writing automated test cases and producing high-quality code, along with comprehensive documentation to support the development process. The ability to quickly learn internal tools and processes is essential, as is a strong foundation in technical design and problem-solving skills. Overall, this position offers an exciting opportunity for a skilled data engineer to contribute to innovative projects while working with cutting-edge technologies in a dynamic team environment.

Responsibilities

• Develop and maintain data pipelines using Python and/or GO programming languages.
• Implement and manage AWS infrastructure and tooling for data processing.
• Utilize AWS services such as Lambda, CloudTrail, SNS/SQS, and others to build scalable solutions.
• Write Infrastructure as Code (IaC) using AWS CloudFormation.
• Work with AWS EKS, AWS ECS, Amazon EMR, and AWS Glue for data orchestration.
• Manage data storage solutions including Amazon S3, DynamoDB, and RDS.
• Collaborate with teams using JIRA for project management and Jenkins for CI/CD processes.
• Write automated test cases to ensure code quality and reliability.
• Produce high-quality code and maintain supporting documentation.
• Participate in Agile development methodologies to enhance team productivity.

Requirements

• Proficiency in Python programming and/or GO programming.
• 5+ years of professional experience with AWS infrastructure and tooling.
• Professional experience with AWS Lambda, AWS CloudTrail, Amazon SNS/SQS.
• Experience with Infrastructure as Code (IaC) using AWS CloudFormation.
• Familiarity with AWS EKS, AWS ECS, Amazon EMR, and AWS Glue.
• Experience with Amazon S3, Amazon DynamoDB, and Amazon RDS.
• Familiarity with Amazon Athena, Amazon API Gateway, and Amazon VPC.
• Proficient in using JIRA, Jenkins, and GitHub for project management and version control.
• Experience with CI/CD pipelines, including Jenkins, AWS Code Pipelines, and Kubeflow Pipelines.
• Strong technical design and problem-solving skills.