Data Engineer - Python

$120,000 - $120,000/Yr

Beacon Hill Staffing Group - Olympia, WA

Full-time - Senior

Remote - Olympia, WA

Administrative and Support Services

About the position

We are seeking a highly skilled Senior Data Engineer to join our team. This remote position requires a candidate who is proficient in coding, infrastructure management, and automation, with a strong emphasis on building and maintaining robust ETL pipelines and tools. The ideal candidate will serve as a key individual contributor, collaborating closely with the director and providing mentorship to junior team members. In this role, you will be responsible for the development and management of ETL pipelines using technologies such as Python, Kafka, Snowflake, and AWS. You will ensure that data is processed efficiently and delivered in a timely manner. Heavy coding will be a significant part of your responsibilities, focusing on Python development to implement data processing, analysis, and automation tasks. You will also utilize SQL-based Infrastructure as Code (IaC) tools, including CloudFormation, Serverless, and Terraform, for effective infrastructure management. Automation and optimization of manual processes will be crucial, as you will work to enhance scalability and performance by redesigning infrastructure. You will take ownership of all aspects of development, from application code to cloud deployment and ongoing observability. Additionally, you will develop automated unit and integration tests and create and maintain CI/CD pipelines using GitHub Actions. As a mentor, you will be a go-to resource for junior team members, providing guidance and support. Collaboration with the director will be essential to align on project goals and contribute to the overall success of the team.

Responsibilities

• Build and manage ETL pipelines using Python, Kafka, Snowflake, and AWS.
• Focus on Python development to implement data processing, analysis, and automation tasks.
• Utilize SQL-based IaC tools (CloudFormation, Serverless, Terraform) for infrastructure management.
• Automate manual processes, optimize data delivery, and redesign infrastructure to enhance scalability and performance.
• Take ownership of all aspects of development, from application code to cloud deployment and ongoing observability.
• Develop automated unit and integration tests, and create and maintain CI/CD pipelines using GitHub Actions.
• Act as a go-to resource for junior team members, providing guidance and support.
• Work closely with the director to align on project goals and contribute to the overall success of the team.

Requirements

• Extensive experience in Python coding, particularly for ETL and data processing.
• Proficiency with AWS services, Kafka, Snowflake, and experience in building and managing ETL pipelines.
• Experience with Terraform, CloudFormation, and Serverless frameworks for infrastructure management.
• Strong understanding of DevOps practices and experience with CI/CD pipelines, including GitHub Actions.
• Experience with relational databases, SQL, and building unit tests.
• Advanced skills with Git or other version control systems.
• Experience with Docker and working in Linux/Unix environments.