Data Engineer - Python

$104,000 - $120,640/Yr

Beacon Hill Staffing Group - Juneau, AK

Full-time

Remote - Juneau, AK

Administrative and Support Services

About the position

The Data Engineer at Beacon Hill Staffing Group, LLC is a pivotal role responsible for designing, developing, and maintaining data pipelines and infrastructure that support data-driven decision-making within the organization. This position is fully remote, operating on Eastern Standard Time (EST) hours, and is offered as an ongoing multi-year contract. The ideal candidate will possess strong proficiency in Python, SQL, cloud technologies, and Kubernetes, ensuring efficient processing, storage, and retrieval of data across various platforms. In this role, the Data Engineer will focus on several key areas, including data pipeline development, cloud data infrastructure, containerization and orchestration, and performance optimization. The engineer will design, build, and maintain scalable and reliable data pipelines using Python and SQL, developing ETL processes to integrate data from various sources into data warehouses and databases. Ensuring data quality and consistency across different data sources and systems is paramount. The position also involves implementing and managing data storage and processing solutions on cloud platforms such as AWS, Azure, or Google Cloud. The Data Engineer will utilize cloud data services like BigQuery, Snowflake, or Redshift to store and analyze large datasets, configuring and managing cloud resources for optimal performance and cost-efficiency. Additionally, the role requires deploying and managing data applications and services using Docker and Kubernetes, developing and maintaining Kubernetes manifests, Helm charts, and CI/CD pipelines to automate data workflows. Monitoring and troubleshooting containerized applications to ensure high availability and reliability is also a critical responsibility. Collaboration is key in this position, as the Data Engineer will work closely with data scientists, analysts, and other stakeholders to understand data requirements and provide necessary support. The engineer will also collaborate with DevOps and infrastructure teams to integrate data solutions with existing systems, documenting data processes, workflows, and configurations for transparency and knowledge sharing.

Responsibilities

• Design, build, and maintain scalable and reliable data pipelines using Python and SQL.
• Develop ETL (Extract, Transform, Load) processes to integrate data from various sources into data warehouses and databases.
• Ensure data quality and consistency across different data sources and systems.
• Implement and manage data storage and processing solutions on cloud platforms (e.g., AWS, Azure, Google Cloud).
• Utilize cloud data services such as BigQuery, Snowflake, Redshift, or similar to store and analyze large datasets.
• Configure and manage cloud resources for optimal performance and cost-efficiency.
• Deploy and manage data applications and services using Docker and Kubernetes.
• Develop and maintain Kubernetes manifests, Helm charts, and CI/CD pipelines to automate data workflows.
• Monitor and troubleshoot containerized applications to ensure high availability and reliability.
• Optimize data processing pipelines for performance and scalability.
• Tune SQL queries and data storage configurations to handle large volumes of data efficiently.
• Implement monitoring and logging solutions to track data pipeline performance and identify issues.
• Work closely with data scientists, analysts, and other stakeholders to understand data requirements and provide support.
• Collaborate with DevOps and infrastructure teams to integrate data solutions with existing systems.
• Document data processes, workflows, and configurations for transparency and knowledge sharing.

Requirements

• Experience as a Data Engineer or in a similar role with a strong focus on Python, SQL, and cloud technologies.
• Proficiency in Python for data engineering tasks, including scripting and automation.
• Advanced SQL skills for querying and manipulating data.
• Hands-on experience with cloud platforms (e.g., AWS, Azure, Google Cloud) and related data services.
• Experience with containerization and orchestration technologies such as Docker and Kubernetes.