Python Pyspark (last round is in-person interview)

Incorporan - Columbus, OH

Full-time - Mid Level

Columbus, OH

Professional, Scientific, and Technical Services

About the position

The Python Pyspark position involves collaborating with a team to enhance a data platform, optimizing data pipelines, and providing guidance to other data professionals. The role requires a solid understanding of AWS tools and a commitment to continuous learning, making it ideal for someone passionate about data engineering and technology.

Responsibilities

• Collaborate with the team to build out features for the data platform and consolidate data assets
• Build, maintain and optimize data pipelines built using Spark
• Advise, consult, and coach other data professionals on standards and practices
• Work with the team to define company data assets
• Migrate CMS data platform into Chase's environment
• Partner with business analysts and solutions architects to develop technical architectures for strategic enterprise projects and initiatives
• Build libraries to standardize how we process data
• Implement automation on applicable processes

Requirements

• 5+ years of experience in a data engineering position
• Proficiency in Python (or similar) and SQL
• Strong experience building data pipelines with Spark
• Strong verbal & written communication skills
• Strong analytical and problem-solving skills
• Experience with relational datastores, NoSQL datastores, and cloud object stores
• Experience building data processing infrastructure in AWS
• Bachelor's Degree in Computer Science/Programming or similar is preferred
• Must have legal right to work in the USA

Nice-to-haves

• Experience with infrastructure as code solutions, preferably Terraform
• Cloud certification
• Production experience with ACID compliant formats such as Hudi, Iceberg or Delta Lake
• Familiar with data observability solutions, data governance frameworks