Senior Lead Data Engineer (Python, Spark, AWS)

$234,700 - $267,900/Yr

Capital One - York, PA

Full-time - Senior

York, PA

Credit Intermediation and Related Activities

About the position

The Senior Lead Data Engineer at Capital One is responsible for driving major transformations within the organization by leveraging data and emerging technologies. This role involves collaborating with Agile teams to design, develop, and implement technical solutions, while leading a team of developers in machine learning and distributed systems. The position focuses on foundational data capabilities, including migrating batch workloads and enhancing data governance and observability, ultimately impacting business outcomes through advanced analytics and machine learning.

Responsibilities

• Collaborate with Agile teams to design, develop, test, implement, and support technical solutions.
• Lead a team of developers with expertise in machine learning and distributed microservices.
• Utilize programming languages such as Java, Scala, and Python, along with RDBMS and NoSQL databases.
• Share knowledge of tech trends and mentor other engineering community members.
• Work with digital product managers to deliver cloud-based solutions that enhance user experiences.
• Perform unit tests and conduct code reviews to ensure high-quality code.

Requirements

• Bachelor's Degree in a relevant field.
• At least 6 years of experience in application development.
• At least 2 years of experience in big data technologies.
• At least 1 year of experience with cloud computing (AWS, Microsoft Azure, Google Cloud).

Nice-to-haves

• Master's Degree in a relevant field.
• 9+ years of experience in application development including Python, SQL, Scala, or Java.
• 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud).
• 5+ years of experience with distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL).
• 4+ years of experience working on real-time data and streaming applications.
• 4+ years of experience with NoSQL implementation (Mongo, Cassandra).
• 4+ years of data warehousing experience (Redshift or Snowflake).
• 4+ years of experience with UNIX/Linux including basic commands and shell scripting.
• 2+ years of experience with Agile engineering practices.

Benefits

• Comprehensive health benefits
• Financial benefits including performance-based incentives
• Support for total well-being
• Diversity and inclusion initiatives