Senior Lead Data Engineer (Python, Spark, AWS)

$234,700 - $267,900/Yr

Capital One - Chicago, IL

Full-time - Senior

Chicago, IL

Credit Intermediation and Related Activities

About the position

The Senior Lead Data Engineer at Capital One plays a crucial role in transforming data capabilities within the organization. This position involves leading a team to design, develop, and implement technical solutions that leverage big data technologies and cloud computing. The role focuses on enhancing data production and consumption processes, ensuring high performance and governance, and tackling complex data challenges that impact business outcomes.

Responsibilities

• Collaborate with Agile teams to design, develop, test, implement, and support technical solutions.
• Lead a team of developers with expertise in machine learning and distributed microservices.
• Utilize programming languages such as Java, Scala, and Python, along with RDBMS and NoSQL databases.
• Share knowledge of tech trends and mentor other engineering community members.
• Work with digital product managers to deliver cloud-based solutions that enhance user experiences.
• Perform unit tests and conduct code reviews to ensure high-quality code.

Requirements

• Bachelor's Degree in a relevant field.
• At least 6 years of experience in application development.
• At least 2 years of experience in big data technologies.
• At least 1 year of experience with cloud computing (AWS, Microsoft Azure, Google Cloud).

Nice-to-haves

• Master's Degree in a relevant field.
• 9+ years of experience in application development including Python, SQL, Scala, or Java.
• 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud).
• 5+ years of experience with distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL).
• 4+ years of experience working on real-time data and streaming applications.
• 4+ years of experience with NoSQL implementation (Mongo, Cassandra).
• 4+ years of data warehousing experience (Redshift or Snowflake).
• 4+ years of experience with UNIX/Linux including basic commands and shell scripting.
• 2+ years of experience with Agile engineering practices.

Benefits

• Comprehensive health benefits
• Financial benefits including performance-based incentives
• Support for total well-being
• Diversity and inclusion initiatives