Senior Data Engineer (Python, Spark & AWS)

$165,100 - $188,500/Yr

Capital One - New York, NY

Full-time - Mid Level

New York, NY

Credit Intermediation and Related Activities

About the position

The Senior Data Engineer at Capital One will play a crucial role in building and pioneering technology solutions that address complex business problems. This position is part of Team Astros, a full-stack and data engineering team responsible for enhancing customer experiences across Capital One's platforms. The role involves working on a greenfield roadmap to develop new capabilities for account management, utilizing various technologies including data lakes, ETL pipelines, and cloud services.

Responsibilities

• Collaborate with Agile teams to design, develop, test, implement, and support technical solutions.
• Work with developers experienced in distributed microservices and full stack systems.
• Utilize programming languages such as Java, Scala, Python, and work with RDBMS and NoSQL databases.
• Stay updated on tech trends, experiment with new technologies, and mentor other engineers.
• Collaborate with product managers to deliver robust cloud-based solutions.
• Perform unit tests and conduct code reviews to ensure high-quality code.

Requirements

• Bachelor's Degree
• At least 3 years of experience in application development (Internship experience does not apply)
• At least 1 year of experience in big data technologies.

Nice-to-haves

• 5+ years of experience in application development including Python, SQL, Scala, or Java
• 2+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud)
• 3+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL)
• 2+ year experience working on Real Time data and streaming applications
• 2+ years of experience with NoSQL implementation (Mongo, Cassandra)
• 2+ years of data warehousing experience (Redshift or Snowflake)
• 3+ years of experience with UNIX/Linux including basic commands and Shell Scripting
• 2+ years of experience with Agile engineering practices.

Benefits

• Comprehensive health benefits
• Financial benefits including performance-based incentives
• Support for total well-being
• Diversity and inclusion initiatives
• Flexible work arrangements