Senior Data Engineer

Capital One - Washington, DC

Full-time - Mid Level

Washington, DC

5,001-10,000 employees

Credit Intermediation and Related Activities

About the position

As a Senior Data Engineer at Capital One, you will play a crucial role in designing and building a scalable, low-latency, fault-tolerant streaming data platform. This position involves leveraging big data technologies to create distributed streaming data pipelines and analytics data stores, ultimately driving a major transformation within the company. You will work in a collaborative, agile environment, focusing on delivering robust cloud-based solutions that empower millions of Americans financially.

Responsibilities

• Design and build an enterprise-level scalable streaming data platform.
• Develop distributed streaming data pipelines and analytics data stores using frameworks like Flink and Spark Streaming.
• Build data pipelines using big data technologies such as Spark, Flink, Kafka, and Snowflake.
• Collaborate with Agile teams to design, develop, test, and implement technical solutions.
• Participate in code reviews and perform unit tests to ensure code quality and performance.
• Stay updated on tech trends and mentor other engineering community members.

Requirements

• Bachelor's Degree in a relevant field.
• At least 3 years of experience in application development (excluding internship experience).
• At least 1 year of experience in big data technologies.

Nice-to-haves

• 5+ years of experience in application development including Python, SQL, Scala, or Java.
• 2+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud).
• 3+ years of experience with distributed data computing tools (Kafka, Spark, Flink).
• 2+ years of experience working on real-time data and streaming applications.
• 2+ years of experience with NoSQL implementation (DynamoDB, OpenSearch).
• 2+ years of data warehousing experience (Redshift or Snowflake).
• 3+ years of experience with UNIX/Linux including basic commands and Shell Scripting.
• 2+ years of experience with Agile engineering practices.

Benefits

• Comprehensive health benefits
• Financial benefits
• Inclusive workplace policies