Lead Data Engineer

Elgebra - Los Angeles, CA

Full-time

Los Angeles, CA

Professional, Scientific, and Technical Services

About the position

The Lead Data Engineer will play a pivotal role in designing and implementing robust data engineering solutions that support real-time data processing and analytics. This position requires a deep understanding of system and subsystem architecture, particularly in cloud environments. The ideal candidate will have over 10 years of experience in cloud development, with a strong focus on AWS services. The Lead Data Engineer will be responsible for building and maintaining data pipelines, ensuring data quality, and optimizing data storage and retrieval processes. In this role, you will collaborate closely with cross-functional teams, including product management, marketing, and regulatory affairs, to understand their data needs and deliver solutions that meet those requirements. You will also be expected to provide thought leadership in both business and technology aspects, guiding teams on best practices in data engineering and cloud optimization. The ability to communicate effectively across different time zones and teams is essential, as is a proactive approach to problem-solving. The Lead Data Engineer will also be hands-on during the development phase, rolling up their sleeves to write code and implement solutions. Agile experience is crucial, as the work environment is dynamic and fast-paced, requiring adaptability and quick decision-making. This position offers an exciting opportunity to lead the development of innovative data solutions in a highly regulated environment, such as medical devices or aerospace.

Responsibilities

• Design and implement data engineering solutions for real-time data processing.
• Build and maintain data pipelines to ensure data quality and accessibility.
• Collaborate with cross-functional teams to understand data needs and deliver solutions.
• Provide thought leadership in data engineering best practices and cloud optimization.
• Write code and implement solutions during the development phase.
• Optimize AWS cloud services for cost efficiency and performance.
• Lead the development of distributed systems and ensure system safety and security.

Requirements

• Bachelor's degree or higher in a relevant field.
• 10+ years of experience in system/subsystem architecture and cloud development.
• Strong experience with real-time data engineering platforms.
• Proficient in AWS Cloud services such as Amazon Elastic Cache, API Gateway, DynamoDB, Lambda, SQS, S3, Apache Storm/Spark, Athena, AWS Glue, and Kinesis.
• Good knowledge of AWS/Cloud cost optimization strategies.
• Strong programming skills in Java or Python.
• Excellent communication skills and ability to collaborate with teams across different time zones.
• Proactive mindset with strong problem-solving abilities.
• Experience working in a dynamic and fast-paced environment.
• Agile experience.

Nice-to-haves

• Experience in designing and developing software-intensive systems in regulated environments (e.g., medical devices or Mil/Aero).
• Knowledge of system safety and security practices.
• Experience in medical device product development.
• Experience in integrating products across multiple platforms.
• Proven track record of leading the development of distributed systems.
• Well-developed collaboration skills with experience interfacing between multiple groups and functions (R&D, PMO, Marketing, Regulatory, QA, and External Experts).