Data Integration Engineer

$122,200 - $220,900/Yr

Leidos - Bethesda, MD

Full-time - Senior

Remote - Bethesda, MD

Professional, Scientific, and Technical Services

About the position

Leidos is seeking a highly skilled Data Integration Engineer to join our dynamic team supporting the National Media Exploitation Center (NMEC) under the DOMEX Technology Platform (DTP) contract. This position is ideal for individuals who are ready to take their careers to the next level through training, teamwork, and exposure to challenging technical work. As a Data Integration Engineer, you will play a crucial role in designing, implementing, maintaining, and monitoring data pipelines that support both research and development prototypes as well as production pipelines. Your success will depend on effective cross-functional collaboration with teams in development, product management, and quality assurance in a fast-paced environment. Most of your work will be conducted on-site at our client location in Bethesda, MD, but we offer a flexible schedule, allowing for some unclassified development tasks to be performed remotely. The percentage of remote work will vary based on client requirements and deliverables. As a senior member of the team, you will leverage your deep expertise in data engineering to collaborate closely with infrastructure and network engineers, data scientists, and system engineers. Key tasks include performing database builds, installations, configurations, administration, and troubleshooting of various database systems such as MariaDB/MySQL, Postgres, Elasticsearch, Qdrant, and Milvus. You will also ensure data integrity by employing data engineering best practices and maintaining comprehensive documentation, including data dictionaries and system diagrams.

Responsibilities

• Design, implement, maintain, and monitor data pipelines for R&D prototypes and production.
• Perform database builds, installations, configurations, administration, and troubleshooting of database systems.
• Ensure data integrity by employing data engineering best practices.
• Maintain database and data pipeline documentation, data dictionaries, and system diagrams.
• Collaborate with cross-functional teams including development, product, and QA.
• Utilize SQL, NoSQL, and vector databases for data management and integration.
• Design and maintain ETL and ELT pipelines using technologies such as Spark, Airflow, and Dagster.
• Implement CI/CD methodologies for data pipelines and cloud-native deployments.

Requirements

• Bachelor's Degree with 12+ years of relevant experience or a Master's Degree with 10+ years of relevant experience.
• Active TS/SCI clearance with the ability to obtain and maintain a TS/SCI with Polygraph.
• Experience with SQL, NoSQL, and vector databases such as MSSQL, MySQL, PostgreSQL, Redis, FAISS, Milvus, and Qdrant.
• Proficiency in designing and maintaining ETL and ELT pipelines using technologies like Spark, Airflow, and Dagster.
• Experience with DevOps/MLOps practices and CI/CD methodologies.
• Strong understanding of database design, implementation, maintenance, monitoring, performance tuning, and optimization.
• Expertise in data profiling techniques and data quality evaluation.
• Development experience with Python and data model development.

Nice-to-haves

• Experience supporting data teams and data scientists.
• Experience in production/enterprise systems.
• Familiarity with air-gapped environments.
• Application development and deployment experience in an AWS environment.
• AWS certifications.

Benefits

• Flexible work schedule with some remote work options.
• Professional development opportunities.
• Collaborative team environment.
• Exposure to challenging technical work.