Nightwing Sa (Pty) Ltd - Sterling, VA

posted about 2 months ago

Full-time - Mid Level
Sterling, VA

About the position

Nightwing is seeking a Database Developer to support a multifaceted software modernization effort. This role is crucial for the successful execution of extract, transform, load (ETL) workflows, which are essential for managing large datasets. The Database Developer will engage regularly with data scientists, analysts, and managers to ensure that data is efficiently processed and utilized. The position is contingent upon funding, customer approval, and successful completion of a background investigation, including a TS/SCI with Polygraph clearance. In this role, the Database Developer will be responsible for loading large datasets into both on-premises and cloud environments, developing and maintaining ingestion algorithms and schemas, and analyzing new large-volume datasets to optimize data ingest processes. The developer will also support the creation of Apache NiFi schemas for new data loads and develop software tools that preprocess, modify, aggregate, load, index, and archive large data collections in near real-time. Ensuring proper access controls and generating metrics to track data ingest statistics are also key responsibilities, along with documenting data flows according to standards set by the sponsor. The ideal candidate will have a strong background in computer science or a related field, with demonstrated experience in AWS cloud services, Databricks, SQL database structures, and Apache NiFi. The role requires proficiency in developing and deploying ETL processes, as well as experience with large data and high-performance compute clusters. Strong coordination and collaboration skills are highly desired, as is the ability to communicate technical concepts to non-technical audiences. This position offers an opportunity to contribute to significant cybersecurity and intelligence initiatives, leveraging over 100 years of engineering expertise to meet the needs of today's mission and stay ahead of tomorrow's threats.

Responsibilities

  • Load large datasets into the client's on-premises and Cloud environments.
  • Develop and maintain ingestion algorithms and schemas for large datasets.
  • Analyze new large-volume datasets to optimize the data ingest processes.
  • Support the creation of Apache NiFi schemas for new data loads.
  • Develop software tools that efficiently preprocess, modify, aggregate, load, index, and archive large data collections into clusters in near real-time.
  • Ensure proper access controls are implemented.
  • Generate metrics to track data ingest statistics to maintain data integrity and provenance.
  • Document the data-flows according to standards set by Sponsor.

Requirements

  • Demonstrated professional experience in Computer Science, Computer Engineering, Systems Engineering, or closely related discipline.
  • Demonstrated professional experience with AWS cloud services, including long-term storage options, and cloud-based database services.
  • Demonstrated experience working with Databricks.
  • Demonstrated experience understanding SQL database structures and mapping them between different SQL databases.
  • Demonstrated professional experience working with Apache NiFi.
  • Demonstrated professional experience working with large data and high performance compute clusters such as Hadoop or similar.
  • Demonstrated experience with API development techniques.
  • Demonstrated experience developing and deploying ETL processes for large data sets.
  • Demonstrated experience creating operating system level scripts to perform ETL operations on SQL databases.
  • Demonstrated professional experience with version control systems, preferably Git.
  • Demonstrated experience testing the development of software solutions for the extraction, transformation, and loading of data using the most efficient languages for the task such as NiFi, Python, and SQL.
  • Demonstrated experience implementing multiprocessing data-flows to parallelize ingest operations.

Nice-to-haves

  • Demonstrated experience exhibiting strong coordination and collaboration skills.
  • Demonstrated experience working with full-stack developers to deploy applications that leverage large data sets.
  • Demonstrated experience communicating technical concepts to non-technical audiences.

Benefits

  • Medical insurance
  • Dental insurance
  • Vision insurance
  • Life insurance
  • Short-term disability
  • Long-term disability
  • 401(k) match
  • Flexible spending accounts
  • Flexible work schedules
  • Employee assistance program
  • Employee Scholar Program
  • Parental leave
  • Paid time off
  • Holidays
  • Annual short-term and/or long-term incentive compensation programs
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service