Database Developer

Nightwing Sa (Pty) Ltd - Herndon, VA

Full-time - Mid Level

Herndon, VA

About the position

The Database Developer position at Nightwing involves supporting a global software deployment project with a focus on extract, transform, load (ETL) workflows. The role requires collaboration with data scientists, analysts, and managers to manage large datasets in both on-premises and cloud environments, ensuring data integrity and efficient processing.

Responsibilities

• Load large datasets into the client's on-premises and Cloud environments.
• Develop and maintain ingestion algorithms and schemas for large datasets.
• Analyze new large-volume datasets to optimize the data ingest processes.
• Support the creation of Apache NiFi schemas for new data loads.
• Develop software tools that efficiently preprocess, modify, aggregate, load, index, and archive large data collections into clusters in near real-time.
• Ensure proper access controls are implemented.
• Generate metrics to track data ingest statistics to maintain data integrity and provenance.
• Document the data-flows according to standards set by Sponsor.
• Work with multiple Database Management Systems (Oracle, SQL, Mongo, etc.).

Requirements

• Demonstrated professional experience in Computer Science, Computer Engineering, Systems Engineering, or closely related discipline.
• Demonstrated professional experience with AWS cloud services, including long-term storage options, and cloud-based database services.
• Demonstrated experience working with Databricks.
• Demonstrated experience understanding SQL database structures and mapping them between different SQL databases.
• Demonstrated professional experience working with Apache NiFi.
• Demonstrated professional experience working with large data and high performance compute clusters such as Hadoop or similar.
• Demonstrated experience with API development techniques.
• Demonstrated experience developing and deploying ETL processes for large data sets.
• Demonstrated experience creating operating system level scripts to perform ETL operations on SQL databases.
• Demonstrated professional experience with version control systems, preferably Git.
• Demonstrated experience testing the development of software solutions for the extraction, transformation, and loading of data using the most efficient languages for the task such as NiFi, Python, and SQL.
• Demonstrated experience implementing multiprocessing data-flows to parallelize ingest operations.
• Demonstrated experience exhibiting strong coordination and collaboration skills.
• Demonstrated experience working with full-stack developers to deploy applications that leverage large data sets.
• Demonstrated experience communicating technical concepts to non-technical audiences.

Benefits

• Medical insurance
• Dental insurance
• Vision insurance
• Life insurance
• Short-term disability
• Long-term disability
• 401(k) match
• Flexible spending accounts
• Flexible work schedules
• Employee assistance program
• Employee Scholar Program
• Parental leave
• Paid time off
• Holidays