Data Engineer

USO - Arlington, VA

Full-time - Mid Level

Remote - Arlington, VA

Religious, Grantmaking, Civic, Professional, and Similar Organizations

About the position

The Data Engineer works collaboratively to understand, obtain, process, store, and manage company data from various internal and external systems and sources. This includes participating in stakeholder interviews to gather data and reporting requirements, developing data pipelines, and maintaining and expanding the USO's data engineering ecosystem. The Data Engineer will apply proven technical expertise and problem-solving skills to contribute to the organization's data-driven journey.

Responsibilities

• Maintain and expand required infrastructure for optimal extraction, transformation, loading, and cataloging of data from various data sources using Azure and Databricks technologies.
• Assemble sets of data of varied size, complexity, and velocity that meet non-functional and functional business requirements.
• Build and maintain information marts to serve curated data produced by pipelines to enable data analysts and reporters to identify business insights and to report on key business performance metrics.
• Work with stakeholders including data, design, programs, and executive teams to understand data requirements and assist them with data-related technical issues.
• Routinely monitor, identify, recommend, and implement improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes.
• Troubleshoot and resolve data issues such as duplicate, inconsistent or inaccurate data, data downtime, and referential integrity.
• Develop and maintain documentation for data engineering processes, data diagrams, workflows, ERDs, dictionaries, lineage, catalogs, and other documentation artifacts.
• Contribute to formalizing organizational data governance and informing evolution of data management.
• Stay up-to-date with emerging trends and technologies in data engineering.
• Other duties as assigned.

Requirements

• Bachelor's degree in the field of computer science, information technology or related field preferred.
• 3-5 years of progressive work experience in data engineering.
• In-depth understanding of data engineering best practices as well as expertise with the tools needed to debug and diagnose issues.
• Proficiency with Python in a data engineering context, including Pandas and PySpark.
• Strong knowledge of SQL and experience with data warehousing, data modeling, and information mart design.
• Demonstrated experience with Azure Data Factory, Azure Data Lake, and Databricks.
• Experience with data visualization tools, preferably Power BI.
• Experience with Git and version control systems as well as CI/CD process with Azure DevOps.
• Familiarity with big data technologies such as Hadoop, Spark, and NoSQL databases.
• Familiarity with machine learning and data science techniques.
• Understanding of data lifecycle, data security, data lineage, master data management, Data Ops, and ML Ops.
• Ability to achieve desired results while working collaboratively in a team environment.
• Strong interpersonal and problem-solving skills.
• Demonstrated ability to exercise sound judgment related to controversial and/or culturally sensitive subjects.
• Demonstrated initiative and self-motivation, with an attitude to welcoming challenges and new responsibilities.
• Ability to work well individually, as a team, and with other teams within the organization.
• Must be a strong advocate of the USO's mission to lift the spirits of America's troops and their families.

Nice-to-haves

• Relevant experience in a not-for-profit, military, multicultural and/or global organization preferred.
• Knowledge of R language is a plus.
• Knowledge of Kubernetes and other Azure services used for data analytics preferred.

Benefits

• Generous paid time off, including volunteer days and military leave.
• Remarkable 401k match.
• Comprehensive benefits.
• Opportunities for training and education reimbursement.