Senior Data Engineer

Techwish - Reston, VA

Full-time - Senior

Hybrid - Reston, VA

Administrative and Support Services

About the position

The Senior Data Engineer is responsible for orchestrating, deploying, maintaining, and scaling cloud or on-premise infrastructure targeting big data and platform data management. This role emphasizes reliability, automation, and performance, focusing on developing solutions that transform the company's platforms to deliver data-driven insights and value.

Responsibilities

• Work with Business Analysts and Product team to gather data requirements
• Design and Build AbInitio data graphs and data pipelines to extract data from various databases, flat files, and message queues
• Transform the data to create a consumable data layer for various application uses
• Support Data pipeline with bug fixes and additional enhancements
• Document Technical design, Operational Runbook, etc.

Requirements

• Bachelor's Degree in Computer Science, Information Technology, Engineering, or related field
• 10+ years of IT experience predominantly in Data Integration/Data Warehouse area
• 5+ years of ETL Design and Development experience using Ab Initio
• 1-2 years of Data Integration project experience on Hadoop Platform, preferably Cloudera
• Experience with AbInitio CDC (Change Data Capture) in a Data Integration/ETL project setting
• Working knowledge of HDFS, Hive, Impala, and other related Hadoop technologies
• Sound understanding of SQL and ability to write well-performing SQL queries
• Good knowledge of OLTP and OLAP data models and other data warehouse fundamentals
• Experience with Unix/Linux shell scripting
• Ability to work independently and support junior developers as needed
• Knowledge of Agile Development practices

Nice-to-haves

• Working knowledge in various AWS services
• Some Java development experience