Data Engineer Lead

Insight Global - Herndon, VA

Full-time - Mid Level

Herndon, VA

Administrative and Support Services

About the position

Insight Global is seeking a data engineer to join one of our federal clients and support a team of engineers working towards some of the industry's most mission-driven projects. Your role will encompass the management of development and deployment frameworks, transforming diverse data sets into coherent and significant insights. You will lead and nurture a versatile team comprising analysts, data engineers, developers, and data users, thriving in a dynamic, agile setting. Leveraging your proficiency in data analysis and scrutiny, you will supervise the evaluation, conception, construction, and upkeep of expandable systems tailored to client needs. This position requires a strong background in application development and data engineering, with a focus on creating scalable solutions that meet the needs of federal clients. You will be responsible for overseeing the entire data lifecycle, from data ingestion to processing and analysis, ensuring that the systems you develop are robust and efficient. Your leadership will be crucial in guiding your team through complex projects, fostering collaboration, and driving innovation in data solutions.

Responsibilities

• Manage development and deployment frameworks for data applications.
• Transform diverse data sets into coherent insights.
• Lead and nurture a team of analysts, data engineers, developers, and data users.
• Supervise the evaluation, conception, construction, and upkeep of expandable systems.
• Oversee the creation of scalable ETL and ELT workflows using tools like Apache Spark and Apache Airflow.
• Develop scripts and programs for converting various types of data into usable formats.
• Support project teams in scaling, monitoring, and operating data platforms.

Requirements

• 7+ years of experience in application development.
• 7+ years of experience designing, developing, operationalizing, and maintaining complex data applications at an enterprise scale.
• 5+ years of experience creating software for retrieving, parsing, and processing structured and unstructured data.
• 5+ years of experience developing scalable ETL and ELT workflows using Apache Spark, Apache Airflow, or Apache Flink.
• Experience developing scripts and programs for converting various types of data into usable formats.
• Experience supervising others and leading projects within a collaborative, cross-functional team environment.
• Ability to obtain a security clearance.
• Bachelor's degree with 10+ years of experience using Python, SQL, Scala, or Java.
• 5+ years of experience with a public cloud, including AWS, Microsoft Azure, or Google Cloud.
• 5+ years of experience with distributed data and computing tools, including Spark, Databricks, Hadoop, Hive, AWS EMR, or Kafka.
• 5+ years of experience working on real-time data and streaming applications.
• 3+ years of experience with Palantir Foundry.
• 5+ years of experience with NoSQL implementation, including MongoDB or Cassandra.
• 5+ years of experience with data warehousing, including DataBricks Lakehouse or Snowflake.
• 5+ years of experience with UNIX and Linux, including basic commands and Shell scripting.
• 5+ years of experience with Agile engineering practices.