Insight Global - Rahway, NJ
posted about 2 months ago
As a Data Engineer at Insight Global, you will play a crucial role in designing, developing, and maintaining data pipelines that extract data from various sources to populate our data lake and data warehouse. Your responsibilities will include developing data transformation rules and enhancing our data modeling capabilities. You will collaborate closely with Product Analysts, Data Scientists, and Machine Learning Engineers to identify and transform data, ensuring it is understandable and actionable for our teams. In this position, you will implement data quality checks and maintain data catalogs, utilizing orchestration, logging, and monitoring tools. A strong understanding of information architecture concepts and their implementation is essential, along with proficiency in Git for version control and familiarity with various branching strategies. You will also employ test-driven development methodologies when building ELT/ETL pipelines, ensuring high-quality deliverables. Good documentation skills throughout the Software Development Life Cycle (SDLC) are required, as is the ability to communicate effectively and collaborate with other teams to develop innovative solutions. You will demonstrate a growth mindset and work alongside enterprise teams to achieve our goals. Additionally, experience with DevSecOps practices, including continuous integration (CI) and continuous delivery (CD), source code version control (e.g., Git), infrastructure-as-code (e.g., CloudFormation, Terraform), and containerization (e.g., Docker) is highly valued.