Data Engineer

Spectraforce Technologies - North Chicago, IL

Full-time - Mid Level

North Chicago, IL

Administrative and Support Services

About the position

The Data Engineer role at Spectraforce Technologies involves partnering with various teams to define and execute data acquisition, storage, transformation, and processing. The position is part of the Product Analytics team, which aims to unlock the potential of enterprise data through a modern cloud-based data lake architecture. The role requires expertise in data sources and technology, with a focus on creating actionable data for operational and analytics initiatives that drive sustainable revenue and growth.

Responsibilities

• Support large scale data pipelines in a distributed and scalable environment
• Enable and optimize production AWS environment for data infrastructure and frameworks
• Create Terraform modules to automate deployments
• Utilize Databricks and Datalake technologies
• Partner with development teams and stakeholders to provide technical solutions
• Lead design and development of innovative batch and streaming data applications using AWS technologies
• Provide technical direction and guidance to the team
• Coordinate and guide cross-functional projects involving various stakeholders
• Manage agile development and delivery in collaboration with project managers and product owners

Requirements

• Bachelor's degree in a quantitative field (statistics, software engineering, business analytics, information systems, aviation management or related degree)
• 5+ years of experience in data engineering or ETL development role
• Experience performing root cause analysis on data and processes
• Strong analytic skills with structured, semi-structured, and unstructured datasets
• Experience with Big Query, SQL server, etc.
• Experience with AWS cloud services: Redshift, S3, Athena, etc.
• Proficiency with SQL and various database interface tools: SSMS, Oracle SQL developer, etc.
• Strong initiative to take ownership of data-focused projects
• Ability to communicate complex quantitative concepts clearly
• Proven proficiency with Microsoft Excel and PowerPoint
• Strong problem-solving skills

Nice-to-haves

• Master's degree
• Experience with Quantum Metrics and Akamai
• Experience with languages: Python, R, etc.
• Strong experience with continuous integration & delivery using Agile methodologies
• Data engineering experience with transportation/airline industry