Ryan Specialty Group - Chicago, IL
posted about 2 months ago
The Data Engineer is responsible for designing, building, and maintaining data pipelines, models, ETL (Extract, Transform, Load) processes, and other data infrastructure to support the organization's data needs. This role collaborates closely with data scientists and analysts to understand data requirements and develop scalable solutions that meet the organization's objectives. The Data Engineer will build and optimize data pipelines, integrating data from various sources while ensuring data quality and consistency. Additionally, they will implement data governance practices to maintain the integrity and security of the data being processed. In this position, the Data Engineer will design, build, and maintain scalable data pipelines that ingest, process, and transform data from various internal and external sources. They will develop and maintain data warehouses or data lakes to store structured and unstructured data efficiently, ensuring that data is accessible and reliable for analytics and reporting purposes. The role also involves integrating data from disparate systems, ensuring consistency, accuracy, and reliability across different platforms and applications within the organization. The Data Engineer will conduct data modeling and database design using cloud-based technologies, performing dimensional modeling for data warehouses and data marts. They will optimize data processing and query performance to ensure timely and efficient data retrieval and analysis, utilizing techniques such as indexing, partitioning, and query optimization. Furthermore, they will manage and maintain data infrastructure components such as databases, data warehouses, and cloud-based services, ensuring high availability, scalability, and performance. Implementing data governance policies and procedures is crucial to ensure compliance with regulatory requirements and data security measures to protect sensitive information. The Data Engineer will also implement data quality checks and processes to identify and rectify data anomalies, inconsistencies, and errors, ensuring data integrity for downstream analytics and reporting. Participation in cross-functional teams to develop innovative data-driven solutions that improve operational efficiency, risk assessment, and customer experience in the insurance industry is also a key aspect of this role. The Data Engineer will partner with data analysts, data scientists, and data consumers across IT and the business to optimize data availability and enhance AI/ML models and algorithms. Staying up to date with new technologies and industry trends in data management and big data technologies is essential for success in this position.