Senior Data Engineer

$76,700 - $115,100/Yr

Pennsylvania State University - State College, PA

Full-time - Mid Level

Remote - State College, PA

10,001+ employees

Educational Services

About the position

Penn State's Office of Planning, Assessment, and Institutional Research (OPAIR) is seeking a motivated Senior Data Engineer to join our Data Engineering Team. The Data Engineering Team collaborates with the Data Modeling, Institutional Reporting, Data Science and Analytics, and Operations Teams. You'll have the opportunity to use your talent and skill to empower informed decision-making through a robust data ecosystem for state and federal reporting, accreditation, and university-wide initiatives. In this role, you will collaborate with a team of data engineers to support OPAIR's Institutional Research, Assessment, Data Governance, Planning, and Data Science teams, contributing to the continual evolution of Penn State's data ecosystem. You will assume a pivotal role as the backup for the Director of Data Engineering, ensuring seamless continuity and essential assistance in their absence. Your responsibilities will include assisting in the development, deployment, and maintenance of cloud-based solutions, with a focus on the Microsoft Azure and Oracle Cloud Infrastructure environments. You will also craft, sustain, and enhance legacy data warehouses/ETL solutions on Microsoft SQL Server and Oracle databases, ensuring their ongoing efficiency and optimization. Monitoring and maintaining the existing portfolio of data acquisition jobs will be a key part of your duties. You will analyze and enhance existing software development methodologies, focusing on source code control, testing procedures, and release management to drive continuous improvement and effective implementation. Articulating progress updates and addressing any concerns transparently with both management and peers will be essential for streamlined communication. You will collaborate with teams to identify risks, issues, requirements, and design solutions for project-level activities. Facilitating collaborative partnerships among university teams by actively exchanging information and lending support to enhance data-related initiatives will also be part of your role. Proactively pursuing continuous professional growth by remaining abreast of the latest advancements in data engineering, technology, and business intelligence practices is encouraged. You will share acquired insights and knowledge with the team to foster a collaborative learning environment. The work location is flexible: fully remote within the United States, fully in-person at our University Park Campus, or a hybrid model. Standard working hours are in the Eastern time zone.

Responsibilities

• Collaborate with a team of data engineers to support OPAIR's Institutional Research, Assessment, Data Governance, Planning, and Data Science teams.
• Assume a pivotal role as the backup for the Director of Data Engineering, ensuring seamless continuity and essential assistance in their absence.
• Assist in the development, deployment, and maintenance of cloud-based solutions, focusing on Microsoft Azure and Oracle Cloud Infrastructure environments.
• Craft, sustain, and enhance legacy data warehouses/ETL solutions on Microsoft SQL Server and Oracle databases, ensuring ongoing efficiency and optimization.
• Monitor and maintain existing portfolio of data acquisition jobs.
• Analyze and enhance existing software development methodologies, focusing on source code control, testing procedures, and release management.
• Articulate progress updates and address any concerns transparently with both management and peers.
• Collaborate with teams to identify risks, issues, requirements, and design solutions for project-level activities.
• Facilitate collaborative partnerships among university teams by actively exchanging information and lending support to enhance data-related initiatives.
• Proactively pursue continuous professional growth by remaining abreast of the latest advancements in data engineering, technology, and business intelligence practices.
• Share acquired insights and knowledge with the team to foster a collaborative learning environment.

Requirements

• Bachelor's degree or higher in an IT-related discipline and at least 6 years of relevant experience, or equivalent combination of education and experience.
• Mastery in writing and optimizing SQL queries, stored procedures, and functions.
• Experience with ETL tools like SSIS (SQL Server Integration Services) and ODI (Oracle Data Integrator) for designing and managing ETL workflows.
• Ability to design efficient data models and schema.
• Strong understanding of Microsoft SQL Server databases, including performance tuning, indexing, and maintenance.
• Expertise in transforming data formats, handling data cleansing, and applying necessary transformations for consistency.
• Proficient in identifying and resolving ETL process and MS SQL / Oracle database issues, error handling, and debugging.
• Familiarity with data warehousing principles and best practices for data integration and storage.
• Skills in optimizing ETL and database processes for efficiency, scalability, and performance.
• Ability to create clear and comprehensive documentation for workflows, processes, and configurations.
• Effective communication skills to collaborate with team members, understand business requirements, and translate them into solutions and accompanying customer documentation.
• Knowledge of data security principles and practices to ensure data integrity and compliance.
• Strong analytical skills to analyze complex data scenarios and design appropriate solutions.
• Ability to adapt to evolving technologies and a willingness to continuously learn and explore new tools and techniques in the applicable domains.

Nice-to-haves

• Mastery in scripting languages like Python and PowerShell for automating tasks and manipulating data efficiently.
• Familiarity with Azure Active Directory (Entra ID) with a focus on Permissions, Groups, and Applications that pertain to AAD applications and SQL Server integration and management.
• Familiarity with Oracle Cloud Infrastructure with a focus on private-link networking and infrastructure to support Oracle Cloud Analytics implementations and integrations with on-premise Oracle databases.
• Knowledgeable about DevOps practices and software to facilitate efficient development, deployment, and operations.
• Understanding the significance of unstructured or semi-structured data in contemporary data analysis, leveraging its potential for insights.
• Hands-on experience in ETL processes across multiple source systems, including Linux, UNIX, Oracle, SAP, and Windows platforms.

Benefits

• Flexible work location options (fully remote, in-person, or hybrid)
• Standard working hours in the Eastern time zone
• Salary range of $76,700.00 - $115,100.00