Senior Data Engineer

Plains All American Pipeline - Houston, TX

Full-time - Senior

Houston, TX

Pipeline Transportation

About the position

The Senior Data Engineer at Plains All American is a pivotal role within the Enterprise Data Engineering team, responsible for leading the technical direction of data engineering initiatives. This position requires a strong technical background in Azure data solutions and a proven ability to align technical deliverables with business requirements. The Senior Data Engineer will work collaboratively with various stakeholders to design and implement scalable data solutions, ensuring compliance with data security and privacy regulations while driving continuous improvement in data practices.

Responsibilities

• Define and drive the data engineering strategy in alignment with business goals.
• Establish software development standards and processes for code reviews.
• Lead designing workshops to fulfill functional and non-functional requirements.
• Develop and maintain advanced Data Engineering processes using Spark, Python, Scala, Databricks, and Azure Data Factory.
• Implement and configure Azure services to build and manage data pipelines and workflows.
• Create best practices for data modeling and indexing for optimal performance.
• Oversee the implementation of data security measures and ensure compliance with regulations.
• Monitor and optimize the performance of data solutions, addressing bottlenecks and fine-tuning queries.
• Provide architectural and design direction to development and AMS teams.
• Translate business requirements into design specifications.

Requirements

• Bachelor's degree in Math, Computer Science, Engineering, Business Administration, or relevant practical experience preferred.
• 10+ years of experience in data engineering or software engineering.
• Proven technology experience in a complex enterprise environment.
• Strong communication skills, both written and spoken.
• Proficiency in Azure Synapse Analytics, Azure Data Factory, Azure Databricks, Azure SQL Database.
• Experience in data modeling, data warehousing, and data lake design and implementation.
• Experience with programming languages such as SQL, Python, or Scala.
• Knowledge of data integration techniques, ETL processes, and data pipeline orchestration.
• Knowledge of software engineering best practices such as code reviews and testing frameworks.

Nice-to-haves

• Knowledge of AI and ML is a plus.
• Experience with agile delivery frameworks.

Benefits

• Medical/Dental/Vision insurance
• Vacation time off
• Flex days
• Hybrid work schedule (combining office and remote work)