Sr Staff SWE, Data Lake Engineering

$156,000 - $208,000/Yr

WEX - Boston, MA

Full-time - Senior

Boston, MA

Credit Intermediation and Related Activities

About the position

The Data Lake Engineering team at WEX builds a unified, scalable data platform supporting AI and machine learning applications across the company. They collaborate with departments like Data & Analytics, Security, and Risk to enhance data accessibility, scalability, and innovation, aiming to drive WEX's tech transformation and support a growing customer base. The Sr Staff SWE, Data Lake Engineering will lead the technical architecture and development of WEX's Data Lake platform. Reporting to the VP of Data Lake and AI Engineering, they'll ensure quality, security, and scalability while integrating with company-wide platforms. The ideal candidate brings deep expertise in data lake environments and hands-on leadership skills.

Responsibilities

• Lead and drive the development of technology and platform for the company's Data Lake needs, ensuring functional richness, reliability, performance, and flexibility.
• Help design the architecture and lead the implementation of the Data Lake platform and services.
• Challenge the status quo and drive for better technical solutions to effectively serve all needs in the broad risk area of WEX.
• Guide the team to leverage data and/or AI technology or tools in design and development for high productivity and better solution quality.
• Partner with other senior engineers to understand business needs and develop technical solutions.
• Collaborate with information security teams to ensure data security and privacy, implementing encryption and access controls as required.
• Work with solutions architecture and LOB product and engineering orgs on integrations of the Data Lake Platform with their products/systems.
• Engage with external partners and vendors to evaluate technologies and services that complement the company's Data Lake strategy.
• Create, prioritize, communicate, manage, and execute roadmaps, project plans, and commitments.
• Report on the status of development, quality, operations, and system performance to management and partners.
• Drive the technical vision and strategy of Data Lake to serve business needs for the near and long term.
• Set a high bar and provide technical guidance and mentorship to engineering teams, fostering an environment of continuous learning and innovation.
• Set clear, measurable quality goals for the group in a data-driven way.
• Partner with security and SRE orgs to ensure continuous improvement and meeting of security requirements, SLOs, and error budget.
• Take strong financial responsibility for the team.
• Hold the team accountable for delivering quality results using defined OKRs.
• Advise Senior Leadership on complex situations impacting the organization and lead teams through initiatives, driving quality results and achieving defined OKRs.
• Provide strategic thought leadership on business initiatives using deep technical and business industry expertise.

Requirements

• 15+ years of software design and development experience at a large scale.
• Strong software development skills in chosen programming language, with readiness to learn and program in Java and Python.
• Broad understanding of Data Lakehouse formats like Apache Hudi, Apache Iceberg or Delta Lake is a big plus.
• Demonstrable experience in Spark programming.
• Experience with Spark on DBT with AWS Glue or Apache Polaris is a plus.
• Broad understanding of cloud architecture tools and services, such as S3, EMR, Kubernetes, Lambda functions; experience in AWS and Azure is highly desirable.
• Deep technical hands-on experience in architecture design, development, deployment, and production operation; familiar with agile software development and modern development tools and frameworks.
• Extensive experience in full software development life cycle E2E including production monitoring.
• Rich experience and deep expertise in Big Data and large scale data platforms, especially in Data Lake.
• Highly experienced and deep understanding of CI/CD automation.
• Rich experience in combined engineering practice and agile development; experienced in leading and guiding teams for effective development using this approach.
• Excellent analytical skills; capability in identifying and resolving complex problems.
• Understanding of and experience in payment organizations is a plus.
• Ability to encourage and foster a culture of visibility and transparency across teams.
• Strong written and verbal communication skills.
• Strong philosophy of continuous improvement.
• Bachelor's degree in Computer Science or a related field; a higher degree in a related field is a plus.

Nice-to-haves

• Experience in payment organizations is a plus.

Benefits

• Health insurance
• Dental insurance
• Vision insurance
• Retirement savings plan
• Paid time off
• Health savings account
• Flexible spending accounts
• Life insurance
• Disability insurance
• Tuition reimbursement
• Quarterly or annual bonus based on role and applicable plan.