Lead Software Engineer (Big Data Engineer)

JPMorgan Chase - Plano, TX

Full-time

Plano, TX

Credit Intermediation and Related Activities

About the position

As a Lead Big Data Software Engineer at JPMorgan Chase within the Consumer & Community Banking sector, you will play a crucial role in an agile team focused on enhancing and delivering market-leading technology products. This position involves critical technology solutions across various business functions, with a strong emphasis on data management and transformation, particularly in migrating from Hadoop to AWS. You will be responsible for creating specifications, developing ETL pipelines, and ensuring data quality and consistency while fostering a culture of diversity and inclusion within the team.

Responsibilities

• Promote to become a data domain expert and drive to know everything about the data on the platform.
• Create Functional and Technical Specifications, Epics and User Stories, Process Flows, Data Analysis, Mapping Documents, Implementation Plan, Agile artifacts.
• Migrate from Hadoop to AWS using Pipelines and EMR.
• Develop, enhance, and test new/existing interfaces.
• Work on developing and enhancing ETL pipelines and design solutions.
• Identify, analyze, and interpret trends or patterns in complex data sets.
• Transform existing ETL logic into the Hadoop Platform.
• Innovate new ways of managing, transforming, and validating data.
• Establish and enforce guidelines to ensure consistency, quality, and completeness of data assets.
• Apply quality assurance best practices to all work products.

Requirements

• Formal training or certification on software engineering concepts and 5 years of applied experience.
• Knowledge of application, data, and infrastructure architecture disciplines.
• Strong experience with documentation and structuring information in tools like Confluence and Jira.
• Experience in SparkSQL, Impala, and Big Data technologies.
• Familiarity with Data Science concepts and applying them to analyze large volumes of data.
• Comfortable with data concepts; Oracle, Java, Python, Spark, Kafka, HDFS, AirFlow, Elastic Search.
• Working proficiency in SDLC CI/CD Execution (GitHub, Jenkins, SNOR, Spinnaker, AIM, etc.).
• Minimum 2 years of experience with AWS Services like Lambda and EC2.
• Experience in real-time streaming data.
• Strong experience with UNIX shell scripting is a must.
• Experience with relational database environments (Oracle, Teradata, SQL Server, etc.) leveraging databases, tables/views, stored procedures, agent jobs, etc.

Nice-to-haves

• Experience in Athena, EMR, Redshift, Glue, Kinesis, Aurora, RDS, S3.
• Knowledge in one or more modern programming languages like Java or Python is a plus.
• AWS Cloud Practitioner certification is a plus for applicants and expected upon joining the team.
• Experience working on AdTech/MarTech platforms is a plus.