Big Data Engineer 4

About the position

The Big Data Engineer is responsible for driving technical innovation and implementing big data technologies, particularly focusing on building data pipelines for the Hadoop platform and Google Cloud. This role requires a subject matter expert with strong Python skills and experience in large data integration initiatives, particularly in the cyber security domain.

Responsibilities

• Lead the design and development of engineering solutions for modernizing the data ecosystem.
• Provide technical expertise for the design, implementation, maintenance, and control of data management services.
• Develop self-service, multitenant capabilities on the cyber security data lake.
• Build, certify, and deploy automated services for data management through various stages of the analytical data lifecycle.
• Provide technical leadership in design, engineering, deployment, and maintenance of solutions.
• Design, code, test, debug, and document programs using Agile practices.
• Review and analyze complex data management technologies.
• Assist in production deployments, including troubleshooting and problem resolution.
• Collaborate with various teams to provide strategic solutions and influence data architecture.
• Provide recommendations for the future direction of data management technology.

Requirements

• Strong experience with Python and big data technologies.
• Hands-on experience building data pipelines for Hadoop and Google Cloud.
• Expertise in data management services and end-to-end data pipelines.
• Experience with Agile development practices.
• Ability to analyze complex data management technologies.

Nice-to-haves

• Experience with Google Cloud data services such as Cloud Storage, Cloud Functions, Cloud Dataflow, and BigQuery.
• Familiarity with Kafka/Kafka Connect and Hive DB development.
• Google Cloud Big Data Specialty certification.

Benefits

• Medical, dental & vision insurance
• Critical Illness, Accident, and Hospital insurance
• 401(k) Retirement Plan with pre-tax and Roth contributions
• Life Insurance (Voluntary Life & AD&D)
• Short and long-term disability insurance
• Health Spending Account (HSA)
• Transportation benefits
• Employee Assistance Program
• Paid Time Off (PTO), Vacation or Sick Leave