Sr Big Data Engineer Airflow and Oozie (GCP)

$116,100 - $170,280/Yr

Rackspace Technology - San Antonio, TX

Full-time - Mid Level

Remote - San Antonio, TX

Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services

About the position

The Senior Big Data Engineer role at Rackspace involves developing and managing batch processing systems using technologies such as Oozie, Airflow, and the Apache Hadoop ecosystem, with a strong emphasis on GCP. The position requires a skilled engineer who can work independently in a remote environment, solve complex problems, and implement automation and DevOps best practices.

Responsibilities

• Develop scalable and robust code for batch processing systems using Hadoop, Oozie, Pig, Hive, Map Reduce, Spark (Java), Python, and Hbase.
• Manage and optimize data workflows using Oozie and Airflow within the Apache Hadoop ecosystem.
• Leverage GCP for scalable big data processing and storage solutions.
• Implement automation and DevOps best practices for CI/CD and Infrastructure as Code (IaC).

Requirements

• Bachelor's degree in Computer Science, software engineering, or a related field.
• 5+ years of experience in customer-facing software/technology or consulting.
• 5+ years of experience with "on-premises to cloud" migrations or IT transformations.
• 5+ years of experience building and operating solutions on GCP (ideally) or AWS/Azure.
• Proficiency in Oozie, Airflow, Map Reduce, and Java.
• Strong programming skills in Java (specifically Spark), Python, Pig, and SQL.
• Expertise in public cloud services, particularly GCP.
• Proficiency in the Apache Hadoop ecosystem with Oozie, Pig, Hive, and Map Reduce.
• Familiarity with BigTable and Redis.
• Experience in Infrastructure and Applied DevOps principles.

Nice-to-haves

• Experience with Terraform for CI/CD and IaC.
• Familiarity with additional cloud services like AWS or Azure.

Benefits

• Health insurance
• 401k plan
• Paid holidays
• Flexible scheduling
• Professional development opportunities