Big Data Engineer (Contract W2 Hybrid onsite)

Xoriant - San Francisco, CA

Full-time - Mid Level

San Francisco, CA

Professional, Scientific, and Technical Services

About the position

The Big Data Engineer position is a long-term contract role based in San Francisco, CA, requiring a hybrid work model with three days onsite. The role focuses on hands-on development using Apache Spark and programming languages such as Scala or Python, with an emphasis on database modeling and big data technologies.

Responsibilities

• Develop and implement data processing solutions using Apache Spark.
• Model databases and work with SQL or NoSQL databases.
• Utilize scripting languages like shell or Python for automation tasks.
• Collaborate with teams to integrate data from various sources using orchestration tools like Airflow or Oozie.
• Maintain version control using Git and build tools like Maven.
• Contribute to software development projects alongside data engineering tasks.

Requirements

• Expertise in Apache Spark and proficiency in programming languages such as Scala or Python.
• Experience with distributed computing and big data technologies.
• Strong knowledge of database modeling and working with SQL or NoSQL databases.
• Familiarity with scripting languages like shell or Python.
• Experience with Cloudera is preferred.
• Knowledge of orchestration tools like Airflow or Oozie is a plus.
• Familiarity with table formats like Delta or Iceberg is advantageous.
• Experience with version control systems like Git and build tools like Maven.

Nice-to-haves

• Experience in software development alongside data engineering.
• Familiarity with additional big data tools and technologies.

Benefits

• Long-term contract opportunity
• Hybrid work model (3 days onsite)