Databricks Developer

Dcm Infotech Limited - Morris Plains, NJ

Full-time

Morris Plains, NJ

About the position

The Databricks Developer role focuses on leveraging Databricks and related technologies to develop and optimize data processing solutions. The position requires expertise in Spark and its various components, as well as experience with cloud services and DevOps practices.

Responsibilities

• Develop and optimize data processing solutions using Databricks.
• Utilize Spark RDD, Data Frames, and SparkSQL for data manipulation and analysis.
• Implement data streaming solutions with Spark Streaming.
• Apply machine learning techniques using Spark ML.
• Integrate and manage data workflows with Snowflake and AWS Cloud services.
• Collaborate with DevOps teams to implement CI/CD pipelines using Jenkins and Terraform.

Requirements

• Proficiency in Spark RDD, Data Frames, and SparkSQL.
• Experience with PySpark and Spark Streaming.
• Knowledge of Spark ML for machine learning applications.
• Familiarity with Databricks platform.
• Experience with Snowflake for data warehousing solutions.
• Understanding of AWS Cloud services and DevOps practices.