Databricks Developer

$57,100 - $134,100/Yr

CGI - Arlington, VA

Full-time - Entry Level

Hybrid - Arlington, VA

Professional, Scientific, and Technical Services

About the position

The Databricks Developer role is a full-time position focused on designing, developing, and maintaining scalable data pipelines and solutions using Databricks and Apache Spark. This position is critical for supporting client-specific development of a cutting-edge BI solution, offering the opportunity to work in a dynamic team environment and contribute to high-stakes projects. The ideal candidate will possess a strong background in data engineering and big data technologies, and will collaborate with various stakeholders to deliver innovative data solutions.

Responsibilities

• Design and develop scalable data pipelines and ETL processes using Databricks and Apache Spark.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
• Optimize and tune data pipelines for performance and scalability.
• Implement data quality checks and validations to ensure data accuracy and consistency.
• Monitor and troubleshoot data pipelines to ensure reliable and timely data delivery.
• Develop and maintain documentation for data pipelines, processes, and solutions.
• Implement best practices for data security, governance, and compliance.
• Participate in code reviews and contribute to the continuous improvement of processes.

Requirements

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• Experience in data engineering or a related field.
• Strong experience with Databricks and Apache Spark.
• Proficiency in programming languages such as Python, Scala, or Java.
• Experience with big data technologies such as Hadoop, Hive, and Kafka.
• Strong SQL skills and experience with relational databases.
• Experience with cloud platforms such as AWS, Azure, or Google Cloud.
• Knowledge of data warehousing concepts and technologies.
• Experience with version control systems such as Git.
• Strong problem-solving skills and attention to detail.
• Excellent communication and collaboration skills.

Nice-to-haves

• Experience with Delta Lake and Databricks Delta.
• Experience with data visualization tools such as Power BI, Tableau, or Looker.
• Knowledge of machine learning and data science concepts.
• Experience with CI/CD pipelines and DevOps practices.
• Certification in Databricks, AWS, Azure, or Google Cloud.

Benefits

• Competitive compensation
• Comprehensive insurance options
• Matching contributions through the 401(k) plan and the share purchase plan
• Paid time off for vacation, holidays, and sick time
• Paid parental leave
• Learning opportunities and tuition assistance
• Wellness and Well-being programs