Databricks Developer

$57,100 - $134,100/Yr

CGI - Lafayette, LA

Full-time - Mid Level

Hybrid - Lafayette, LA

Professional, Scientific, and Technical Services

About the position

The Databricks Developer role at CGI Technologies and Solutions, Inc. involves designing, developing, and maintaining scalable data pipelines and solutions using Databricks and Apache Spark. This position is critical for supporting client-specific development of cutting-edge BI solutions, requiring collaboration with data scientists and analysts to deliver high-quality data solutions. The role offers the opportunity to work in a dynamic environment with a focus on innovation and performance optimization.

Responsibilities

• Design and develop scalable data pipelines and ETL processes using Databricks and Apache Spark.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
• Optimize and tune data pipelines for performance and scalability.
• Implement data quality checks and validations to ensure data accuracy and consistency.
• Monitor and troubleshoot data pipelines to ensure reliable and timely data delivery.
• Develop and maintain documentation for data pipelines, processes, and solutions.
• Implement best practices for data security, governance, and compliance.
• Participate in code reviews and contribute to the continuous improvement of processes.

Requirements

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• Experience in data engineering or a related field.
• Strong experience with Databricks and Apache Spark.
• Proficiency in programming languages such as Python, Scala, or Java.
• Experience with big data technologies such as Hadoop, Hive, and Kafka.
• Strong SQL skills and experience with relational databases.
• Experience with cloud platforms such as AWS, Azure, or Google Cloud.
• Knowledge of data warehousing concepts and technologies.
• Experience with version control systems such as Git.
• Strong problem-solving skills and attention to detail.
• Excellent communication and collaboration skills.

Nice-to-haves

• Experience with Delta Lake and Databricks Delta.
• Experience with data visualization tools such as Power BI, Tableau, or Looker.
• Knowledge of machine learning and data science concepts.
• Experience with CI/CD pipelines and DevOps practices.
• Certification in Databricks, AWS, Azure, or Google Cloud.

Benefits

• Competitive compensation
• Comprehensive insurance options
• Matching contributions through the 401(k) plan and the share purchase plan
• Paid time off for vacation, holidays, and sick time
• Paid parental leave
• Learning opportunities and tuition assistance
• Wellness and Well-being programs