Data Engineer

Jade Biz Services - Houston, TX

Full-time

Houston, TX

About the position

We are seeking a skilled Data Engineer to join our team, focusing on building and maintaining scalable data solutions and supporting data infrastructure. This role requires a blend of data engineering, platform engineering, and software engineering skills, with hands-on experience in Databricks and orchestration tools. The Data Engineer will work closely with cross-functional teams to deliver high-quality data solutions in an enterprise environment.

Responsibilities

• Design, develop, and maintain data pipelines using Databricks, ensuring data reliability and scalability.
• Collaborate with platform and software engineering teams to create robust data solutions.
• Implement data integration, transformation, and orchestration processes using tools like Apache Airflow, Azure Data Factory, or equivalent.
• Monitor, troubleshoot, and optimize data workflows to ensure performance and reliability.
• Work with stakeholders to understand business data requirements and translate them into technical solutions.
• Ensure data quality, consistency, and security through effective data governance practices.
• Provide support for data infrastructure and assist in problem-solving when needed.

Requirements

• Bachelor's degree in Computer Science, Data Engineering, or a related field.
• 3+ years of experience in data engineering or related roles.
• Hands-on experience with Databricks and Spark for data processing and transformation.
• Proficiency in data orchestration tools such as Apache Airflow, Azure Data Factory, or similar.
• Strong SQL skills and experience with Python or Scala for data manipulation and pipeline development.
• Experience with cloud platforms such as Azure, AWS, or Google Cloud Platform.
• Familiarity with software engineering practices like version control, CI/CD, and code reviews.
• Strong problem-solving skills and the ability to work collaboratively with cross-functional teams.

Nice-to-haves

• Experience with containerization tools such as Docker or Kubernetes.
• Knowledge of data warehousing solutions and BI tools.
• Understanding of data governance, security, and compliance best practices.