Lead Data Engineer

Alcority - Dallas, TX

Full-time - Mid Level

Dallas, TX

About the position

The Lead Data Engineer will play a crucial role in designing and developing robust data pipelines and scalable data architectures to support data-driven initiatives within the organization. This position requires collaboration with cross-functional teams to ensure data integrity, availability, and accessibility, while also providing technical leadership and mentoring to junior engineers.

Responsibilities

• Lead the design, development, and maintenance of scalable data pipelines, ETL/ELT processes, and data models.
• Focus on Kafka, HDFS, HBase, Parquet, Spark, and Flink to ensure efficient data ingestion, storage, and processing.
• Build and manage data ingestion frameworks using big data technologies, ensuring high efficiency and reliability.
• Develop and optimize data storage solutions, including data warehousing, data lakes, and NoSQL databases on cloud platforms such as AWS, Azure, or Google Cloud.
• Collaborate with data scientists, analysts, and business stakeholders to understand data needs and deliver solutions that meet those needs.
• Ensure data quality, governance, and security through the implementation of best practices and robust monitoring systems.
• Work with various data formats, including Parquet, Avro, and others, to ensure efficient data processing and storage.
• Mentor and provide technical leadership to junior data engineers, fostering a culture of continuous learning and improvement.

Requirements

• Bachelor's degree in Computer Science, Engineering, or a related field; Master's degree preferred.
• 3+ years of proven experience as a Data Engineer or in a similar role, with a strong focus on data warehousing, ETL/ELT processes, and data modeling.
• Proficiency in programming languages such as Python, Java, or Scala.
• Hands-on experience with big data technologies such as Kafka, Spark, HDFS, HBase, Flink, DBT, SQLMesh, etc.
• Experience with cloud platforms such as AWS, Azure, or Google Cloud.
• Strong SQL skills and experience working with both relational and NoSQL databases.
• Familiarity with data visualization tools such as Tableau, Power BI, or Looker.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration skills, with the ability to work effectively in a team environment.

Nice-to-haves

• Experience with building and managing data pipelines in a multi-cloud environment.
• Familiarity with data governance and data quality frameworks.
• Experience in mentoring or leading a team of data engineers.

Benefits

• 25 days of PTO for full-time employees and 12 company holidays.
• Life insurance, Short-term disability, Long-term disability, Paid parental leave, Employee Assistance Program, and medical insurance in our high deductible health plan.
• Optional medical insurance in our EPO plan, Dental benefits, and Vision benefits.
• Health Savings Accounts, Flexible Spending Accounts, Supplemental Life insurance, and more.
• 401(k) eligible after 60 days with a discretionary company match of 50% up to the first 6% of contributions.