Big Data Engineer

$100,100 - $157,300/Yr

Dish - Littleton, CO

Full-time - Mid Level

Littleton, CO

Broadcasting and Content Providers

About the position

DISH, an EchoStar company, is seeking a Big Data Engineer to join our innovative Technology team in Littleton, Colorado. This role is pivotal in reimagining the future of connectivity and enhancing our capabilities across various industries. As a Big Data Engineer, you will serve as an evangelist for the data engineering function, leveraging big data processing frameworks to create and optimize data engineering pipelines for analytics projects. You will support data and cloud transformation initiatives, working closely with software engineers and data scientists to contribute to our cloud strategy based on your prior experience. In this position, you will be expected to understand the latest technologies in a rapidly innovative marketplace and work independently with all stakeholders across the organization to deliver enhanced functionality. Your expertise will be crucial in gathering and understanding data requirements, ensuring high-quality data ingestion, and building systems capable of processing and transforming large volumes of data. The ideal candidate will have a strong background in Big Data Engineering/Data Analysis, with a minimum of 6-12 years of experience in the field. You will be responsible for creating and maintaining automated ETL processes, focusing on data flow, error recovery, and exception handling. Your role will also involve working with various data processing frameworks and tools, ensuring that our data engineering practices are efficient and effective.

Responsibilities

• Evangelist for data engineering function leveraging big data processing framework
• Creation and optimization of data engineering pipelines for analytics projects
• Support data and cloud transformation initiatives
• Support software engineers and data scientists
• Contribute to cloud strategy based on prior experience
• Independently work with stakeholders across the organization to deliver enhanced functionality
• Gather and understand data requirements to achieve high quality data ingestion
• Create and maintain automated ETL processes with focus on data flow, error recovery, and exception handling

Requirements

• 6-12 years of experience in Big Data Engineering/Data Analysis
• Experience in Data Warehouse/Big Data background
• Advanced Apache Spark processing framework experience
• Proficiency in spark programming languages such as Scala/Python/Advanced Java
• Sound knowledge in shell scripting
• Experience with Core Spark, Spark Streaming, DataFrame API, Data Set API, RDD APIs & Spark SQL programming
• Advanced SQL experience using Hive/Impala framework including SQL performance tuning
• Experience with ElasticSearch (OpenSearch) and Kibana Dashboards
• Experience in writing spark streaming jobs integrating with streaming frameworks such as Apache Kafka or AWS Kinesis
• Knowledge of resource management frameworks such as standalone Spark, Yarn or Mesos
• Experience in physical table design in Big Data environment
• Experience working with external job schedulers such as autosys, aws data pipeline, airflow
• Experience working in Key/Value data store such as Hbase
• Experience in AWS services such as EMR, Glue, S3, Athena, IAM, Lambda and CloudWatch

Benefits

• Versatile health perks
• Flexible spending accounts
• Health Savings Account (HSA)
• 401(k) Plan with company match
• Employee Stock Purchase Plan (ESPP)
• Career opportunities
• Flexible time away plan