Sr Data Engineer

Apple - San Diego, CA

Full-time - Mid Level

San Diego, CA

Computer and Electronic Product Manufacturing

About the position

Apple's Software Engineering group is renowned for its work in user experience, integration, functionality, and lifestyle. We are responsible for developing and delivering every state-of-the-art operating system, application, and firmware to Apple customers. This role is critical to Apple's ecosystem and continues to break new ground every day! The Software Engineering team leverages compute resources for a variety of workloads to develop every piece of software. We are looking for a data engineer to bring better visibility to these workloads, partner with engineering teams to drive improvements in their efficiency, and ensure that our costs remain as low as possible. If you're interested in driving efficiency that enables Apple to develop software at a global scale, then this is the place for you!

Responsibilities

• Develop and maintain Tableau dashboards and other analytical tools to provide visibility into compute resourcing.
• Collaborate with engineering teams across Software Engineering to leverage shared compute resources wherever possible.
• Monitor service expenditures, alert teams of abnormal or inefficient usage, and advise them on improving efficiency.
• Automate the data flow between systems such that a real-time view of utilization is visible at all times.

Requirements

• Bachelor's or Master's degree in Computer Science, Information Systems, Software Engineering, Data Science, or a related field.
• 5+ years experience in architecting, designing, and developing large scale data solutions.
• Deep understanding and strong development experience with distributed data processing frameworks such as Hadoop, Spark and others.
• 5+ years experience in building and maintaining large-scale ETL/ELT pipelines (batching and/or streaming) that are optimized for performance and can handle data from various sources, structured or unstructured.
• Experienced in writing and maintaining high-quality code using standard methodologies such as code reviews, unit testing, and continuous integration.

Nice-to-haves

• Experience with monitoring and alerting (e.g. Honeycomb, Splunk, Grafana, or similar).
• Familiarity with Kubernetes and orchestration/containerization technologies.
• Expertise in Python, Swift, Unix Shell scripting and Dependency driven job schedulers.
• Stay up-to-date with the latest trends and technologies in data infrastructure, architecture, big data analytics, and apply them to improve the system.
• Familiarity with other related fields, such as data science, machine learning, and artificial intelligence, to design solutions that can accommodate advanced analytics.
• Collaborative mindset to work with various teams, including software engineers, financial analysts, and site reliability engineers.
• Excellent design and development experience with SQL and NoSQL database, OLTP and OLAP databases.