Data Engineer

Cloudious - Fountain Hills, AZ

Full-time

Fountain Hills, AZ

Administrative and Support Services

About the position

The Data Engineer role focuses on the design, build, and operationalization of data warehouses, data lakes, and analytics platforms on Google Cloud Platform (GCP). The position requires a solid understanding of large-scale data solutions and involves monitoring data lakes, creating scalable data ingestion pipelines, and collaborating with cross-functional teams to meet business needs.

Responsibilities

• Monitor the Data Lake and engage appropriate support teams as needed.
• Design, build, and test scalable data ingestion pipelines.
• Automate the ETL process for various datasets being ingested.
• Extract application telemetry data and structure it for reporting tools like Kafka and Splunk.
• Utilize data cataloging and metadata management tools such as Collibra, Dataplex, and Alation.
• Create reports to monitor usage data for billing and SLA tracking.
• Work with business and cross-functional teams to gather and document requirements.
• Provide support to ensure the availability and performance of ETL/ELT jobs.
• Offer technical assistance and cross-training to business and internal team members.
• Collaborate with business partners for continuous improvement opportunities.

Requirements

• Bachelor's Degree in Computer Science, Information Technology, Engineering, or related field.
• 6 years of experience in Data Engineering with a focus on Data Warehousing and Data Analytics.
• 4 years of experience with a leading public cloud.
• 4 years of experience in designing and building scalable data pipelines for extraction, transformation, and loading.
• 4 years of experience with Python and working knowledge of Notebooks.
• 2 years of experience with Kafka, PubSub, Docker, and Kubernetes.
• 2 years of hands-on experience with GCP Cloud data implementation projects including Dataflow, DataProc, Cloud Composer, Big Query, Cloud Storage, GKE, and Airflow.
• At least 2 years of experience in Data Governance and Metadata Management.
• Ability to work independently and solve problems while updating stakeholders.
• Strong understanding of relational and dimensional data modeling.
• Experience in DevOps and CI/CD related technologies.
• Excellent written and verbal communication skills, including technical documentation and communication with senior business managers and executives.