Cloud Data Engineer

Cloudious - San Francisco, CA

Full-time - Mid Level

Remote - San Francisco, CA

Administrative and Support Services

About the position

We are seeking an experienced Cloud Data Engineer with 10 to 14 years of experience to join our dynamic team. The ideal candidate will have strong technical skills in PySpark, Palantir, AWS Cloud, AWS Big Data, and AWS Batch. This hybrid role requires proficiency in English and offers a day shift schedule. The Architect will play a crucial role in designing and implementing innovative solutions that drive our company's success.

Responsibilities

• Design and implement scalable data processing solutions using PySpark and AWS Big Data technologies.
• Oversee the integration of Palantir into our data architecture to enhance data analysis and decision-making capabilities.
• Provide expertise in AWS Cloud services to optimize performance, security, and cost-efficiency.
• Develop and manage AWS Batch workflows to ensure efficient data processing and job scheduling.
• Collaborate with cross-functional teams to gather requirements and translate them into technical solutions.
• Ensure data quality and integrity through robust data validation and cleansing processes.
• Monitor and troubleshoot data pipelines to maintain seamless data flow and minimize downtime.
• Implement best practices for data governance and compliance with industry standards.
• Mentor and guide junior team members to foster a culture of continuous learning and improvement.
• Conduct performance tuning and optimization of data processing jobs to achieve optimal efficiency.
• Stay updated with the latest advancements in data technologies and incorporate them into our architecture.
• Provide technical support and troubleshooting for data-related issues to ensure timely resolution.
• Prepare and present technical documentation and reports to stakeholders for transparency and informed decision-making.

Requirements

• Possess a strong background in PySpark, Palantir, AWS Cloud, AWS Big Data, and AWS Batch.
• Demonstrate excellent problem-solving skills and the ability to work independently and collaboratively.
• Exhibit proficiency in English with strong communication skills for effective interaction with team members and stakeholders.
• Show a proven track record of designing and implementing complex data solutions in a hybrid work environment.
• Have a deep understanding of data governance, security, and compliance best practices.
• Display the ability to mentor and guide junior team members to achieve project goals.
• Maintain a proactive approach to staying updated with the latest industry trends and technologies.