AWS Cloud Data Engineer

Shimento - San Francisco, CA

Full-time - Mid Level

San Francisco, CA

Professional, Scientific, and Technical Services

About the position

The AWS Cloud Data Engineer position is a critical role within our organization, focusing on the design, development, and implementation of complex data warehouse and business intelligence solutions. This role requires a deep understanding of various relational database management systems (RDBMS) such as Microsoft SQL Server and Oracle, as well as expertise in SQL development and analysis. The ideal candidate will possess advanced knowledge of data modeling, including the ability to create quality entity-relationship diagrams (ERDs) and understand the benefits and limitations of different data structures in various use cases. In this role, you will leverage AWS Cloud Data Warehousing Technologies, utilizing core AWS services such as S3, DMS, Glue, and Lambda to build and support data warehouse solutions. You will also have development and modeling experience with Amazon Redshift and be proficient in using AWS service APIs, AWS CLI, and SDKs to build applications. A strong understanding of CI/CD pipelines for deploying applications on AWS, as well as the ability to write code for serverless applications while adhering to AWS security best practices, is essential. The position also involves working with AWS Aurora Postgres for database management, AWS Athena for data access, and AWS Quicksight for data analytics. Familiarity with tools such as Alteryx, Collibra, Immuta, and Okta is a plus, as is experience with visualization tools like Tableau. You will be expected to create scripts using Python and work effectively within an Agile team environment, demonstrating a solid understanding of application lifecycle management and the use of containers in the development process. As part of your responsibilities, you will design, develop, modify, test, and automate solutions for data warehouse and business intelligence applications. This includes making architecture recommendations, managing quality, creating metadata and repositories, troubleshooting issues, and tuning warehouse applications. You will also develop transition and implementation plans and recommend changes in development, maintenance, and standards. The role requires advanced analytical abilities, technical skills, and the capacity to provide innovative solutions to meet technical needs and business requirements. Excellent communication skills are necessary to translate complex technical information into non-technical terms for stakeholders at all levels of the organization.

Responsibilities

• Contributes to the design, development, testing, implementation, and review of complex data warehouse and business intelligence solutions.
• Develops all or part of complex data warehouse applications, develops software from established requirements, builds reports and dashboards, plans and coordinates work of lower-level programmers to meet delivery commitments, creates prototypes; offers insight on the feasibility of system designs.
• Contributes to the design of technology infrastructure and configurations, recommends process improvements.
• Reviews complex patches and new versions of data warehouse applications. Implements complex software packages and deploys code.
• Key participant in cross-functional team initiatives and process improvement projects.

Requirements

• 5 to 9+ years of relevant work experience.
• Bachelor's degree or equivalent experience.
• Expert in developing and analyzing complex SQL on a variety of RDBMS (Microsoft SQL Server, Oracle).
• Expert knowledge of data modeling and understanding of different data structures and their benefits and limitations under particular use cases.
• Experience with ETL tools (Informatica).
• Ability to create quality ERDs (entity-relationship diagrams).
• Excellent writing skills for writing user and system documentation.
• AWS Cloud Data Warehousing Technologies experience.
• Experience using core AWS services to build and support data warehouse solutions leveraging AWS architecture best practices (S3, DMS, Glue, Lambda).
• Development/Modeling experience with Amazon Redshift.
• Experience using the AWS service APIs, AWS CLI, and SDKs to build applications.
• Proficiency in developing, deploying, and debugging cloud-based applications using AWS.
• Ability to use a CI/CD pipeline to deploy applications on AWS (GitLab, Terraform, DBMaestro).
• Ability to apply a basic understanding of cloud-native applications to write code.
• Proficiency writing code for serverless applications.
• Ability to write code using AWS security best practices (e.g., not using secret and access keys in the code, instead using IAM roles).
• Ability to author, maintain, and debug code modules on AWS.
• Experience creating scripts with Python.
• Experience working on an Agile team.
• Understanding of application lifecycle management.
• Understanding of the use of containers in the development process.

Nice-to-haves

• Familiarity working with or knowledge of Alteryx, Collibra, Immuta, Okta a plus.
• Experience with visualization tools (Tableau).