GCP Data Engineer

Tata Consultancy Services - Woonsocket, RI

Full-time

Woonsocket, RI

Professional, Scientific, and Technical Services

About the position

The GCP Data Engineer position at Tata Consultancy Services Limited involves working with Google Cloud architecture and technologies to design, implement, and manage cloud infrastructure. The role requires a deep understanding of database and data warehouse technologies, particularly those offered by Google, such as Big Query, Cloud SQL, Spanner, and Big Table. The engineer will be responsible for building the infrastructure necessary for optimal extraction, transformation, and loading (ETL) of data from various sources using SQL and Google Cloud's big data technologies. This includes constructing data pipelines utilizing tools like Data Proc Cluster, Data Flow, Pub/Sub, Cloud Composer (Airflow), and Cloud Functions. In addition to technical skills, the position demands fluency in object-oriented programming languages, preferably Python, and a strong grasp of distributed systems architecture. The engineer will also engage in data quality processes, which encompass data cleansing, audits, alerts, triage mechanisms, and ensuring referential integrity. Familiarity with CI/CD processes, including release and deployment using Google Cloud Build and Git flow, is essential. The role also requires experience in cloud management platforms, virtualization strategies, and automation and orchestration strategies, with a focus on optimizing cloud performance and capacity. The ideal candidate will have a comprehensive understanding of cloud best practices to minimize operational costs while maintaining high service level objectives (SLOs), such as five nines availability. Knowledge of cloud container services like GKE, storage services, and database services is crucial, as is experience in managing projects related to highly virtualized solutions, digital convergence, and security. The engineer will also need to demonstrate a deep understanding of GCP's AI/ML services and be capable of identifying and solving relevant use cases across the enterprise. Furthermore, the role involves defining cloud adoption strategies and transformation roadmaps for large-scale enterprise clients, ensuring that the cloud infrastructure aligns with business objectives.

Responsibilities

• Design, architect, implement, and manage cloud infrastructure on Google Cloud Platform.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from various sources using SQL and Google Cloud technologies.
• Construct data pipelines using Data Proc Cluster, Data Flow, Pub/Sub, Cloud Composer (Airflow), and Cloud Functions.
• Engage in data quality processes including data cleansing, audits, alerts, and ensuring referential integrity.
• Utilize CI/CD processes for release and deployment with Google Cloud Build and Git flow.
• Develop enterprise cloud applications and manage projects related to highly virtualized solutions and digital convergence.
• Identify and solve AI/ML use cases across the enterprise using GCP's AI/ML services.
• Define cloud adoption strategies and transformation roadmaps for large-scale enterprise clients.

Requirements

• 4+ years' experience architecting with GCP Database Technologies.
• 2+ years' hands-on experience on GCP.
• Deep knowledge of cloud best practices to reduce operational costs and maintain high SLOs.
• Working knowledge of cloud container services (GKE), storage services (Cloud Storage), and database services (Spanner, Big Query).
• Experience in developing enterprise cloud applications and managing projects related to digital convergence and security.

Nice-to-haves

• Certification like Google Cloud Professional Cloud Architect or Google Professional Data Engineer would be a plus.
• Experience with cloud management platforms and automation strategies.
• Understanding of microservices patterns and hands-on experience with microservices.