Python/Pyspark Developer

Capgemini - Atlanta, GA

Full-time - Mid Level

Atlanta, GA

10,001+ employees

Professional, Scientific, and Technical Services

About the position

The Python/Pyspark Developer role at Capgemini involves designing, developing, and maintaining ETL processes using PySpark for data transformation and loading into data lakes or warehouses. The position requires strong proficiency in Python and SQL, along with experience in automation and data workflow management. The developer will work in an Agile environment, conducting code reviews, testing, and deploying code while providing support during various testing phases.

Responsibilities

• Design, develop, and maintain ETL processes using PySpark.
• Write efficient and reusable Python scripts for data manipulation, integration, and analysis.
• Develop and implement Shell scripts for automation of data processing tasks.
• Manage and schedule data workflows using Autosys.
• Optimize data processing performance and troubleshoot issues in data pipelines.
• Conduct code reviews, test, and deploy code.
• Provide fixes to issues/bugs reported from test and production environments.
• Support during System Integration Testing (SIT) and User Acceptance Testing (UAT).
• Fix defects raised in production.

Requirements

• 6+ years of experience in designing, developing, and maintaining ETL processes using PySpark.
• 6+ years of experience in writing efficient and reusable Python scripts.
• Experience in developing and implementing Shell scripts for automation.
• Strong proficiency in SQL and experience with relational databases (e.g., MySQL, SQL Server).
• Hands-on experience in optimizing data processing performance.
• Knowledge of CI/CD Agile tools and source control.

Nice-to-haves

• Experience working with Agile methodology.
• Experience in mentoring and coaching team members.

Benefits

• 401(k)
• Adoption assistance
• Dental insurance
• Paid holidays
• Paid parental leave
• Paid time off