Sr. Data Engineer

Genesis10 - Atlanta, GA

Full-time - Mid Level

Atlanta, GA

Professional, Scientific, and Technical Services

About the position

Genesis10 is currently seeking a Senior Data Engineer Consultant for our client in the transportation industry, located in Atlanta, GA. This is a 12+ month contract position that starts on September 23, 2024, and ends on September 22, 2025. The client is on a transformative journey to become the best IT organization in the airline industry, aiming to innovate and create meaningful solutions. As a Senior Data Engineer, you will play a crucial role in this transformation by designing, implementing, and maintaining Teradata databases, optimizing queries for performance, and developing ETL processes for data integration and transformation. In this role, you will utilize tools such as Teradata Utilities (BTEQ, FastLoad, MultiLoad, TPT, etc.) and work closely with stakeholders to understand data requirements and design effective solutions. You will be responsible for identifying and resolving performance bottlenecks, ensuring compliance with data governance policies, and implementing data security best practices. Additionally, you will manage the migration of data from on-premises Teradata environments to AWS cloud-based instances, ensuring seamless integration with AWS services like S3, Redshift, RDS, and Lambda. Your expertise in AWS will be essential as you design and deploy cloud infrastructure to support Teradata environments, automate infrastructure provisioning, and implement networking and security best practices. You will also leverage AWS data services to enhance Teradata workloads and ensure compliance with relevant data governance policies, including encryption standards. This position requires a strong collaboration with data architects, analysts, and other stakeholders to deliver effective data solutions that meet business needs.

Responsibilities

• Design, implement, and maintain Teradata databases
• Optimize and tune queries for performance
• Monitor and troubleshoot database issues
• Develop and maintain ETL (Extract, Transform, Load) processes for data integration and transformation
• Use tools like Teradata Utilities (e.g., BTEQ, FastLoad, MultiLoad, TPT, etc.)
• Create and maintain data models to support business processes
• Work with stakeholders to understand data requirements and design solutions
• Identify and resolve performance bottlenecks in SQL queries, system configuration, or data design
• Regularly monitor and optimize system performance
• Implement data security best practices
• Ensure compliance with data governance policies and regulations
• Implement and manage Teradata on AWS infrastructure, leveraging AWS services for scalability and performance
• Manage data migration from on-premises Teradata environments to AWS cloud-based Teradata instances
• Ensure seamless integration between Teradata and AWS services like S3, Redshift, RDS, and Lambda
• Design and deploy cloud infrastructure (VPCs, EC2 instances, S3 buckets) to support Teradata environments
• Implement networking and security best practices in AWS to ensure secure data access and storage
• Automate infrastructure provisioning, scaling, and monitoring using AWS tools like CloudFormation, Terraform, or AWS CLI
• Use AWS Lambda for automating ETL processes or handling specific workloads
• Leverage AWS data services (e.g., Redshift, Athena, Glue) to complement Teradata workloads
• Implement data pipelines using AWS services for ETL processes, integrating data from various sources into Teradata
• Implement AWS security best practices, such as IAM roles and policies, to secure Teradata environments in the cloud
• Ensure compliance with relevant data governance policies, including encryption at rest and in transit
• Work closely with data architects, analysts, and other stakeholders to understand data needs and deliver solutions

Requirements

• In-depth knowledge of Teradata architecture and components (e.g., AMP, Nodes, Parsing Engines, etc.)
• Proficiency in Teradata SQL and Teradata utilities
• Strong SQL skills for querying and managing relational databases
• Scripting experience (e.g., Shell, Python) for automating tasks
• Experience with ETL tools and technologies used in Teradata environments
• Knowledge of query optimization techniques, indexing, partitioning, and workload management in Teradata
• Strong understanding of data warehousing principles, including star/snowflake schemas, OLAP, and ETL processes
• Proficiency with core AWS services: EC2, S3, RDS, Lambda, CloudWatch, CloudFormation, etc.
• Experience with AWS networking (VPC, subnets, security groups, NACLs) and security management
• Familiarity with AWS storage options, such as S3, EFS, and Glacier, in the context of large-scale data storage and retrieval
• Experience deploying and managing Teradata in AWS cloud environments
• Knowledge of AWS services that enhance Teradata performance, such as using S3 for staging data or integrating with Redshift for analytics
• Hands-on experience with AWS CloudFormation, Terraform, or similar tools for automating infrastructure deployment
• Proficiency in automating ETL pipelines and data processing workflows using AWS Lambda, Step Functions, or AWS Glue
• Understanding of encryption and security protocols for data stored and processed in AWS
• Experience implementing compliance standards such as GDPR, HIPAA, or other data protection regulations in cloud environments

Benefits

• Access to hundreds of clients, most who have been working with Genesis10 for 5-20+ years
• The opportunity to have a career-home in Genesis10; many of our consultants have been working exclusively with Genesis10 for years
• Access to an experienced, caring recruiting team (more than 7 years of experience, on average)
• Behavioral Health Platform
• Medical, Dental, Vision
• Health Savings Account
• Voluntary Hospital Indemnity (Critical Illness & Accident)
• Voluntary Term Life Insurance
• 401K
• Sick Pay (for applicable states/municipalities)
• Commuter Benefits (Dallas, NYC, SF)
• Remote opportunities available