Data Engineer

Ark Infotech Spectrum - Atlanta, GA

Full-time

Atlanta, GA

Professional, Scientific, and Technical Services

About the position

The Data Engineer role focuses on developing and maintaining big data processing systems utilizing technologies such as Apache Spark, Airflow, Databricks, and AWS. The position requires proficiency in programming languages and familiarity with CI/CD and ETL pipelines, aiming to enhance data processing capabilities within the organization.

Responsibilities

• Develop big data processing systems using Apache Spark, Airflow, Databricks, and AWS.
• Implement CI/CD pipelines for efficient deployment.
• Design and maintain ETL pipelines for data integration.
• Utilize containerization technologies for application deployment.

Requirements

• Proficiency in one or more programming languages such as Scala (preferred), Java, or Python.
• Experience with big data technologies like Apache Spark and Airflow.
• Familiarity with CI/CD pipelines and ETL processes.
• Knowledge of AWS services related to data engineering.

Nice-to-haves

• Experience using web service APIs.
• Knowledge of microservices and scaling techniques.