Data Engineer

Transcend It Solutions - Quincy, MA

Full-time - Mid Level

Quincy, MA

Professional, Scientific, and Technical Services

About the position

The Sr. Data Engineer position within the CASM Platform Product & Engineering team focuses on building robust and scalable Cyber AI applications to manage vulnerabilities, risks, and security configurations across the bank. The role requires an experienced data engineer with a strong background in cybersecurity, cloud-native tools, and a passion for automation and resilience. The engineer will work in Agile cross-functional teams to enhance the CASM platform, providing secure and insightful visibility into network assets and risks.

Responsibilities

• Integrate cybersecurity data sources and build data APIs for real-time insights and streamlined access.
• Design and optimize large-scale ETL/ELT pipelines on Databricks using Python and PySpark.
• Implement automated data checks, manage data lineage, and ensure compliance with cybersecurity standards.
• Create data models, visualizations, and dashboards for threat and vulnerability insights using Databricks and React.js.
• Develop secure, scalable environments on AWS (S3, ELB, Lambda) for data storage and processing.
• Use CI/CD pipelines to automate testing and deployment in alignment with Agile practices.
• Deploy ML models for threat detection, anomaly detection, and risk scoring.
• Guide junior engineers, establish best practices, and foster a high-performance, innovative culture.

Requirements

• B.S., M.S., or Ph.D. in Computer Science, Data Science, Information Systems, or a related field, or equivalent professional experience.
• 8+ years in data engineering with strong skills in Python, PySpark, SQL, and extensive experience with Databricks and big data frameworks.
• Advanced proficiency in AWS, including EC2, S3, Lambda, ELB, and container orchestration (Docker, Kubernetes).
• Proven experience implementing SCAS, SAST, DAST/WAS, and secure DevOps practices within an SDLC framework.
• Demonstrated ability to design and implement complex data architectures, including data lakes, data warehouses, and lake house solutions.
• Hands-on experience with automated data quality checks, data lineage, and governance standards.
• Proficiency with analytics and visualization tools such as Databricks, Power BI, and Tableau.
• Experience building CI/CD pipelines that automate testing, security scans, and deployment processes.
• Deep experience in Agile/Scrum environments.

Nice-to-haves

• Expertise in designing data models for cybersecurity data analytics, emphasizing data lineage, federation, governance, and compliance.
• Experience deploying ML algorithms, predictive models, and anomaly detection frameworks.
• Background in mentoring engineers in data engineering best practices.