Principal Engineer - Site Reliability

About the position

The Aviatrix SRE team is a small but highly skilled global group of Systems Engineers/SREs dedicated to ensuring the reliability, availability, and performance of Aviatrix’s critical systems and services. Our mission is to build and maintain a robust, resilient infrastructure that enables Aviatrix to deliver high-quality services with agility through automation, best practices, and a culture of operational excellence. As an SRE – Principal Engineer, you’ll play a key role in designing, implementing, and maintaining highly available, fault-tolerant, and scalable systems. You’ll focus on automation, proactive monitoring, and Infrastructure-as-Code (IaC) to drive efficiency and reliability across our services.

Responsibilities

• Ensure uptime for crucial services and systems based on business required SLOs.
• Minimize service disruptions through proactive monitoring, capacity planning and fault-tolerant design.
• Design and architect complex, scalable and reliable systems.
• Develop and implement automation tools and frameworks to automate routine tasks to reduce human error.
• Streamline and improve operational processes to increase efficiency.
• Define, build, deploy, maintain, and extend observability and monitoring tools.
• Maintain an effective on-call rotation to ensure 24/7 coverage.
• Respond to incident response procedures to swiftly address and mitigate service disruptions.
• Help define and monitor Service level Indicators (SLIs) and Service Level Objectives.
• Work closely with product engineering to ensure service-level objectives and reliability targets are met.
• Respond to escalations by troubleshooting complex system and application incidents.
• Perform root cause analysis and implement necessary corrective actions.
• Stay up to date with latest industry trends and emerging technologies.

Requirements

• 8+ years of experience maintaining and deploying highly available, fault-tolerant systems at scale.
• Proficiency in Golang or Python is required.
• Deep understanding of Terraform core components with real-world experience using Terraform for infrastructure provisioning and management.
• At least one cloud service provider experience (e.g., AWS, GCP, Azure, OCI).
• Good knowledge with Kubernetes.
• Solid experience developing Automation tools and frameworks.
• Experience with Logging Solutions (e.g., Loki, Syslog, Elasticsearch, Logstash, Kibana, Filebeat, Fluentbit, etc.).
• Experience with Monitoring and Metrics Solutions (e.g., Prometheus, Grafana, Victoria Metrics).
• Practical experience with Linux system administration.
• Experience with Version control system (e.g., Git, GitHub) and code review.
• Excellent communication skills are required.

Nice-to-haves

• Experience with Terragrunt.
• Knowledge of cdk8s and operators.

Benefits

• 100% of employee premiums and 88% of dependent(s) premiums for medical, dental and vision coverage.
• 401(k) match.
• Short and long-term disability.
• Life/AD&D insurance.
• $1,000/year education reimbursement.
• Flexible vacation policy.