DevOps and SRE Interview Guide: The Full Reliability Engineer Loop

TL;DR

• DevOps and SRE loops simulate the actual job: a system just broke, the information is incomplete, and the clock is running.
• They reward operational maturity (mitigation before root cause, blameless postmortems, SLO-driven alerting) more than algorithmic depth.
• Walk Linux troubleshooting layer by layer, narrate networking from DNS to HTTP/3, drive incident scenarios with assess-contain-communicate-diagnose-recover-learn, and ground every design choice in the error-budget lens.

Introduction

DevOps and Site Reliability Engineer loops are some of the most unusual interviews in tech. They simulate what your actual job would feel like: something just broke, you have incomplete information, and the clock is running. The strongest candidates do not memorize answers. They develop a muscle for reasoning under ambiguity, with systems they cannot see, using tools they half-remember.

This guide is written for engineers with real infrastructure scars. If you have ever been paged at three in the morning for a full disk on a log-forwarder, you already have the right instinct. The goal here is to turn that instinct into an interview performance, covering every round you are likely to see and the signals that matter in each.

Table of Contents

1. What Makes DevOps and SRE Loops Different
2. Linux and Troubleshooting Round
3. Networking Deep Dive
4. Incident Response Drill
5. Infrastructure as Code Round
6. On-Call Readiness and Operational Judgement
7. Scripting and Automation Round
8. Systems Design for SREs
9. Sample Questions with Full Walkthroughs
10. Frameworks for Structured Answers
11. Common Mistakes That Tank Otherwise-Strong Candidates
12. Study Plans
13. FAQ
14. Conclusion

What Makes DevOps and SRE Loops Different

Most software engineering loops are primarily about code. DevOps and SRE loops are primarily about systems, and the code they contain is usually glue code or tooling, not feature code. You will be judged on whether you can reason through a production incident, design a reliable deployment pipeline, explain what happens when a packet leaves your laptop, and write a script that behaves well in the presence of partial failure.

The calibration is also different. At most companies, SREs are evaluated on a rubric that leans heavily on operational maturity: do you understand blameless post-mortems, do you know when to roll back instead of root-cause first, do you understand the difference between reliability, availability, and correctness, and do you know how to prioritize work against SLOs and error budgets. A candidate who is technically brilliant but lacks operational maturity will often fail at senior levels.

The final distinguishing feature is breadth. An SRE is expected to be competent in Linux internals, networking from layer two to layer seven, at least one cloud provider in depth, at least one configuration language like Terraform or Pulumi, at least one scripting language usually Python or Go, container orchestration typically Kubernetes, observability tooling, and incident management. Nobody is world-class in all of these, but you need to avoid embarrassing gaps in any of them.

Linux and Troubleshooting Round

The Linux round is almost always a live troubleshooting simulation. The interviewer describes a symptom: "A service is returning 502s intermittently" or "disk is full on host X" or "this process is pinned at 100 percent CPU," and you are expected to reason your way to root cause, narrating every command you would run and what you would look for in the output.

Strong candidates structure their troubleshooting around a clear mental model. A useful one is the four layers of a running system: the process, the OS, the host, and the network. For any symptom, you walk the layers and confirm or rule out hypotheses at each. For a 502, you start at the process, check logs for application errors, then move to the OS to check file descriptors and memory, then to the host to check CPU and disk, then to the network to check connectivity to downstream services.

Commands you must be able to use without hesitation include ps, top, and htop for process state, vmstat and iostat for host health, ss and netstat for socket state, tcpdump for packet inspection, strace for syscall tracing, lsof for open files, dmesg for kernel messages, journalctl for systemd-managed services, and the standard /proc filesystem for kernel-level inspection. You should know which of these have replacements in modern tooling, for example replacing netstat with ss and ifconfig with ip, but be fluent in both.

The most common live scenarios are high CPU or high memory with no obvious cause, intermittent latency spikes, a process that will not die cleanly, disk pressure from log rotation misconfiguration, zombie processes, and connection pool exhaustion. Practice each of these on a local VM until you can narrate the investigation in your sleep.

A subtle signal interviewers look for is whether you measure before guessing. A candidate who reaches for strace after the first symptom has not formed a hypothesis and is flailing. A candidate who says "before I run strace I want to check whether this is a CPU or an IO problem by looking at vmstat" is showing judgment.

Networking Deep Dive

The networking round is where many otherwise-strong candidates collapse, because modern engineers often work several layers above the wire and have let networking rust. The bar is not that you remember every TCP flag. The bar is that you can walk through what happens when a user types a URL into a browser, and name at least one thing that can go wrong at every layer.

The canonical question is "explain what happens when you type a URL into a browser and hit enter, as far down as you can go." The answer should cover DNS resolution including caching and TTLs, TCP connection setup including the three-way handshake and TLS negotiation, HTTP request and response including keep-alives and HTTP/2 or HTTP/3 multiplexing, server-side routing through load balancers and reverse proxies, and any layers beyond that depending on the application. Interviewers will stop you and drill into whichever layer they are curious about, so have depth everywhere.

Other recurring topics include the difference between layer four and layer seven load balancing, how TLS termination works and where certificates live, how to debug a DNS issue, the difference between Unix domain sockets and TCP sockets, MTU and fragmentation, how NAT breaks end-to-end assumptions, the role of BGP in wide-area routing, and at least a basic understanding of the routing table and subnetting. If you work in a cloud environment, know how VPCs, subnets, route tables, and security groups interact, because most real-world production issues live there.

A specific drill worth doing: pick a symptom like "requests from service A to service B fail intermittently with connection reset" and enumerate every possible cause from the kernel up. A strong candidate can list at least ten causes, from SYN cookie exhaustion through middlebox RST injection to certificate rotation timing.

Incident Response Drill

Many companies run an explicit incident response round, especially for senior SRE candidates. You will be handed a scenario mid-incident: "It is two in the morning. Your primary database is returning high-latency errors and the on-call playbook is out of date. Walk me through what you do in the first ten minutes." The interviewer will play the role of a noisy incident channel, occasionally injecting new information or asking what you would do next.

The strongest candidates work from a disciplined incident framework rather than improvising. A useful one is assess, contain, communicate, diagnose, recover, learn. In the first minute you assess the blast radius: who is affected, what surfaces are down, how bad is it. You start communications immediately, posting a short status to the incident channel with what you know and do not know. You contain the damage if possible, for example by failing over or rolling back a recent deploy. Only after containment do you move into deeper diagnosis.

Interviewers are specifically listening for a few markers. First, do you declare and escalate appropriately, or do you try to hero it alone. Second, do you separate mitigation from root cause, or do you chase the cause while the fire spreads. Third, do you communicate at each step, or do you go silent for five minutes while you investigate. Fourth, after the scenario is resolved, do you treat the post-mortem as a blameless learning exercise or do you reach for a culprit.

The post-mortem discussion is often the highest-signal part of the round. Candidates who can articulate contributing factors, distinguish them from root causes, propose concrete action items tied to specific failure modes, and push back gently on any implied blame are calibrated at senior or staff level.

Infrastructure as Code Round

The IaC round usually combines a conceptual discussion with a small live exercise. Conceptually, expect questions about the trade-offs between declarative and imperative provisioning, how state management works in Terraform, why you might choose Pulumi or CDK over Terraform, how to organize modules for reuse, how to structure environments for blast radius isolation, and how to handle secrets in infrastructure code.

The live exercise is usually "write a Terraform module that provisions X," where X is a load balancer, a managed database, a Kubernetes cluster, or a serverless function. You will be judged less on perfect syntax and more on structure: is the module properly parameterized, does it expose clean outputs, does it handle lifecycle concerns like create-before-destroy, does it avoid hardcoding environment-specific values.

Topics you should be ready to discuss in depth include the state locking problem and how to solve it with a remote backend, the difference between modules and workspaces, strategies for refactoring existing Terraform without destroying resources, the role of policy-as-code tools like OPA or Sentinel, and the specific pitfalls of your primary cloud provider. An Azure-heavy candidate who has never touched AWS can still pass an AWS shop interview by demonstrating strong IaC fundamentals and admitting the specific knowledge gap rather than bluffing.

A common trap is candidates who conflate GitOps with IaC. IaC is about how infrastructure is described. GitOps is about how changes are promoted. Being crisp about both and their interaction, including pull-based deployment systems like Argo CD or Flux and how they relate to your Terraform pipeline, signals maturity.

On-Call Readiness and Operational Judgement

On-call readiness questions are almost always behavioral, and they are testing whether you would be a trustworthy on-call partner. Expect questions like "tell me about the worst incident you were part of," "how do you decide when to wake someone up," "how do you handle alert fatigue," and "what would you change about your current on-call rotation." These are not gotcha questions. They are genuine attempts to figure out if you think about operability carefully.

Great answers share a few patterns. They distinguish between operational load and toil, and they have concrete examples of reducing toil. They talk about alerts as a design problem: every alert should be actionable, every alert should have a runbook, and every alert that fires too often is a bug. They talk about error budgets as a way to negotiate between reliability investment and feature investment with product partners. They acknowledge that on-call is a shared responsibility and that senior SREs should be making the rotation easier for their juniors over time.

A specific question worth preparing for is "what are your SLO thoughts for a service you have not seen before." A weak answer picks a percentile number and moves on. A strong answer talks about identifying user journeys, defining SLIs that actually reflect user experience, setting SLOs conservatively based on historical data and user expectations, and using error budgets to drive behavior on the team.

Scripting and Automation Round

Almost every SRE loop has a scripting round, usually in Python, Bash, or Go. The scope is rarely a pure algorithms question. It is almost always a realistic automation task: parse a log file and compute error rates per endpoint, implement a retry with exponential backoff, write a health-check script that handles partial failures, or implement a simple reconciler loop.

The bar is idiomatic, defensive scripting. Does your code handle empty input, malformed lines, missing environment variables, and interrupted execution. Does it fail loudly rather than silently. Does it have obvious observability like structured logging or at least timestamped output. Does it use context cancellation or signal handling appropriately. Interviewers see endless scripts that work on the happy path and collapse on real data, and they are screening for the candidates who reach for robustness by default.

Warm-up drills worth doing: write a Bash one-liner that tails a log, filters for errors, and computes a per-minute count with awk; write a Python script that takes a list of URLs and returns response times with a concurrency limit and a retry policy; write a Go program that implements a worker pool over a channel with graceful shutdown on SIGTERM. If those are easy, you are ready.

Systems Design for SREs

SRE systems design questions are usually flavored differently from software engineering systems design. The question might be "design a metrics collection pipeline" or "design a deployment system" or "design a DNS service at scale." The angle is reliability first, features second.

Expect to reason about push versus pull architectures for metrics, at-least-once versus exactly-once semantics, cardinality control in time-series databases, the specific failure modes of control planes versus data planes, the blast radius of a misconfigured system, and multi-region considerations. Candidates who can articulate the difference between a control-plane outage that takes down deploys and a data-plane outage that takes down traffic, and who can reason about which is worse for a given system, are showing calibration.

A useful framework specific to SRE design is the error-budget lens. Every design decision should be evaluated for its impact on the error budget: does this add reliability cost, does it reduce reliability cost, does it make reliability observable. This framing, used naturally, distinguishes mid-level from senior candidates.

Sample Questions with Full Walkthroughs

Consider the scenario "a deploy went out thirty minutes ago and now p99 latency on the checkout service is double. Walk me through what you do." A strong candidate starts by confirming the observation: is the dashboard actually showing what the interviewer described, is the alert correlated, is it user-visible or just internal. Then they ask about blast radius: is this one region, one host, all hosts. Then they consider rollback as a mitigation: if the timeline matches the deploy, roll back first and investigate second. While the rollback is in motion they look at what the deploy changed, they check for downstream dependencies that might be implicated, and they prepare a status update. Only after mitigation do they move into root cause: was it a code bug, a config change, a migration, a dependency version bump. Throughout, they communicate and keep a timeline.

Or take "explain how you would design alerting for a new service." A weak answer lists alerts on CPU, memory, and disk. A strong answer starts with user journeys: what does success look like from the user's perspective, what SLIs capture that, what SLOs are appropriate, and what alerts on SLO burn rate would let you catch problems before you exhaust the budget. Infrastructure-level alerts on CPU and memory are secondary and often noisy; they should either be rolled up into health checks or demoted to dashboards.

Or the networking question "an engineer reports that service A cannot reach service B. Walk me through the investigation." The structured approach is to verify at each layer: is DNS resolving the hostname correctly, is TCP connecting, is TLS handshaking, is the HTTP response what you expect, is the response being consumed. At each layer you have specific tools: dig for DNS, nc or curl for TCP and HTTP, openssl s_client for TLS, packet captures for everything underneath. Naming the tools while walking the layers signals fluency.

Frameworks for Structured Answers

For troubleshooting: walk the layers from process to OS to host to network, and measure before guessing. For incidents: assess, contain, communicate, diagnose, recover, learn. For design: open with requirements and scale, then functional design, then data model, then scaling, then failure modes, then operational concerns. For on-call: every alert is a design artifact that should be actionable, with a runbook and a clear SLO link. For post-mortems: blameless narrative, timeline, contributing factors, root cause, action items with owners.

For scripting: clarify inputs and outputs, ask about scale and failure modes, sketch pseudocode before writing real code, and narrate defensive choices as you make them. For IaC: separate module boundaries from environment boundaries, name your state layout explicitly, and discuss the promotion path from local to production.

Common Mistakes That Tank Otherwise-Strong Candidates

The most common mistake is skipping mitigation in incident questions and diving straight into root cause. Senior SREs are judged heavily on whether they understand the asymmetry between keeping the site up and learning why it failed, and candidates who rush to debug while traffic bleeds are failing the bar even when their debugging is technically strong.

The second common mistake is over-reliance on the current toolchain. A candidate who describes everything in terms of one specific company's internal tools, without ever stepping back to the first-principles concept, signals that they would struggle to adapt to a new environment. Talk about concepts first, tools second, and use your specific tool as an example rather than the frame.

The third mistake is treating SRE interviews like SWE interviews and over-investing in algorithms at the expense of systems and networking depth. The scripting round is rarely a hard algorithms problem; the networking round is where the depth signal lives.

The fourth mistake is shallow observability answers. Saying "I would add monitoring" is almost meaningless. Say which SLIs, which dashboards, which alerts, and how the alerts tie to runbooks. Specificity is the signal.

The fifth mistake is failing to show operational humility. SRE interviews reward candidates who admit uncertainty, who ask clarifying questions, who acknowledge when a decision is a judgment call under pressure. Candidates who project false confidence often get read as risky additions to an on-call rotation.

Study Plans

For a two-week sprint assuming you are already in an SRE role: one hour per day of troubleshooting drills on a personal VM, two mock incident response sessions per week, one networking deep-dive per week, one IaC module written from scratch per week, and two full mock loops in the second week. Spend your last three days on behavioral preparation and a rest day.

For a six-week plan if you are transitioning from software engineering: weeks one and two on Linux and networking fundamentals end-to-end, weeks three and four on IaC and cloud platform depth, week five on incident response and observability, week six on full-loop simulations. Build one production-quality project during the plan: a full pipeline from application code to cloud infrastructure with real observability and a real SLO. That project is your narrative anchor.

Throughout, keep a running list of gaps that came up in mock interviews and dedicate the last hour of every study session to the bottom of that list.

FAQ

How much Kubernetes do I need to know?

If the target company uses Kubernetes in production, a lot. You should understand the control plane components including the API server, scheduler, and controller manager, the role of etcd, how pods become running containers, how the kubelet interacts with the container runtime, how the service abstraction works, how networking differs between plain services and ingress, and at least one CNI plugin in depth. If the company does not use Kubernetes, you still need enough conceptual understanding to reason about orchestration, because the questions often use Kubernetes as a thought vehicle.

Should I prefer DevOps or SRE titled roles?

Titles vary wildly by company. At some companies DevOps implies more build-and-release work and SRE implies more production operations; at others the titles are interchangeable. Read the job description carefully and pay special attention to the balance of coding, on-call, and tool-building. If on-call load is not mentioned, ask about it directly in the recruiter call.

How important is cloud certification?

Less than most candidates think. Certifications verify breadth of exposure but do not distinguish senior from staff engineers. A strong set of production projects and a clear incident narrative in interviews outweighs a shelf of certifications. Certifications can help you clear the initial resume screen if your background is unconventional.

Do I need to know Go?

For senior SRE roles at infrastructure-heavy companies, Go is becoming the default for tooling and operators. You do not need to be an expert, but you should be comfortable reading and writing idiomatic Go, especially around concurrency primitives, context handling, and error wrapping. Python alone is rarely a blocker, but Go fluency is increasingly expected.

How do I handle a round where I do not know a tool the interviewer expects?

Be honest and pivot to the underlying concept. If the interviewer asks about a tool you have not used, say "I have not worked with X specifically but I have solved this class of problem with Y, and here is how I would approach it." Senior interviewers respect the honesty and are often willing to map your experience to their tool on the fly. Bluffing is almost always detected and far more costly than admitting a gap.

What is the bar for staff versus senior SRE?

Staff SRE bar is that you set technical direction for reliability at the org level, not just individual systems. You should be able to talk about cross-team programs, multi-quarter reliability initiatives, the economics of reliability investment, and the social systems that surround on-call rotations. Senior SRE bar is that you own the reliability of major systems end-to-end and can lead the team through incidents. The distinction often comes out in the scope of examples candidates reach for.

Conclusion

DevOps and SRE loops reward engineers who can think about systems as a whole and who have the operational maturity to act wisely under pressure. The best preparation is not memorization but practice. Run your own small production system, break it on purpose, fix it, and reflect on what you learned. Then walk into the interview loop ready to narrate your thinking with discipline and humility, and the rest follows.

Use the frameworks in this guide as structure, but let your real production experience be the substance. Interviewers can tell the difference between a candidate who has read about incidents and a candidate who has been in them, and the signal that separates the two is specificity. Be specific, be structured, and be honest about what you do not know.

Frequently asked questions

How is an SRE interview loop different from a software engineer loop?

SWE loops are primarily about code. SRE loops are primarily about systems, with the code mostly tooling and glue. You will be judged on production reasoning, blameless postmortems, knowing when to roll back instead of root-cause first, and prioritizing work against SLOs and error budgets. A technically brilliant candidate without operational maturity will fail at senior levels; the calibration leans heavily on real on-call instincts.

What framework should I use for an incident response round?

Use assess-contain-communicate-diagnose-recover-learn. In the first minute, assess blast radius and start communications. Contain damage (fail over, roll back) before deep diagnosis. Interviewers specifically watch whether you separate mitigation from root cause, escalate appropriately rather than hero-ing it alone, and treat the postmortem as a blameless learning exercise. Skipping mitigation to chase root cause while traffic bleeds is the most common senior-level fail.

How deep do I need to go on networking for an SRE interview?

Deep enough to walk through what happens when a user types a URL into a browser, with at least one failure mode at every layer: DNS resolution and caching, TCP handshake, TLS termination and certificate rotation, HTTP request and response with keep-alives and HTTP/2 or HTTP/3, load balancer routing, and any backend layers. Drill enumerating ten causes for symptoms like 'service A intermittently gets connection reset from service B' from the kernel up.

What is a strong answer for designing alerts on a new service?

Start with user journeys, not infrastructure. Define SLIs that capture user experience, set conservative SLOs from historical data, and alert on SLO burn rate so you catch problems before exhausting the budget. Demote CPU and memory alerts to dashboards or roll them into health checks. Every alert should be actionable, have a runbook, and tie back to a specific SLO; alerts that fire too often are bugs.

How important is Kubernetes knowledge for SRE interviews?

If the target company runs Kubernetes in production, you need depth: API server, scheduler, controller manager, etcd, kubelet, container runtime interaction, the service abstraction, ingress versus plain services, and at least one CNI plugin. If they do not run Kubernetes, you still need conceptual orchestration fluency because interviewers often use it as a thought vehicle for control-plane vs data-plane questions.