Altamar Labs
Back to Blog
reliability· 2 min read

Graceful degradation: designing so the secondary fails without taking down the essential

When a non-critical component depends on an external service, the most important design question is what happens the day that service stops responding.

Any system with enough time in production ends up depending on some external component that, eventually, is going to fail: a specialized search engine, a geocoding provider, a third-party service that enriches data. The question that separates a resilient system from a fragile one isn't whether that component will fail, it's what happens to the rest of the system when it does.

The most common mistake: treating everything as critical

It's common for a failure in a secondary component, like a specialized search engine that improves the experience but isn't essential to the product's basic function, to end up taking down functionality that shouldn't depend on it at all. This happens because the code doesn't distinguish between "this improves the experience" and "this is essential," handling both cases with the same rigidity: if the component doesn't respond, the whole feature goes down.

Designing the dependency hierarchy explicitly

The alternative isn't avoiding dependencies on external services, that's practically impossible in any real system. It's explicitly designing what happens when each dependency fails, classifying each one by its actual criticality:

Dependencies with an equivalent functional fallback. A specialized search engine that offers better relevance and speed can have, behind it, a simpler query against the main database as a backup. The experience degrades (less refined results, maybe slower) but the functionality doesn't disappear.

Dependencies with a partial fallback. A data enrichment service that doesn't respond can simply be skipped, showing the base information without the enrichment, instead of blocking the entire operation.

Genuinely critical dependencies. For these, there's no reasonable fallback, and the right design question isn't how to degrade, it's how to maximize availability of that specific dependency (redundancy, replicas, aggressive monitoring), because its failure should indeed be treated as a whole-system incident.

Why this isn't just abstract "best practices"

Classifying dependencies this way changes concrete architecture decisions. It determines whether a call to an external service should be blocking or can run with a short timeout and a default result. It determines whether a failure gets logged as a silent alert (something degraded, but the user never noticed) or as an incident that wakes someone up at 3am. Without that explicit classification, every dependency ends up treated, by default, as equally critical, which is almost never true and almost always more expensive than necessary.

The lesson

Resilience doesn't come from eliminating external dependencies, it comes from clearly deciding what happens when each one fails, before it fails in production. A system that degrades gracefully isn't one that never has problems. It's one where a secondary component's problem stays contained to that component, instead of spreading to functionality that never should have depended on it in the first place.