Opscraft
Service mesh in depth · Lesson 3 of 6 · 3 min read

Traffic management

Istio routing resources

Resource Purpose
VirtualService How requests to a host are routed: match on path or headers, split by weight, set timeouts, retries, fault injection
DestinationRule What happens after routing to a service: subsets (versions by label), load-balancing algorithm, connection pools, outlier detection, TLS mode
Gateway (Istio) Listeners at the edge of the mesh for ingress or egress
Gateway API (HTTPRoute, GRPCRoute) The Kubernetes-standard alternative that Istio supports and recommends for new setups

A canary with Istio resources:

Enlarge diagramcallerGET /chargeVirtualServiceweights 95 / 5subset v1pods version=v1 · 95%subset v2pods version=v2 · 5%
Figure 1. A VirtualService splits traffic between subsets that a DestinationRule defines by label.
Swipe sideways to see all of it, or tap to open it full screen.
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata: { name: payments }
spec:
  host: payments
  subsets:
    - { name: v1, labels: { version: v1 } }
    - { name: v2, labels: { version: v2 } }
  trafficPolicy:
    connectionPool:
      http: { http1MaxPendingRequests: 100, maxRequestsPerConnection: 50 }
    outlierDetection:
      consecutive5xxErrors: 5
      interval: 10s
      baseEjectionTime: 30s
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata: { name: payments }
spec:
  hosts: [payments]
  http:
    - match: [{ headers: { x-canary: { exact: "true" } } }]
      route: [{ destination: { host: payments, subset: v2 } }]
    - route:
        - { destination: { host: payments, subset: v1 }, weight: 95 }
        - { destination: { host: payments, subset: v2 }, weight: 5 }
      timeout: 2s
      retries: { attempts: 2, perTryTimeout: 500ms, retryOn: "5xx,reset,connect-failure" }

Flagger and Argo Rollouts automate this: they shift weights step by step, check metrics (error rate, latency) at each step and roll back automatically.

Resilience features, used carefully

  • Timeouts bound how long a caller waits; set them on every route, shorter than the caller's own deadline.
  • Retries hide transient failures, but retries at every layer multiply load: three layers retrying three times turns one failure into 27 calls. Retry only idempotent requests, at one layer, with a per-try timeout and a budget.
  • Circuit breaking in Envoy is a connection-pool limit: requests beyond the pending or connection limits fail fast with 503 instead of queuing.
  • Outlier detection temporarily ejects an endpoint that returns consecutive errors, so one bad pod stops receiving traffic.
  • Fault injection adds delays or errors for a percentage of requests in test environments, to check that callers really handle them.
Enlarge diagramfrontend3 attemptscheckout3 attempts eachpayments3 attempts eachbank API27 callsRetry at one layer, only idempotent calls, with per-try timeouts and a budget.
Figure 2. Retries multiply: three layers each trying three times turn one failing call into 27.
Swipe sideways to see all of it, or tap to open it full screen.