Opscraft
Jenkins · Lesson 7 of 11 · 3 min read

Performance and scaling

Symptoms of an overloaded controller

Slow UI, queue not moving, builds hanging at "Waiting for...", OutOfMemoryError, long GC pauses, agents disconnecting.

Causes and fixes

Cause Fix
Builds running on the controller 0 executors; agents only
Too little heap / bad GC Right-size heap, G1GC, GC logging; monitor with the Metrics/Prometheus plugin
Huge build history (tens of thousands of builds) buildDiscarder everywhere; global default via JCasC
Large archived artifacts and stashes Store in Nexus/S3; keep stashes small
Heavy Groovy in pipelines (CPS) Move logic into shell scripts or tools; @NonCPS helpers for pure computation
Pipeline durability overhead options { durabilityHint('PERFORMANCE_OPTIMIZED') } for pipelines where resume-after-restart isn't needed
Massive console logs Reduce verbosity, avoid printing huge outputs
Too many plugins, some slow Audit and remove; check thread dumps for plugin hotspots
Polling hundreds of repos Webhooks instead of pollSCM
One controller for the whole company Split by domain/team

Monitoring: the Prometheus metrics plugin exposes queue length, executor usage, build durations, JVM stats — graph them in Grafana and alert on queue length and heap.

Enlarge diagramOverloaded controllerHealthy controllerBuilds on the controllerYears of build historyHeavy Groovy, big stashesHundreds of repos polled0 executors, pod agentsbuildDiscarder everywhereWork in shell steps; artifacts in Nexus/S3Webhooks; split controllers by team
Figure 1. Scaling Jenkins: keep the controller light and push work out to agents.
Swipe sideways to see all of it, or tap to open it full screen.

Making builds faster

Measure stage timings first (Stage View/Pipeline Graph View). Then: parallel stages, dependency caching (proxy repositories, cache volumes), Docker layer caching, incremental builds, smaller agent images, splitting slow test suites (by timing), and skipping work that the change doesn't touch (when { changeset ... } in monorepos).