Refresh
How the refresh loop polls providers, backs off when nothing changes and switches between active and idle.
Pipedash polls every provider in the background and tells the UI when pipeline data changes. The loop starts after the providers load and stops when the app quits.
The loop
The loop wakes every 5 seconds. Each wake runs 3 checks in order.
| Value | Setting |
|---|---|
| Tick | 5 seconds |
| Starting interval | 10 seconds. The first fetch runs on the first tick. |
| Backoff trigger | 3 quiet cycles in a row |
| Backoff step | Double the interval, capped at 300 seconds |
| Reset | Any change returns to 10 seconds and clears the quiet count |
| Provider timeout | 30 seconds each |
| Whole fetch timeout | 60 seconds |
| Concurrency | 10 providers at a time |
The interval keeps doubling on every later quiet cycle: 10, 20, 40, 80, 160, then 300 seconds.
A change means a different pipeline count, a new pipeline, or a different status or last run than the cached copy. One failing provider does not stop the others. Pipedash stores its error as the provider's fetch status and shows it in the sidebar. Only a failure of the whole fetch, for example the 60 second timeout, raises a refresh error.
Active and idle mode
Active mode polls. Idle mode skips the provider fetch entirely. The priority queue and the metrics cleanup still run in idle mode. Pipedash starts in active mode.
- Desktop: window focus sets
activeand losing focus setsidle. - Browser: a visible tab sets
activeand a hidden tab setsidle.
REST uses PUT and GET /api/v1/refresh/mode. Any value other than active counts as idle. A mode change leaves the interval as it is.
Priority queue
Adding a provider queues it once. The next tick fetches it. This path runs before the mode check, so a new provider loads even while the window is idle.
Per-provider refresh interval
Each provider stores a refresh interval in seconds. The default is 30 in the app and in [providers.<id>] in config.toml, and 60 when a REST API call creates or updates a provider.
The update route PUT /api/v1/providers/{id}/refresh-interval rejects values below 5 with Refresh interval must be at least 5 seconds.
The current version stores this value but does not use it to schedule polling. The loop above always uses its own adaptive interval. A value of 0 does not switch a provider to manual refresh either. The same applies to general.default_refresh_interval, whose default is 30.
Manual refresh
POST /api/v1/refresh/all clears all run history caches and then fetches every provider. The desktop and web interfaces have no button for it. The header refresh action re-reads the stored pipelines, and live updates arrive through events.
Live updates
The desktop app and the web server push an event to the interface when pipeline data changes, and the interface refetches the affected data. The web server sends events over the WebSocket at /api/v1/ws. The REST API page lists them.