Monitoring of Flux Kustomizations and HelmReleases in Pipedash with a kubeconfig, RBAC rights, reconciliation history, Reconcile and a kind test cluster.
The FluxCD plugin (fluxcd) monitors Flux Kustomizations and HelmReleases on a Kubernetes cluster. It shows the state of each resource, lists its reconciliation history and requests a reconciliation.
Requirements
The plugin reads the cluster through a kubeconfig and has no token field. The kubeconfig path points to a file on the machine that runs Pipedash. In the web app, that machine is the server and not the browser host.
The Kubernetes user in the kubeconfig needs these rights. The "Check permissions" button in the Add provider dialog tests each one with a SelfSubjectAccessReview and opens a "Kubernetes permissions" dialog.
| Permission | Kubernetes right | Used for | Required |
|---|---|---|---|
| Read Kustomizations | get and list on kustomizations in kustomize.toolkit.fluxcd.io | Status, conditions and history | Yes |
| Read HelmReleases | get and list on helmreleases in helm.toolkit.fluxcd.io | Status, conditions and release history | Yes |
| Request Kustomization reconciliation | patch on kustomizations | Reconcile | No |
| Request HelmRelease reconciliation | patch on helmreleases | Reconcile | No |
| Read Flux controller deployments | get on deployments in apps, named source-controller, kustomize-controller and helm-controller, in the Flux namespace | Controller health | No |
With an empty namespaces value, Pipedash lists both kinds across the cluster. That needs list at cluster scope, so a ClusterRole and a ClusterRoleBinding. A namespace-scoped Role works when namespaces names those namespaces.
Validation fails when the cluster serves neither the Kustomization nor the HelmRelease API. If only one API exists, Pipedash lists that kind.
Configuration
| Key | Type | Required | Default | Description |
|---|---|---|---|---|
kubeconfig | Text | No | $KUBECONFIG if set, else ~/.kube/config | The "Kubeconfig" field. Path to one or more kubeconfig files. : separates paths on Unix and ; on Windows. Pipedash merges the files that exist. ~ and environment variables expand. |
token | TextArea | No | None | The "Kubeconfig content" field. Kubeconfig YAML in place of a path. It wins over kubeconfig, and Pipedash stores it with the other credentials. |
context | Select | No | None | The "Kubernetes context" field. A context from the kubeconfig. Empty uses current-context. |
namespaces | Text | No | None | Comma-separated namespaces to monitor, for example flux-system,apps. Empty monitors every namespace. |
flux_namespace | Text | No | flux-system | Namespace of the Flux controllers. Controller health and its permission check read the deployments there. |
selected_items | Text | No | None | Comma-separated Kind__namespace__name values, for example Kustomization__apps__web. The resource picker writes it. Empty monitors every discovered resource. |
status_page_url | Text | No | None | The "Status page URL (optional)" field. Statuspage address for a custom status source. |
Namespace names use lowercase letters, numbers and hyphens, start and end with a letter or number, and have at most 63 characters. Pipedash drops duplicates. A selected resource outside namespaces makes the configuration invalid.
The pasted content in token suits devices that have no kubeconfig file, such as iOS and Android. On those systems, a kubeconfig user that authenticates through an exec credential plugin fails with an error. Examples are aws, gke-gcloud-auth-plugin and kubelogin. A token or a client certificate works there.
Cluster connection
FluxCD has no base URL. The kubeconfig supplies the API server and the credentials, and context picks the cluster entry. The plugin stops with an error when none of the listed kubeconfig paths exists.
Resource discovery
Pipedash lists Kustomizations and HelmReleases in each namespace from namespaces, or in all namespaces when the value is empty. Lists load in pages of 500 items.
The picker shows the namespace as the organization. Each entry has the name Kind/name and carries the path or chart and the source. Search matches the identity, name, source and description.
After the selection step, only the chosen resources appear. The list has the title "FluxCD resources" and groups them by source. A chosen resource that no longer exists stays in the list with the status pending and the label "Not found". The other resources keep refreshing.
A change to kubeconfig, context or namespaces in the Add provider dialog clears the saved selection. A new kubeconfig value also clears context. When editing a provider, the dialog says "Select resources again after changing connection scope." and disables "Save" until the user picks the resources again. Other edits, such as the credentials, keep the selection.
Supported kinds and API versions
| Kind | API version | Source shown | Path or chart shown |
|---|---|---|---|
| Kustomization | kustomize.toolkit.fluxcd.io/v1 | spec.sourceRef | spec.path |
| HelmRelease | helm.toolkit.fluxcd.io/v2 | spec.chart.spec.sourceRef, else spec.chartRef | spec.chart.spec.chart with @ and the chart version |
Pipedash reads no other kind and no other API version.
Reconcile
The "Reconcile" action replaces the Trigger action. It sits in the row menu of a resource, in the header of the history page and in the "Reconciliation details" dialog. The dialog "Reconcile: <resource>" has no parameters.
Pipedash sets the annotation reconcile.fluxcd.io/requestedAt on the resource to the current time, with the field manager pipedash. This needs the patch right. Flux processes the request asynchronously, so status and history update after the controller handles it.
- Configuration: A reconcile uses the current configuration of the resource. It does not restore the revision of a history record.
- Suspended resources: Pipedash refuses the request and never resumes or suspends a resource. The dialog tells the user to resume the resource in Flux first.
- Missing resources: The dialog says "This resource was not found. Update the selection or recreate it in Flux."
- Result: A notification titled "Reconciliation requested" follows a request. A failure shows "Couldn't request reconciliation".
- Cancel: Flux has no cancel for a reconciliation, and the plugin offers no cancel action.
A Reconcile request also clears the cached history of that resource. The next read fills it again from status.history.
Status mapping
Pipedash applies these rules from top to bottom to the resource.
| Pipedash status | Label | Flux state |
|---|---|---|
skipped | Suspended | spec.suspend is true |
failed | Not ready | Stalled is True, or Ready is False while Reconciling has the reason ProgressingWithRetry |
running | Reconciling | Reconciling is True |
success | Ready | Ready is True |
failed | Not ready | Ready is False |
running | Reconciling | Ready is Unknown |
pending | Pending | No current Ready condition |
cancelled | None | Not produced by this plugin |
A condition counts only when its observedGeneration is at least metadata.generation. A Ready condition from an older generation reads as missing. The status is then pending with the reason StatusOutdated.
The status filter of the resource list offers Ready, Not ready, Reconciling, Pending and Suspended.
Each history record has its own status:
| Kind | Pipedash status | Flux state |
|---|---|---|
| Kustomization | success, labeled "Reconciled" | lastReconciledStatus is ReconciliationSucceeded |
| Kustomization | failed, labeled "Reconcile failed" | Any other lastReconciledStatus |
| HelmRelease | success | Release status deployed or superseded |
| HelmRelease | failed | Release status failed |
| HelmRelease | running | Release status starting with pending-, or uninstalling |
| HelmRelease | pending | Any other release status |
History and cache
Flux keeps a bounded status.history and groups repeated reconciliations. Pipedash builds the run history from it, so the history is not a full execution log. The history page says so with the note "FluxCD retains a limited reconciliation history. Repeated reconciliations may be grouped, and older entries may be unavailable."
- Kustomization: One record per history group, named "Reconciliation" with the first 7 characters of the revision digest. The record starts at
lastReconciledminuslastReconciledDurationand ends atlastReconciled. - HelmRelease: One record per history entry, named "Release #
<version>". It starts atlastDeployed, or atfirstDeployedwhen that is missing. Only the newest attempt has a duration. - Current state: Pipedash adds a "Current state" record when the newest history record does not match the live state. Otherwise the newest record shows the current reason and message.
The history table shows the column "Recorded" in place of "Started" and sorts records by that time, newest first. A group that Flux reconciles again keeps its record, and the record moves up. The page shows "Flux has not recorded a reconciliation for this resource yet." for an empty history.
Unlike other providers, FluxCD keeps its run cache across refreshes. Pipedash keeps a finished record that Flux no longer reports, as long as the resource keeps the same uid. Such a record gets the suffix "(cached)" on its status, and its details dialog shows the alert "Cached reconciliation". Pipedash drops any other record that Flux no longer reports. These are running and pending records, current-state records and records of a recreated resource. Clearing the run history cache in Settings removes the kept records, as does a Reconcile request.
Metrics count finished history records. They update a grouped record without a second count. The current-state record is a snapshot and never enters metrics. A record that started before the metrics retention window does not enter metrics, so reading the history never recreates metrics outside that window. The Metrics and Cache pages cover both features.
Controller health and status page
Pipedash reads the deployments source-controller, kustomize-controller and helm-controller in flux_namespace. A deployment scaled to zero or with no available replica reports a major outage. A deployment with fewer available replicas than desired reports a degraded state. The provider status dialog lists these controllers under "Controllers".
status_page_url adds a custom status source. It takes an https URL without credentials, query or fragment. A literal loopback address may use http. Pipedash reads /api/v2/summary.json below that URL. The Provider health page covers both signals.
TOML example
Without selected_items, Pipedash monitors every resource in the listed namespaces.
[providers.flux-prod]
name = "Flux Prod"
type = "fluxcd"
refresh_interval = 30
[providers.flux-prod.config]
kubeconfig = "~/.kube/config"
context = "prod-cluster"
namespaces = "flux-system,apps"
flux_namespace = "flux-system"
selected_items = "Kustomization__apps__web,HelmRelease__apps__api"To paste the kubeconfig in place of a path, set the top-level token of the provider to the kubeconfig YAML.
Test environment with kind
The folder examples/fluxcd holds manifests that run Flux in a kind cluster. They create healthy, failed and suspended Kustomizations and HelmReleases and a read-only service account for Pipedash. All sources use the podinfo Git tag 6.7.1 and the podinfo chart 6.7.1.
Create the cluster
The config kind-cluster.yaml names the cluster pipedash-fluxcd and uses the node image kindest/node:v1.37.0.
DOCKER_CONTEXT=<ctx> KUBECONFIG=~/.kube/pipedash-fluxcd.kubeconfig \
kind create cluster --config examples/fluxcd/kind-cluster.yaml
chmod 600 ~/.kube/pipedash-fluxcd.kubeconfigThe API server listens on 127.0.0.1:46443 of the Docker host only. For a remote Docker host, forward the port:
ssh -N -L 127.0.0.1:46443:127.0.0.1:46443 <docker-host>Install Flux and the fixtures
export KUBECONFIG=~/.kube/pipedash-fluxcd.kubeconfig
kubectl apply -f https://github.com/fluxcd/flux2/releases/download/v2.9.6/install.yaml
kubectl -n flux-system wait --for=condition=Available deploy --all --timeout=300s
kubectl apply -k examples/fluxcdThe fixtures create these objects in the namespace pipedash-examples:
| Kind | Name | Expected state |
|---|---|---|
| Kustomization | podinfo-healthy | Ready |
| Kustomization | podinfo-failed | Not ready, path not found |
| Kustomization | podinfo-suspended | Suspended |
| HelmRelease | podinfo-healthy | Ready |
| HelmRelease | podinfo-failed | Not ready, install failed |
| HelmRelease | podinfo-suspended | Suspended |
In the failed Kustomization, the path does not exist. The failed HelmRelease uses a nonexistent image tag and a 30-second timeout. Its controller message may describe an installation timeout and not the image pull failure.
The file rbac.yaml adds the service account pipedash-readonly. It has get and list on the two kinds in pipedash-examples and nothing else, so the optional rights in the permission dialog show as missing. The account has no token secret. A short-lived token comes from this command:
kubectl -n pipedash-examples create token pipedash-readonly --duration 1hConnect Pipedash
- Select "FluxCD" in the Add provider dialog.
- Set "Kubeconfig" to the absolute path of
~/.kube/pipedash-fluxcd.kubeconfigon the machine that runs Pipedash. - Select the context
kind-pipedash-fluxcdand enter the namespacepipedash-examples. - Select "Check permissions", continue to the resource selection and select the 6 examples.
The resource list shows the states Ready, Not ready and Suspended. Opening a resource shows the controller messages and the reconciliation history.
Check the cluster
kubectl -n pipedash-examples get kustomizations,helmreleases
kubectl -n pipedash-examples annotate kustomization/podinfo-healthy --overwrite \
reconcile.fluxcd.io/requestedAt="$(date +%s)"The annotate command sets the annotation that "Reconcile" sets. The read-only service account cannot run it, because it has no patch right.
Standalone library checks
The application that consumes the plugin chooses the Kubernetes API model of k8s-openapi. The environment variable K8S_OPENAPI_ENABLED_VERSION supplies that choice for a standalone check of the crate. The workspace model is 1.32.
K8S_OPENAPI_ENABLED_VERSION=1.32 cargo check -p pipedash-plugin-fluxcd --lib
K8S_OPENAPI_ENABLED_VERSION=1.32 cargo package -p pipedash-plugin-api -p pipedash-plugin-fluxcdThe package command names both crates. This makes the verification use the plugin API from the checkout and not its previously published version. The CI and release workflows set the same variable when they package and publish the plugin crates.
ArgoCD
Monitoring of ArgoCD applications and sync history in Pipedash, with Git organization filters and sync triggers.
Dashboard
The Pipelines page groups workflows by repository, filters, sorts and pins them. It also covers the sidebar, theme, keyboard shortcuts, context menus, accessibility, confirmations and forms.