Repository navigation
Releases
v1.17.0
🚀 New Features
ClusterHealthCheck: Notification Policy
A ClusterHealthCheck notification can now carry an optional policy that reduces the number of messages sent while a cluster is changing. onlyOnTransition notifies only when a cluster moves between passing and failing, not when the failure message changes while it keeps failing. minInterval sets the minimum time between two notifications for the same cluster, and changes inside the interval are coalesced. failingFor sets how long a cluster must keep failing before the failure is notified, so a failure that ends sooner is never sent, nor is its recovery. What was last delivered is recorded in the NotificationSummary, so the decisions survive a restart. Notifications without a policy behave as before. The failure message now lists resources in a stable order, so the same failures no longer look like a change.
PRs: libsveltos #699, healthcheck-manager #443, sveltos #808
Report Drifted Resources
The ClusterSummary now says which resources drifted. Each feature has a lastDrift with the detection time and the drifted resources (group, kind, namespace, name and, for Helm, the release). It is kept after the drift is corrected and replaced by the next one, and lists at most 20 resources with a truncated flag when more drifted. It is only set for profiles with syncMode ContinuousWithDriftDetection. The dashboard shows the drift history of each profile on a cluster, and sveltosctl has a new command, show drift-history, that lists the drifted resources and when they drifted, filtered by cluster, profile or feature. In pull mode a Helm chart is now re-applied only when it changed or drifted, as in push mode. Before, a change or a drift on one chart re-applied every chart of the profile.
PRs: libsveltos #701, addon-controller #2024, dashboard #199, sveltosctl #454
Kubernetes Events for Failures and Cluster State
Errors that were only in the logs or in a status field are now recorded as events, so they show in kubectl describe and in the events of the namespace. addon-controller records FeatureFailed when a feature fails and is retried. Before, an event was recorded only for failures that are not retried. Classifier records ClassifierFailed and ClassifierLabelConflict, and event-manager records EventTriggerFailed when an EventTrigger fails to deploy or an EventReport cannot be processed. sveltoscluster-manager records an event when a SveltosCluster goes down (AgentUnhealthy in pull mode, ClusterUnreachable in push mode) and ClusterRecovered when it is back. An event is recorded when the failure is new or different, not on every retry, and not for expected states such as a paused cluster. All reasons come from a shared vocabulary in libsveltos. See the upgrade note: the reasons of the events addon-controller already recorded changed.
PRs: libsveltos #703, libsveltos #704, classifier #523, sveltoscluster-manager #390, addon-controller #2032, event-manager #526
Health Monitoring of Deployed Resources
ClusterProfile, Profile and ClusterPromotion have an opt-in healthMonitoring section. When enabled, once every feature of a ClusterSummary is provisioned, addon-controller creates a HealthCheck and a ClusterHealthCheck for it. The default checks cover Deployment, StatefulSet, DaemonSet, Job and PersistentVolumeClaim and report Healthy, Progressing, Degraded or Suspended. Only the resources deployed by the profile are evaluated, including the ones from Helm charts. A ClusterProfile can add its own HealthChecks, or turn the default checks off and use only those. This answers what is unhealthy on a cluster among what Sveltos deployed, separately from validateHealths, which only gates the deployment. The dashboard shows a health badge on each cluster card, a Health column in the Profiles tab and an Unhealthy resources tab with filters. sveltosctl has a new command, show unhealthy-resources, that lists the resources that are not healthy with the profile owning each of them. sveltos-agent caches every object of the monitored kinds in the cluster, about 20 KB of heap per object, whatever the number of profiles that enable monitoring.
PRs: addon-controller #2037, addon-controller #2042, ui-backend #193, dashboard #201, sveltosctl #455
ClusterHealthCheck: ClusterRefs
A ClusterHealthCheck can list clusters by reference in clusterRefs, in addition to selecting them with clusterSelector. The two sets are added together and a cluster that is in both is counted once.
PR: healthcheck-manager #445
Dashboard: Matching Resources per Cluster on an EventTrigger
An EventTrigger can match hundreds of resources in a cluster, and listing all of them made the details page long and slow. Each cluster card now shows the number of matching resources, and clicking it opens a dialog with the list. The list is loaded on demand from a new paginated endpoint, with filters on kind, namespace and name. In ui-backend, /event returns matchingResourcesCount for each cluster instead of the resources array.
PRs: ui-backend #195, dashboard #202
Dashboard: Profile Rollout State
When a profile sets maxUpdate, the clusters not yet reached still run the previous configuration, but they looked provisioned. The profile page now shows a rollout state for each matching cluster: updated, updating, waiting because maxUpdate is reached, paused or not ready. A summary shows the number of clusters in each state and the resolved maxUpdate while a rollout is pending.
PRs: ui-backend #198, dashboard #204
⚙️ Improvements
addon-controller: Stop Polling Profile Deletion While Clusters Are Paused
The ClusterSummary of a paused cluster is not removed until the cluster is unpaused, and a ClusterProfile or Profile being deleted polled for it every 10 seconds, logging an error each time. On a management cluster with a few paused clusters that was about 2,100 errors in 35 minutes. The profile now logs once, waits, and is reconciled again when a paused cluster is unpaused. It keeps polling as before if any remaining ClusterSummary is for a cluster that is not paused.
PR: addon-controller #2023
SveltosCluster with Connection Down Is Not Ready to Be Configured
A SveltosCluster whose connection is down kept Status.Ready true, so it was still processed. It is now skipped and reconciled again when the connection is back. addon-controller no longer tries to collect ResourceSummaries from, or upgrade drift-detection-manager in, a cluster it cannot reach, which removes the repeated compatibility check and i/o timeout errors. Pull mode is unaffected.
PRs: libsveltos #702, addon-controller #2028
ui-backend: Skip the Per-Cluster Access Check When the User Can Get All Clusters
GET /profile checked access with one SubjectAccessReview for each matching cluster. It now first asks once per cluster type whether the user can get all clusters, and if so skips the per-cluster checks. For a profile matching 1,000 clusters that is one call instead of 1,000, and a request went from 238 ms to 1.4 ms on envtest. A user limited to some clusters gets the same result as before, through the per-cluster check.
PR: ui-backend #199
🐞 Bug Fixes
ui-backend: Malformed Bearer Header and OIDC Proxy Diagnostics
An Authorization header with Bearer and no token made ui-backend panic and return a 500. A malformed header or an empty token now gets a 401. When the token of a dashboard user is refused, the log now says what kind of token it was (JWT, Kubernetes ServiceAccount token or opaque), its issuer, audience and expiry, and the host it was sent to, never the token itself. With an OIDC proxy, a ServiceAccount token also gets a hint that the proxy needs token passthrough enabled.
PR: ui-backend #188
ui-backend: ClusterProfile Content Created in the Wrong Namespace
Creating a ClusterProfile from pasted YAML in the dashboard stored its ConfigMap in the hardcoded projectsveltos namespace, so it failed when Sveltos is installed in another namespace. ui-backend now creates it in the namespace it runs in, read from the NAMESPACE environment variable set through the downward API, and falls back to projectsveltos when it is not set.
PR: ui-backend #197
addon-controller: upgradeCRDs Skipp...
v1.16.1
Security
This release fixes a Profile namespace restriction bypass in addon-controller. Details are in the security advisory: GHSA-p45x-hwmp-fvrj.
A Profile is namespace-scoped and is meant to match clusters and reference resources only within its own namespace. addon-controller enforced this for most references, but not for the namespace of these three fields:
spec.helmCharts[].valuesFrom[].namespacespec.patchesFrom[].namespacespec.templateResourceRefs[].resource.namespace
A user allowed to create Profiles in one namespace could point these fields at another namespace. addon-controller would then read the referenced ConfigMaps, Secrets and other resources from that namespace with its own permissions.
From this release, the namespace of these fields is always set to the Profile's namespace, like the other references of a Profile. ClusterProfile is not affected: it is cluster-scoped and administrator-managed, so cross-namespace references are by design.
Affected versions: >= 0.21.0, < 1.16.1
Patched version: v1.16.1
Upgrade notes
Upgrade to v1.16.1. Profiles that reference ConfigMaps, Secrets or resources in a namespace other than their own through the three fields above will stop resolving those references. Move the referenced resources to the Profile's namespace, or use a ClusterProfile if the cross-namespace reference is intended.
If you cannot upgrade yet:
- Limit who can create and update Profiles to trusted users.
- Review existing Profiles for entries in the three fields whose namespace differs from the Profile's.
- Optionally add an admission policy that rejects Profiles where any of those namespaces is set and differs from
metadata.namespace.
Credits
Reported by Sathwik Yadlapati
v1.16.0
🚀 New Features
Pull Mode with Short-Lived Tokens
Pull mode no longer needs long-lived credentials. The agents in the managed cluster authenticate with short-lived tokens, which shrinks the exposure window if a credential leaks.
PRs: sveltoscluster-manager #382, sveltosctl #450, sveltos #794
Pull Mode with Limited RBACs
The components that run in pull mode can work with a reduced set of permissions. The agents only get the access they need on the managed cluster.
PR: classifier #515
CleanupGracePeriod
A grace period can now be set for cleanup of Sveltos resources on a cluster. This avoids premature removal during short disconnections or transient issues.
PRs: libsveltos #686, sveltoscluster-manager #384
sveltos-agent Reports Processing Errors
Errors that sveltos-agent hits while processing EventSources, Classifiers and HealthChecks are now reported. You can see why an instance is not being evaluated without digging through agent logs.
HealthCheck: Consecutive Checks
A HealthCheck can require several consecutive evaluations before the result changes. One-off flaps are filtered out without extra tooling.
PRs: sveltos #803, libsveltos #694
RemoteURL as a Template
remoteURL can be expressed as a template, so one profile can resolve to a different source for each matching cluster.
PR: addon-controller #2001
Dashboard: OIDC Proxy Support and Profile Editing
The dashboard can sit behind an oidc-proxy. Profiles can also be created and edited directly from the dashboard.
PRs: ui-backend #185, dashboard #190
🐞 Bug Fixes
addon-controller: Report Why a Profile Is Not Deployed When a Dependency Is Missing
When a profile depends on one that does not exist, the status now says so. Before, the profile just stayed undeployed with no explanation.
PR: addon-controller #1969
addon-controller: Stale Helm Conflict Not Cleared
A Helm conflict that had been resolved could stay in the status. It is now cleared.
PR: addon-controller #1975
addon-controller: Pull-Mode Helm Ignored createNamespace: false
In pull mode, Helm charts created the target namespace even when createNamespace was set to false. The setting is now honored.
PR: addon-controller #1982
addon-controller: Respect Order When Uninstalling Helm Releases
Releases are now uninstalled in the correct order, so ordering dependencies no longer cause failures on removal.
PR: addon-controller #1984
classifier: Label Keys Removed from spec.classifierLabels Were Not Dropped
When a key was removed from spec.classifierLabels, the label stayed on the cluster. The classifier now removes it.
PR: classifier #518
addon-controller: Deploy CRDs First When policyRefs Reference a remoteURL with an OCI Image
CRDs are now deployed before the other resources, so custom resources from the same OCI image no longer fail to apply.
PR: addon-controller #2005
addon-controller: MaxConsecutiveFailures Honored for validateHealth Failures
MaxConsecutiveFailures now also applies to validateHealth failures. Before, it did not.
PR: addon-controller #2006
addon-controller: ClusterConfiguration Cache Race with ClusterProfile Reconciliations
A race between the ClusterConfiguration cache and ClusterProfile reconciliations could leave stale data. It is fixed.
PR: addon-controller #2007
addon-controller: ClusterPromotion and Referenced Resource Issues
This fixes issues with ClusterPromotion and the resources it references.
PR: addon-controller #2012
🔧 Maintenance
All components advanced to Go v1.27.1.
All components advanced to Cluster API v1.14.2.
v1.15.0
🚀 New Features
Namespace-Scoped RBAC for Agentless Watches
sveltos-agent and drift-detection-manager, in agentless mode, previously assumed they could watch and list any resource anywhere, and would just 403-loop against a kubeconfig whose RBAC is scoped to a subset of namespaces. Both now support a --watch-namespaces flag: the watch registry keys watchers by (GVK, namespace) instead of a bare GVK, so a ResourceSelector with an explicit namespace watches just that namespace, and an empty ("match anywhere") selector fans out across only the configured namespaces instead of cluster-wide. Cluster-scoped resources are always let through.
Nothing needs to be set on the agents directly: a managed Cluster/SveltosCluster can carry the annotation agent.projectsveltos.io/watch-namespaces (a comma-separated namespace list), and addon-controller and classifier relay it down automatically. addon-controller passes it to drift-detection-manager's --watch-namespaces flag when deploying it in agentless mode, and also scopes its own stale-resource cleanup search to the same namespaces, so its own RBAC-restricted credential doesn't 403 on cleanup either. classifier does the same for sveltos-agent, threading the annotation into the manifest it deploys and folding its value into the deployment's config hash so a changed annotation triggers a redeploy rather than being silently ignored.
PRs: addon-controller #1939, classifier #504
Generic OIDC Workload Identity Provider
WorkloadIdentityConfig gains a fourth provider, OIDC, alongside AWS/GCP/Azure: a standard RFC 6749 client credentials grant against any compliant IdP (Dex, Keycloak, Okta, and similar), for managed clusters not on one of the three cloud providers with their own federation mechanism. Unlike the cloud cases, this is not secretless: Sveltos holds a standing client_id/client_secret in a referenced Secret and exchanges it directly at the IdP's token endpoint. The resulting token feeds into the same cache, proactive-refresh, and auth-failure eviction machinery the other three providers already use.
PR: libsveltos #675
Smooth ClusterProfile/Profile Transitions (transitionFrom)
Moving a cluster between profiles via a label swap used to force a full undeploy-then-redeploy cycle: the old profile tore down everything it managed, including resources identical to what the new profile was about to deploy, before the new one could even start. A ClusterProfile/Profile can now declare, via transitionFrom, that it replaces one or more named predecessors: teardown of the predecessor is deferred until the successor reaches Provisioned on that cluster, and the successor may take over the predecessor's resources in place, ignoring tier restrictions. Support spans addon-controller (the core deferral/takeover logic), sveltos-applier (equivalent handling for pull-mode clusters), and new documentation.
PRs: addon-controller #1951, sveltos #787
🐞 Bug Fixes
addon-controller: RemoteURL Polling Interval Blocked Drift-Detection Recovery
A PolicyRef/KustomizationRef using RemoteURL with an Interval set NextReconcileTime on the ClusterSummary to schedule its periodic re-fetch, but that same field suppressed every other reconcile of that ClusterSummary until the cooldown expired, including one triggered by drift detection. With SyncMode: ContinuousWithDriftDetection and, say, a 10-minute RemoteURL interval, a configuration drift would sit unrecovered until that cooldown passed. NextReconcileTime is no longer set in this case.
PR: addon-controller #1929
ui-backend: Three Authentication and Authorization Bugs
/stats returned a 500 on a management cluster with no Cluster API installed; it now short-circuits to zero CAPI clusters when the CRD is absent. Every RBAC check (canListSveltosClusters, canGetClusterProfile, and similar) was sending the Kubernetes Kind as the SubjectAccessReview's resource field instead of the plural REST name RBAC rules actually use, so no real scoped ClusterRole could ever match, only a wildcard role like cluster-admin. And every SubjectAccessReview only ever set Spec.User, never Spec.Groups, so RBAC granted via a Group subject was silently denied regardless of what the role granted.
PR: ui-backend #181
dashboard: OIDC Login Rejected by Audience-Checking API Servers
The dashboard's OIDC login hardcoded its scope to openid profile email offline_access, requesting no specific resource. Providers that issue single-resource access tokens (Entra ID, notably) then audience the token to the dashboard's own app registration rather than the target API server, so
dashboard: OIDC Login Rejected by Audience-Checking API Servers
The dashboard's OIDC login hardcoded its scope to openid profile email offline_access, requesting no specific resource. Providers that issue single-resource access tokens (Entra ID, notably) then audience the token to the dashboard's own app registration rather than the target API server, so a Kubernetes API server that authorizes by audience (e.g. AKS with Microsoft Entra ID authorization) rejected it with "failed to validate token." A deployer can now set an optional VITE_OIDC_SCOPE/OIDC_SCOPE to request a resource-specific scope instead; unset, behavior is unchanged.
PR: dashboard #187
libsveltos: Drift Exclusion Crashed on a Path Absent From the Helm Render
Helm's PostRenderer fed driftExclusion patches straight to kustomize with no missing-path check, unlike drift evaluation's own path, which already treats a remove on an absent path as a no-op. A chart render missing the excluded path failed hard instead of skipping it. The PostRenderer path now filters patches the same way drift evaluation does before handing them to kustomize.
PR: libsveltos #673
addon-controller: ClusterSummary Status Update Could Lose a Race
updateStatusForNonReferencedHelmReleases did a single Get plus a single Status Update with no conflict retry, unlike its two siblings on the same pass. A concurrent status write could bump the ClusterSummary's resourceVersion in between, losing the update to a resource-version conflict even though the Helm deploy had already succeeded: the feature hash never advanced, the Helm feature kept reporting Failed, and drift-detection registration was never reached. Now wrapped in the same conflict-retry the other two call sites already use.
PR: addon-controller #1937
addon-controller: Silent ResourceSummary Removal Failures
A failure removing a ResourceSummary, in either prepareForDeployment or cleanupBeforeFinalizerRemoval, was only logged, never written to ClusterSummary.status. An ongoing failure was invisible to kubectl get clustersummary: status just sat wherever it last was, silently retrying every reconcile with no visible sign anything was wrong. Both call sites now surface the error via status.
PR: addon-controller #1947
libsveltos: Auth-Rejected Clients and Workload-Identity Tokens Not Evicted From Cache
A cached client or workload-identity token rejected server-side for a reason its own recorded expiry doesn't predict (early revocation, an IdP restart, a permissions change) kept being reused until whatever proactive refresh window applied eventually caught up. GetKubernetesRestConfig/GetKubernetesClient and the workload-identity path now both wrap the returned config's transport so any 401/403 response, from any client built off it, evicts the corresponding cache entry immediately instead of waiting on a schedule.
PRs: libsveltos #674, libsveltos #676
addon-controller: Deferred Teardown Not Reflected in ClusterSummary Status
When teardown was deferred by a still-existing DependsOn dependent or a TransitionFrom successor not yet Provisioned, ClusterSummary.status.featureSummaries[].status stayed on whatever it was before, implying active work even though nothing was happening. status.dependencies already explained the wait; status itself did not. A new Blocked FeatureStatus value now reflects it directly.
PR: addon-controller #1953
Stop Polling Reports From Pull-Mode Clusters With a Dead sveltos-applier
healthcheck-manager, event-manager, and classifier each kept listing HealthCheckReports/EventReports/polling a pull-mode cluster every collection cycle regardless of whether its sveltos-applier agent was still sending a heartbeat. A cluster whose agent has died will never produce a new report, so this work, and its retry cadence, ran indefinitely for nothing. All three now check the agent's heartbeat status first and skip a cluster whose sveltos-applier has gone stale.
PRs: healthcheck-manager #430, event-manager #517, classifier #507
🔧 Maintenance
- All components advanced to Go v1.27.1
- All components advanced to Cluster API v1.14.2
v1.14.0
🚀 New Features
Job-Based Health Checks
ValidateHealth gains JobCheck, a third check type alongside the existing Lua Script and CEL evaluation: point it at a Job manifest (via JobRef, stored in a Secret or ConfigMap) and its Complete/Failed outcome becomes the check result, mutually exclusive with the other two. Since ValidateHealth is embedded by both ClusterProfile/Profile and ClusterPromotion, it's available to PreDeployChecks, ValidateHealths, PreDeleteChecks, PostDeleteChecks, and PostDelayHealthChecks with no further changes.
PRs: libsveltos #654, sveltos #775
Per-Resource Force Recreate
KustomizationRef.Force/PolicyRef.Force applied all-or-nothing to every resource in a reference. A single resource can now opt into force-recreate on its own by carrying the projectsveltos.io/forceRecreate annotation, even when the reference-level Force is false.
PR: libsveltos #655
Gzip-Compressed and bsdtar-Produced OCI RemoteURL Layers
RemoteURL's oci:// path had two gaps that caused real-world OCI artifacts to fail or get misread: a gzip-compressed tar layer failed the tar parse and fell through to the raw-blob path, so the compressed bytes were handed downstream as if they were YAML; and AppleDouble/PAX sidecar entries produced by tools like macOS's bsdtar (e.g. ._ns.yaml next to ns.yaml) were matched by extension alone and swept into the manifest as binary garbage. Both are now filtered and decompressed correctly.
PR: addon-controller #1911
RemoteURL Tarball Support and Insecure-Connection Options
PolicyRefs' remoteURL over HTTP/HTTPS now accepts a tarball of multiple YAMsed or plain), not just a single raw document. RemoteURL/RemoteKustomizeURLalso gain two optional fields: plainHTTP to connect to an OCI registry over plain HTTP, and insecureSkipTLSVerify to skip server certificate verification, matching the trust optalready available for Helm OCI registries.
PRs: sveltos #780, addon-controller #1913
⚙️ Improvements
Reduced CRD Watcher Memory Usage
WatchCustomResourceDefinition kept every CustomResourceDefinition on the cluster in a local indexed cache for the life of the process, even though every caller only reads the GVK carried by the event that fires and none of them ever read the cache back. It's now backed by a Reflector that forwards each event straight to the handler and discards it, keeping the same list-then-watch/relist-on-error behavior without the resident memory. Most visible on sveltos-agent, in clusters with many CRDs installed.
PR: libsveltos #659
Helm Drift Redeploy Scoped Per Chart
With syncMode: ContinuousWithDriftDetection deploying multiple Helm charts from one ClusterProfile, drift on a single resource used to mark the entire Helm feature for redeploy and unconditionally upgrade every chart, regardless of whether it had drifted or even changed. Redeploy now checks a per-chart flag first, so a chart that neither drifted nor changed is left alone.
PR: addon-controller #1915
sveltos-agent: Reduced ConfigMap Memory Usage in Agentless Mode
In agentless mode, sveltos-agent's cache already scoped ConfigMaps by cluster namespace, but cached every ConfigMap in that namespace in full, including unrelated Helm-values or raw-YAML ConfigMaps deployed via policyRefs. It's now filtered by name to just the ConfigMap relevant to that cluster.
🐞 Bug Fixes
addon-controller: DeploymentType Local Cleanup on a Self-Managed Cluster
When the managed cluster is a self-managed SveltosCluster (its remote client/config resolve back to the management cluster itself), the remote-cluster stale-resource cleanup pass scanned the management cluster with no per-ClusterSummary scoping, deleting same-GVK resources deployed there by other ClusterSummary instances via deploymentType: Local. Every deployed resource is now tagged with its owning ClusterSummary, and both cleanup passes honor that tag, checking it only as a protective signal so pre-upgrade resources without it still fall through to the existing ownership checks.
PR: addon-controller #1892
sveltos-agent: Watching GVKs From Other Clusters in Agentless Mode
The GVK-watch-building path for Reloaders, EventSources, and HealthChecks listed every instance in the management cluster instead of scoping to the managed cluster an agent instance is responsible for, so an agent could enable ConfigMap/Secret watches or start informers for kinds only relevant to a different cluster's policies. Now filtered against each cluster's own tracking ConfigMap, matching the scoping the reconcilers already applied.
addon-controller: Reloader Not Cleaned Up on ClusterProfile Delete
Deleting, or unmatching, a Helm-based ClusterProfile with spec.reloader: true left its Reloader instance, and in agentless mode its per-cluster ConfigMap entry, behind indefinitely, referencing Deployments from a chart no longer deployed. The undeploy path was tagging the cleanup with the Kustomize feature instead of Helm, so both lookups silently missed. Resources- and Kustomize-based profiles were unaffected.
PR: addon-controller #1898
healthcheck-manager: Stale ClusterHealthCheck Conditions and Notifications
Three compounding bugs kept Status.Conditions and notifications frozen even as the underlying HealthCheckReport kept updating: Spec.HealthCheckName was never populated on HealthCheckReport, the watch mapper relied on that same empty field and resolved to zero ClusterHealthChecks to reconcile, and re-evaluation was skipped for clusters already Provisioned. Also fixes related Discord/Slack formatting bugs.
PR: healthcheck-manager #421
addon-controller: Failed Helm Upgrade Could Silently Uninstall the Release
A failed Helm upgrade still leaves a release record behind, stamped with the version it tried, and failed, to reach. That made the install/upgrade decision misread the release as already handled, and after a couple more failures deleted it and reinstalled from scratch. The decision now checks whether a release exists at all instead of inferring it from version/status, and the install-retry threshold is now its own setting, HelmInstallOptions.RecoverAfterConsecutiveFailures (default 5), instead of being borrowed from Helm's revision-retention MaxHistory.
PR: addon-controller #1912
addon-controller: Four ClusterSummary Reconcile Bugs
A transient, non-NotFound error checking whether a cluster still exists was treated as "cluster present," which could stall a ClusterSummary in Terminating indefinitely. One cluster's stale ResourceSummary cleanup could be blocked by an unrelated cluster's same-named ResourceSummary. Helm chart downloads via LocateChart had no timeout and could wedge a deployer worker permanently. And deletion didn't honor dependsOn ordering, letting a prerequisite and its dependent undeploy concurrently in any order.
PR: addon-controller #1918
Expired Kubeconfig Tokens Not Evicted From Cache
An expired token only surfaces on first actual use of a cached client, not when the client is built, so classifier, event-manager, and healthcheck-manager kept retrying the same stale credentials indefinitely. The client is now evicted from cache on an auth error so the next cycle rebuilds it from a freshly read kubeconfig Secret.
PRs: event-manager #512, classifier #500, healthcheck-manager #424
healthcheck-manager: Notification Events Truncated to Kubernetes' Limit
Notification Event messages could exceed the 1024-character limit Kubernetes enforces on Event messages; they're now truncated to fit.
PR: healthcheck-manager #425
🔧 Maintenance
- All components advanced to Cluster API v1.14.0
- All components advanced to Go 1.26.6
v1.13.0
🚀 New Features
Detect Outdated Helm Chart Versions
Sveltos now periodically checks whether a Helm chart deployed through a ClusterProfile or Profile has a newer version, or a newer same-minor patch, published upstream (HTTP repository or OCI registry). This is detection only: nothing about what gets deployed changes automatically. The result is recorded directly on the ClusterSummary that owns the release, alongside the chart identity already resolved at deploy time, so it's visible per cluster rather than blended across a fleet. latestVersion/latestPatchVersion are only populated when something newer genuinely exists; lastCheckedTime advances on every check regardless of outcome, distinguishing "confirmed up to date" from "not yet checked." A background process (main instance only, to avoid duplicate work across shards) groups deployments that are the same chart from the same source and checks each once; unreachable repositories/registries are skipped and logged rather than blocking the rest, and each check has a timeout. Metrics expose which deployments are behind and the health of the checker itself (last run, failure count). The check interval is configurable and can be disabled.
The dashboard surfaces this in three places: cluster list cards show an icon when a cluster has one or more releases with a newer version or patch available; the addon table on a cluster's detail page gains an "Update Available" column; and a new "Updates" button in the cluster heading queries mcp-server on demand and lists outdated releases with their current and latest available versions.
sveltosctl gets the CLI equivalent: a new show helm-updates command displays, per cluster, every Helm release with a newer version or patch available, filterable by namespace, cluster name, and cluster type. Separately, MCP query failures (as opposed to a successful-but-empty response) now show a distinct "Unable to reach the MCP server" message across all MCP-backed dashboard panels, instead of being indistinguishable from "nothing to report."
PRs: addon-controller #1883, sveltos #772, ui-backend #177, dashboard #180, sveltosctl #442
RemoteURL Support for KustomizationRefs
KustomizationRef.RemoteURL lets a KustomizationRef fetch its content directly from an HTTP/HTTPS endpoint or an OCI registry PolicyRef.RemoteURL. Unlike PolicyRef's OCI path, which flattens all YAML/JSON files across layers into one concatenated blob, this path preserves the fetched content's directory structure, since Kustomize needs it. Namespace/Name/Kind are now optional on KustomizationRef (required only when RemoteURL is unset), and hashing/periodic re-fetch both cover RemoteURL-based refs the same way they already do for PolicyRef.
PR: addon-controller #1882
Instantiate Path/Components Using TemplateResourceRefs
PolicyRef.Path and KustomizationRef.Path/Components can now be Go templates instantiated using TemplateResourceRefs data, not just cluster information. This lets a Kustomize overlay path or PolicyRef directory be selected dynamically based on resources read from the management cluster, instead of being fixed per profile.
PR: addon-controller #1868
Helm Post-Render Strategies
HelmChart.Options gains PostRenderStrategy, controlling whether Helm hooks are included when Patches/PatchesFrom are applied as a post-renderer during install/upgrade: combined (Helm's own default, hooks and templates sent together), separate (hooks and templates sent to the post-renderer in independent invocations, avoiding duplicate-resource errors from post-renderers that de-duplicate by resource identity), or nohooks (only regular templates are post-rendered, matching Helm 3's behavior). Only relevant when Patches or PatchesFrom is set.
PR: addon-controller #1866
EventTrigger Errors Surfaced in EventReport Status
When event-manager processes an EventReport and fails to instantiate the resulting ClusterProfile(s), the error used to be visible only in event-manager's own logs. It's now recorded directly on EventReport.Status, so the failure is visible on the object itself instead of requiring log access.
PR: event-manager #502
mcp-server: Event/Classifier Failure Reporting and Fleet-Wide Label Visibility
Event pipeline tracing now surfaces the error event-manager records when it fails to instantiate a ClusterProfile from an EventReport, so tracing an event-driven deployment tells you directly why no profile was created. Classifier pipeline tracing now also covers ManagementClusterClassifier: tracing by name checks both kinds automatically, reporting cluster match, reconcile failures (bad Lua, resource collection errors), and label ownership/conflicts for either. A new fleet-wide tool lists every label currently being managed by any Classifier or ManagementClusterClassifier across all (or a filtered subset of) managed clusters, along with label ownership and conflicts, mirroring sveltosctl show classifier-labels. Two diagnostic bugs were also fixed: installation health checks reporting agent verification failures as a blank reason, and blocked-dependency names being concatenated with no separator in deployment-error/dependency-analysis output.
The Sveltos MCP Server now exposes 22 tools in total.
PR: mcp-server #63
Dashboard: Classifier View
The dashboard now displays Classifier and ManagementClusterClassifier instances, each showing the number of labels it would set and the number of matching clusters. Selecting an instance lists all matching clusters, inline with the existing Events tab. An "Analyze Pipeline" action calls a new MCP tool that verifies a classifier/cluster pair end to end. The overview section also gains Classifier/ManagementClusterClassifier counts.
PRs: ui-backend #176, dashboard #179
Metrics Across the Fleet
addon-controller, event-manager, and sveltoscluster-manager gain a consistent metrics model, replacing ad hoc or cardinality-unsafe gauges/histograms with properly labeled ones (cluster_type, cluster_namespace, cluster_name, and a *_name label per component's own resource): reconcile duration, reconcile outcome (success/failure with the owning profile/trigger), and last-success timestamp, plus component-specific additions: matching_clusters (addon-controller, event-manager), matching_resources (event-manager), connection_failures and agent_last_heartbeat_timestamp_seconds (sveltoscluster-manager). sveltoscluster-manager also fixes two correctness bugs: a stale Kubernetes-version label combination was never removed after a cluster upgraded, and no metric was ever cleaned up when a cluster was deregistered. All components now correctly prefix metrics with projectsveltos_ and ship the auth-proxy RBAC needed to actually scrape /metrics. A Grafana dashboard is available covering the new metrics.
PRs: sveltoscluster-manager #372, event-manager #503, addon-controller #1872, sveltos #768
Recover From Rejected Updates via Force Recreate
PolicyRef/KustomizationRef gain a Force option. Server-side apply cannot always reconcile a manifest change: if a field was set by API server defaulting (or is owned by an unknown field manager) and the new manifest introduces a conflicting value, the apply is rejected and no patch can clear it (e.g. a Deployment moving to strategy.type: Recreate while the API server's previously-defaulted rollingUpdate is still set). When Force is set and an apply is rejected with an error only a delete+recreate can resolve, the object is now deleted and recreated instead of surfacing the error. Never applies to CustomResourceDefinitions, since deleting one cascades to every instance of it, and conflicts (already handled by retry-in-place) are excluded.
PRs: libsveltos #642, addon-controller #1877
⚙️ Improvements
Shared, Cached Managed-Cluster Clients
The cluster rest.Config/mapper/discovery-client cache moves from addon-controller-local code into lib/clustercache in libsveltos, so addon-controller, classifier, healthcheck-manager, event-manager, and access-manager all share the same cache instead of each building a fresh client (with a live discovery round-trip) on every call, or managing their own cache independently. The shared cache also adds InvalidateOnAuthError, evicting a cluster's cached client (and the underlying workload-identity cache entry) as soon as the API server rejects credentials with 401/403, instead of waiting for TTL expiry or an explicit cluster delete.
PRs: libsveltos #644, classifier #489, healthcheck-manager #417, event-manager [#505](https://github.com/projectsveltos/event-ma...
v1.12.0
🚀 New Features
Supply Chain Verification for Helm Charts
Sveltos can now verify the integrity and origin of a Helm chart before deploying it, with two mechanisms targeting different chart sources. For charts pulled from OCI registries, Sveltos verifies the Cosign signature attached to the chart: a PublicKey provider checks it against a static key stored in a Kubernetes Secret, while a Keyless provider verifies the Fulcio-issued certificate against an expected OIDC issuer/subject and confirms the signing event was recorded in the Rekor transparency log, so the chart must have been signed by a specific pipeline in a specific repository. Both providers support the Sigstore Bundle v0.3 OCI referrer format and fall back to the legacy tag-based signature format. For charts pulled from HTTP repositories, Sveltos verifies the Helm .prov provenance file against a GPG keyring stored in a Secret. In both cases a failed verification blocks the deployment and the reason is recorded on the ClusterSummary status; charts without a verification field deploy as before.
PRs: addon-controller #1842, sveltos #753
Workload Identity Support
SveltosCluster now supports authenticating to a managed cluster using the cloud provider's native workload identity instead of a stored kubeconfig Secret: AWS (IRSA / EKS Pod Identity), GCP (Workload Identity Federation), and Azure (Azure Workload Identity). When configured, Sveltos obtains short-lived credentials directly from the cloud provider, caching them in-process and refreshing proactively before expiry. sveltosctl register cluster has been extended to configure workload identity when registering a cluster.
PRs: libsveltos #636, sveltosctl #434
OCI Support in PolicyRef
RemoteURL in PolicyRef now accepts oci:// URLs in addition to http:// and https://. Sveltos pulls the OCI artifact from the registry on each reconciliation at the configured interval, computes a content hash, and redeploys when the content changes, identical to the existing HTTP polling behavior. Authentication uses the same secretRef field, supporting a bearer token, basic auth, or a custom CA certificate. Both a tar archive (the standard ORAS/Flux format) and a raw YAML/JSON blob are supported as artifact layouts.
PR: addon-controller #1851
Classify Clusters from Management Cluster Resources
Classifier evaluates rules against resources inside each managed cluster, which leaves a gap when the classification signal instead lives on the management cluster itself, such as a Crossplane Composite Resource created when a team orders an addon on an Internal Developer Platform. A new ManagementClusterClassifier resource closes that gap: it watches resources on the management cluster and runs a Lua function that receives the full set of matched resources and returns which managed clusters should be labeled. A ManagementClusterClassifierReport tracks label ownership per classifier/cluster pair, giving the same conflict detection the existing Classifier provides.
PR: classifier #482
sveltosctl: show classifier-labels
A new sveltosctl show classifier-labels command displays the labels that Classifier and ManagementClusterClassifier instances are actively managing on each cluster, along with the name of the instance that owns each label.
PR: sveltosctl #437
⚙️ Improvements
sveltos-agent: Reduced Memory Usage in Agentless Mode
In agentless mode, each sveltos-agent instance runs in the management cluster and is responsible for a single managed cluster, but its controller-runtime cache had no namespace or label restrictions, so every agent's informer held HealthCheckReport, EventReport, and ConfigMap objects for all managed clusters, causing O(N) memory per agent and O(N²) total. The cache is now scoped per agent: HealthCheckReport and EventReport are restricted with a label selector matching the agent's cluster name and type, and the per-cluster ConfigMap is restricted with a field selector on its name.
PR: sveltos-agent #493
addon-controller: Condition-Based Wait for CRD Reapply
When a Helm chart is deployed with UpgradeCRDs: true, the controller used to sleep for 30 seconds after applying the chart's CRD files on every reconcile, regardless of whether the CRDs had actually changed, which was especially costly in ContinuousWithDriftDetection sync mode. The sleep is replaced with a poll against the destination cluster's CRD status, returning as soon as each CRD's Established and NamesAccepted conditions are true, effectively zero delay when CRDs are already established and no more than necessary for a genuine upgrade.
PR: addon-controller #1844
🐞 Bug Fixes
sveltos-agent: Reloader Feature Broken in Agentless Mode
In agentless mode, sveltos-agent watched every Reloader instance in the management cluster instead of only the subset belonging to its own managed cluster, and watched ConfigMap/Secret objects in the management cluster instead of the managed cluster. Both are now scoped correctly.
PR: sveltos-agent #494
addon-controller: Tier Change Not Triggering Takeover
Once a ClusterProfile was managing a chart, its tier was never compared against other conflicting profiles on later reconciliations, so raising a profile's tier above a challenger stuck in FailedNonRetriable never woke the challenger up. Tier comparison now runs on every reconcile even for the profile currently managing the chart, so a higher-tier challenger correctly reclaims it.
PR: addon-controller #1837
access-manager: Configurable Namespace
The Sveltos namespace was hardcoded to projectsveltos; this closes a gap missed in the previous release by allowing it to be configurable, part of the broader effort to remove that limitation across components.
PR: access-manager #348
classifier: Upgrade Blocked by Deleted Clusters
The migration init container that upgrades deprecated ClusterInfo entries into ClassifierReport objects would crash-loop and block the upgrade entirely if a managed cluster had been deleted before the upgrade, since its namespace no longer existed. Entries whose cluster namespace is gone are now logged at debug level and skipped, while all other entries migrate normally.
PR: classifier #480
sveltos-agent: Wait for Informer to Be Synced
In agentless mode, sveltos-agent can restart its internal controller-manager without a pod restart, for example when a managed cluster's kubeconfig token expires or a CRD change is detected. If an EventSource was evaluated in the window before the new informers had synced, it returned an empty result that was misread as zero matching resources, causing event-manager to delete previously created ClusterProfiles. An unsynced informer is now treated as a transient error and the evaluation is retried once the informers have synced.
PR: sveltos-agent #495
🔧 Maintenance
- All components advanced to Cluster API v1.13.3
- All components advanced to Kubernetes client-go v1.36.2
v1.11.1
🐞 Bug Fixes
ClusterProfile deletion no longer stalls (addon-controller #1829 (#1829))
Fixed a bug where deleting a ClusterProfile could get stuck indefinitely. The allMatchingProfilesProcessed check was not skipping other profiles that were themselves already being deleted, causing the controller to wait forever for work that would never complete.
MCP compare-clusters tool reliability (mcp-server #57 (projectsveltos/mcp-server#57))
Corrected several issues in the compare_managed_clusters MCP tool. A not-found cluster now surfaces a clear error instead of silently returning an empty result. Additionally, when a cluster exists but its ClusterConfiguration has not yet been created (resources still deploying), the tool now returns a non-fatal warning in the output rather than returning misleading empty comparison data.
Spurious pending-updates indicator in the UI (ui-backend #171 (projectsveltos/ui-backend#171))
Fixed incorrect cluster status reporting in the UI backend. A cluster was wrongly flagged as having pending updates when a profile matched it and that profile had declared dependencies. The status is now computed correctly in those cases.
v1.11.0
🚀 New Features
Health Checks: Metric-Based Validation
ValidateHealth now supports querying a Prometheus-compatible endpoint as an additional data source. Named scalar values are exposed to the Lua evaluate() function via a metrics table, enabling checks such as "error rate below 5 %". In push mode the endpoint must be reachable from the management cluster; in pull mode the sveltos-applier agent running inside the managed cluster reaches it directly via in-cluster DNS.
PR: addon-controller #1816 (#1816)
Kubernetes Events for Deployment Failures
The addon-controller now raises Kubernetes Warning events to make failure causes immediately visible via kubectl describe or any event-watching tool. Events are emitted for conflicts, missing referenced resources, template instantiation errors, and when the controller gives up after reaching the maximum consecutive failure count.
PR: addon-controller #1815 (#1815)
Dashboard: Cluster Deployment Health Signals
The cluster list now surfaces deployment state at a glance. An amber alert icon appears when one or more profile deployments are failing; a blue clock icon appears when deployment is actively in progress with no failures. The ui-backend computes these signals as O(1) in-memory indexes updated by the existing ClusterSummary watcher, so no per-request scanning is required.
PRs: dashboard #173 (projectsveltos/dashboard#173), ui-backend #167 (projectsveltos/ui-backend#167)
HealthCheck: Surface "No Resources Found" as a Degraded Status
When a HealthCheck's resource selectors matched nothing, the resulting HealthCheckReport was silently empty and sveltosctl show resources showed nothing. The Lua evaluate() function can now detect an empty resources table and return a top-level degraded status (e.g. "No deployments found in namespace metrics"), which is surfaced in the report and in sveltosctl.
PR: sveltos-agent #489 (projectsveltos/sveltos-agent#489)
MCP Server: Classifier Pipeline Analysis Tool
A new tool has been added to the Sveltos MCP server to analyze classifier pipelines, making it easier to inspect and reason about classifier configuration via AI-assisted workflows.
PR: mcp-server #55 (projectsveltos/mcp-server#55)
Shard-Controller: Patch Support
A new --shard-components-config flag accepts the name of a ConfigMap in the Sveltos namespace. The ConfigMap holds one or more patches (JSON6902 or strategic-merge) that are applied to the five Deployments shard-controller creates per shard. Target selectors allow a single ConfigMap to patch only specific components. When the ConfigMap changes, a dedicated reconciler re-deploys all active shards immediately so patches take effect without restarting shard-controller.
PR: shard-controller #216 (projectsveltos/shard-controller#216)
🐞 Bug Fixes
addon-controller: Logging, Pull-Mode Status Handler, Helm Data Race, and Dependency Manager
Fixed a logging mistake where cluster name was populated with the cluster namespace in two places. Corrected the pull-mode agent status handler, which could reach a nil dereference when the agent returned an error without a status payload. Fixed a data race in the Helm chart manager where a shared map was read without holding the mutex. The dependency manager's background update loop now snapshots work, releases the write lock before making API calls, and re-acquires it only to clear completed entries — unblocking concurrent reconcilers. Its startup rebuild loop now waits 5 seconds between retries instead of spinning at full speed on API errors.
PR: addon-controller #1823 (#1823)
sveltos-applier: ClassifierReport Retry, Discovery Loop, and Namespace Client Reuse
ClassifierReports were silently marked as delivered even when the push to the management cluster failed, so they were never rld also continue with an empty Classifier object on non-NotFound API errors. The discovery retry loop in the REST mapperrefresh executed exactly once regardless of the loop variable, causing immediate failure on transient API server slowness. A new Kubernetes client — and with it a new HTTP connection pool — was being created for every single resource that needed a namespace ensured; it is now
created once per reconciliation pass.
PR: sveltos-applier #89 (projectsveltos/sveltos-applier#89)
sveltos-agent: Handful Bugs in the Evaluation Package
Nine bugs in the evaluation package have been corrected.
PR: sveltos-agent #490 (projectsveltos/sveltos-agent#490)
🔧 Maintenance
- All components advanced to Go v1.26.4
- All components advanced to golangci-lint v1.12.1
v1.10.0
🚀 New Features
OIDC Authentication in Dashboard
Users can now log into the Sveltos dashboard using OpenID Connect (Authorization Code Flow with a public client), as an alternative to manual token authentication.
PR: dashboard #160 (projectsveltos/dashboard#160)
🐞 Bug Fixes
EventManager: Ordered Removal of Resources
When stale ClusterProfile resources were being cleaned up by the event-manager, referenced resources could be removed before the ClusterProfile was fully deleted, causing ordering violations. Sveltos now waits for stale ClusterProfiles to be fully deleted before removing their referenced resources.
PR: event-manager #485 (projectsveltos/event-manager#485)
addon-controller: Guaranteed Helm Chart Handoff Between Profiles
When a cluster atomically switched from one ClusterProfile to another (both referencing the same Helm chart), a race condition could cause a delete-and-reinstall instead of an in-place upgrade. Sveltos now verifies that every matching profile has had its ClusterSummary fully processed by the chart manager before allowing an uninstall, avoiding unnecessary downtime.
PR: addon-controller #1780 (#1780)
addon-controller: Surface Errors for Missing Non-Optional TemplateResourceRefs
When a non-optional resource referenced in TemplateResourceRefs was missing, no error was reported in the ClusterSummary, making it difficult to diagnose why a profile was not being deployed. The failure message is now surfaced directly in the ClusterSummary status.
PR: addon-controller #1790 (#1790)
shard-controller: Correct Flags for Init Container in Agentless Mode
When running in agentless mode, the agent-in-mgmt-cluster flag was not being set correctly for the addon-controller init container.