Maintain previously resolved manifests when bundle resolution fails #3042

Summary

When bundle resolution fails, continue maintaining and progressing content that was successfully resolved earlier, regardless of the failure reason. Do not apply the current ClusterExtension generation's .spec.config to a bundle resolved under an earlier generation. Maintenance of existing content must not be mistaken for successful resolution of the current request.

Current behavior

handleResolutionError uses catalog existence and a literal comparison of the requested version string with the installed version to decide whether to retry or fall back. If no matching catalog exists, it selects the installed bundle and returns no error. A subsequent successful apply can report Progressing=Succeeded, hiding the resolution failure and foregoing an error-driven retry.

Fallback also does not guarantee preservation of the previously rendered manifests. UnpackBundle still attempts to pull the installed bundle's image. If the pull succeeds, the applier receives bundle content together with the current ClusterExtension spec. Thus, a bundle resolved and rendered at generation 1 can be re-rendered with generation 2's config even though resolution failed at generation 2. The existing stored-manifest maintenance paths are taken only when contentFS is nil.

The active-rollout path skips fresh resolution but also proceeds through image pull and apply with the latest spec. It needs to preserve the already-resolved rollout's manifests when a newer generation changes render-dependent config.

Relevant code:

• Resolution failure and fallback
• Pull after fallback
• Active-rollout path
• Helm stored-release maintenance and Boxcutter stored-revision maintenance

Expected behavior

• If an active rollout has already been resolved, continue progressing its persisted revision and manifests. Do not replace it with an older installed revision or re-render it with newer config.
• Otherwise, if an installed revision exists and resolution fails, maintain its persisted manifests regardless of the error cause, catalog existence, or whether an upgrade or config change was requested. Do not create a new revision or apply the unresolved generation's config.
• If there is no previously resolved content, report the resolution failure without attempting an install.
• When resolution was attempted and failed, keep that failure visible in status even if maintenance succeeds. Retain appropriate retry or terminal-error behavior; report maintenance failures as well. Once resolution succeeds, render and apply the then-current desired config normally.

Suggested approach

Make "maintain previously resolved content" an explicit reconcile path. On resolution failure, choose the appropriate persisted revision and use its stored manifests, rather than pulling and re-rendering an old bundle with the current spec. Carry the resolution error through maintenance so successful maintenance cannot overwrite it with Progressing=Succeeded. Remove catalog existence and literal version comparison from the generic maintenance decision.

Do not simply return on resolution error. PR #2439 (implementation commit 5652b14, merged as 6e4f192) introduced workload resilience after catalog deletion. Preserve its key guarantees: existing resources self-heal, Helm corrects drift from the stored release, Boxcutter maintains its stored revision, and an already-started rollout continues. Review discussion explicitly calls for resolution failures not to stop reconciliation of managed objects.

Acceptance tests

1. Install with generation 1 config. Change .spec.config at generation 2 and fail resolution while the old image remains pullable. Verify the persisted manifests do not change, no new revision is created, existing resources are maintained, and the resolution failure is visible. Restore resolution and verify generation 2 config is applied. Cover Helm and Boxcutter.
2. Repeat with the old image unavailable. Stored-manifest maintenance and drift recovery must still work.
3. Request a different bundle version while resolution fails. Maintain prior content without installing the requested version, and report the unresolved request.
4. Change .spec.config while a previously resolved revision is rolling out. Verify that the rollout continues with its persisted manifests.
5. With no previously resolved content, report the resolution failure without attempting an install.

Status compatibility to review

The catalog-deletion e2e scenario currently expects Progressing=Succeeded after a preflight-only spec update while the catalog is absent. This expectation must be reviewed and changed deliberately if failed resolution remains visible as Progressing=Retrying; the scenario does not establish that new .spec.config can safely be applied without resolution. The RFC linked by PR #2439 also illustrates Installed=True alongside Progressing=Retrying during maintenance of previously installed content.