🐛 (fix) Fix race condition in Helm to Boxcutter migration during OLM upgrades #2440

camilamacedo86 · 2026-01-11T18:35:21Z

When upgrading OLM from standard (Helm runtime) to experimental (Boxcutter runtime), the BoxcutterStorageMigrator creates a ClusterExtensionRevision from the existing Helm release. However, the migrated revision was created without status conditions, causing a race condition where it wasn't recognized as "Installed".

This fix sets an initial Succeeded=True status on migrated revisions, ensuring they're immediately recognized and allowing version upgrades to proceed correctly after OLM upgrades

Real-World Scenario

What We're Doing

Day 1: You install OLM standard edition and install PostgreSQL operator v2.0.0

Everything works great ✅
Your databases are running ✅

Day 2: You want to try the new Boxcutter runtime (experimental features)

You upgrade OLM from standard to experimental
OLM upgrade completes successfully ✅
PostgreSQL still runs fine ✅

Day 3: PostgreSQL v2.1.0 is released with bug fixes you need

We try to upgrade PostgreSQL from v2.0.0 → v2.1.0
It FAILS ❌
Error: "Cannot determine installed version"
You're stuck on old version with bugs ❌

What Was Happening (Before Fix)

The Migration Process

When OLM upgrades from Helm to Boxcutter:

Migration starts - OLM needs to convert Helm storage to Boxcutter storage
Creates ClusterExtensionRevision - Copies all your installed manifests
BUT - Forgets to mark it as "successfully installed"
4. Race condition - System checks what's installed before status is set
System thinks - "Nothing is installed yet, this is still rolling out"
Result - Can't compute upgrade path without knowing current version

What's Fixed Now

After the Fix

When OLM upgrades from Helm to Boxcutter:

Migration starts - Same as before
Creates ClusterExtensionRevision - Same as before
✨ NEW: Immediately marks it as succeeded - No waiting!
System checks - "What's installed?"
System knows - "v2.0.0 is installed and working"
Result - Upgrade to v2.1.0 works perfectly!

netlify · 2026-01-11T18:35:26Z

✅ Deploy Preview for olmv1 ready!

Name	Link
🔨 Latest commit	`353a41e`
🔍 Latest deploy log	https://app.netlify.com/projects/olmv1/deploys/697cecaf39fce6000839fc15
😎 Deploy Preview	https://deploy-preview-2440--olmv1.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Copilot

Pull request overview

This pull request fixes a race condition that occurs during the upgrade from standard OLM (Helm runtime) to experimental OLM (Boxcutter runtime). The issue arose because migrated ClusterExtensionRevisions were created without a Succeeded=True status condition, causing them not to be recognized as "Installed" until the ClusterExtensionRevision controller reconciled them. This timing gap led to version resolution failures during OLM upgrades.

Changes:

Added a new ClusterExtensionRevisionReasonMigrated constant for tracking migration status
Set initial Succeeded=True status condition on migrated revisions immediately after creation
Enhanced documentation explaining the race condition and its resolution

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
`api/v1/clusterextensionrevision_types.go`	Added new `ClusterExtensionRevisionReasonMigrated` constant for status condition reasons
`internal/operator-controller/applier/boxcutter.go`	Added status update logic to set `Succeeded=True` condition on migrated revisions with comprehensive documentation

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

codecov · 2026-01-11T18:49:30Z

Codecov Report

❌ Patch coverage is 64.00000% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.48%. Comparing base (4d4f894) to head (353a41e).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
internal/operator-controller/applier/boxcutter.go	64.00%	10 Missing and 8 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2440      +/-   ##
==========================================
- Coverage   69.48%   69.48%   -0.01%     
==========================================
  Files         102      102              
  Lines        8249     8297      +48     
==========================================
+ Hits         5732     5765      +33     
- Misses       2063     2071       +8     
- Partials      454      461       +7

Flag	Coverage Δ
e2e	`46.33% <0.00%> (-0.54%)`	⬇️
experimental-e2e	`13.30% <0.00%> (-0.09%)`	⬇️
unit	`57.56% <64.00%> (+0.04%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

internal/operator-controller/applier/boxcutter.go

internal/operator-controller/applier/boxcutter_test.go

When upgrading OLM from standard (Helm runtime) to experimental (Boxcutter runtime), the BoxcutterStorageMigrator creates a ClusterExtensionRevision from the existing Helm release. However, the migrated revision was created without status conditions, causing a race condition where it wasn't recognized as "Installed". Assisted-by: CLAUDE

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

internal/operator-controller/applier/boxcutter_test.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

internal/operator-controller/applier/boxcutter.go

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-01-30T17:29:52Z

internal/operator-controller/applier/boxcutter_test.go

+		client.
+			On("List", mock.Anything, mock.AnythingOfType("*v1.ClusterExtensionRevisionList"), mock.Anything).
+			Run(func(args mock.Arguments) {
+				list := args.Get(1).(*ocv1.ClusterExtensionRevisionList)
+				list.Items = []ocv1.ClusterExtensionRevision{existingRev}


This test sets up List() but does not mock client.Get(). BoxcutterStorageMigrator.Migrate() currently calls Client.Get() when revisions exist and revision 1 is not already Succeeded=True, so this will trigger an unexpected mock call and fail the test. Either add a Get() expectation here or change ensureMigratedRevisionStatus to skip non-migrated revision 1 before calling ensureRevisionStatus (so no Get() occurs).

pedjak

/lgtm

tmshort · 2026-01-30T19:34:23Z

/approve

openshift-ci · 2026-01-30T19:34:29Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tmshort

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [tmshort]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

camilamacedo86 · 2026-01-31T00:04:10Z

/override codecov/patch

openshift-ci · 2026-01-31T00:04:17Z

@camilamacedo86: Overrode contexts on behalf of camilamacedo86: codecov/patch

Details

In response to this:

/override codecov/patch

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Copilot AI review requested due to automatic review settings January 11, 2026 18:35

openshift-ci bot requested review from ankitathomas and joelanford January 11, 2026 18:35

Copilot started reviewing on behalf of camilamacedo86 January 11, 2026 18:35 View session

camilamacedo86 changed the title ~~🐛 (fix) Helm to Boxcutter migration during OLM upgrade~~ WIP 🐛 (fix) Helm to Boxcutter migration during OLM upgrade Jan 11, 2026

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 11, 2026

Copilot AI reviewed Jan 11, 2026

View reviewed changes

camilamacedo86 changed the title ~~WIP 🐛 (fix) Helm to Boxcutter migration during OLM upgrade~~ 🐛 (fix) Helm to Boxcutter migration during OLM upgrade Jan 11, 2026

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 11, 2026

camilamacedo86 requested review from pedjak and perdasilva January 11, 2026 18:44

camilamacedo86 requested review from Copilot and removed request for ankitathomas, Copilot, joelanford, pedjak and perdasilva January 12, 2026 00:23

camilamacedo86 changed the title ~~🐛 (fix) Helm to Boxcutter migration during OLM upgrade~~ WIP 🐛 (fix) Helm to Boxcutter migration during OLM upgrade Jan 12, 2026

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 12, 2026

Copilot started reviewing on behalf of camilamacedo86 January 12, 2026 00:23 View session

camilamacedo86 closed this Jan 12, 2026

camilamacedo86 reopened this Jan 29, 2026

camilamacedo86 force-pushed the fix-only-migration branch from 3f3514f to 46c7962 Compare January 29, 2026 16:42

camilamacedo86 changed the title ~~WIP 🐛 (fix) Helm to Boxcutter migration during OLM upgrade~~ 🐛 (fix) Helm to Boxcutter migration during OLM upgrade Jan 29, 2026

openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 29, 2026

camilamacedo86 changed the title ~~🐛 (fix) Helm to Boxcutter migration during OLM upgrade~~ 🐛 (fix) Fix race condition in Helm to Boxcutter migration during OLM upgrades Jan 29, 2026

camilamacedo86 requested review from pedjak and perdasilva January 29, 2026 16:48

camilamacedo86 force-pushed the fix-only-migration branch 2 times, most recently from 932f0af to 7ed8d21 Compare January 30, 2026 16:33

Copilot AI review requested due to automatic review settings January 30, 2026 16:33

Copilot started reviewing on behalf of camilamacedo86 January 30, 2026 16:34 View session

camilamacedo86 force-pushed the fix-only-migration branch from 7ed8d21 to c07a0fa Compare January 30, 2026 16:35

Copilot AI reviewed Jan 30, 2026

View reviewed changes

internal/operator-controller/applier/boxcutter.go Outdated Show resolved Hide resolved

internal/operator-controller/applier/boxcutter.go Outdated Show resolved Hide resolved

internal/operator-controller/applier/boxcutter_test.go Show resolved Hide resolved

camilamacedo86 force-pushed the fix-only-migration branch from c07a0fa to 2a26e0c Compare January 30, 2026 16:49

camilamacedo86 force-pushed the fix-only-migration branch from 2a26e0c to bdc017a Compare January 30, 2026 16:51

Copilot AI review requested due to automatic review settings January 30, 2026 16:51

Copilot started reviewing on behalf of camilamacedo86 January 30, 2026 16:53 View session

Copilot AI reviewed Jan 30, 2026

View reviewed changes

internal/operator-controller/applier/boxcutter_test.go Outdated Show resolved Hide resolved

Update internal/operator-controller/applier/boxcutter_test.go

aed04b4

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings January 30, 2026 17:06

Copilot started reviewing on behalf of camilamacedo86 January 30, 2026 17:07 View session

Copilot AI reviewed Jan 30, 2026

View reviewed changes

internal/operator-controller/applier/boxcutter.go Outdated Show resolved Hide resolved

Update internal/operator-controller/applier/boxcutter.go

d9cc479

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings January 30, 2026 17:20

Copilot started reviewing on behalf of camilamacedo86 January 30, 2026 17:21 View session

Copilot AI reviewed Jan 30, 2026

View reviewed changes

Add test requested by copilot

353a41e

pedjak reviewed Jan 30, 2026

View reviewed changes

openshift-ci bot assigned pedjak Jan 30, 2026

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jan 30, 2026

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 30, 2026

openshift-merge-bot bot merged commit 492c7e5 into operator-framework:main Jan 31, 2026
29 of 30 checks passed

🐛 (fix) Fix race condition in Helm to Boxcutter migration during OLM upgrades #2440

🐛 (fix) Fix race condition in Helm to Boxcutter migration during OLM upgrades #2440

Conversation

camilamacedo86 commented Jan 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Real-World Scenario

What We're Doing

What Was Happening (Before Fix)

The Migration Process

What's Fixed Now

After the Fix

Uh oh!

netlify bot commented Jan 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for olmv1 ready!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

codecov bot commented Jan 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Jan 30, 2026

Choose a reason for hiding this comment

Uh oh!

pedjak left a comment

Choose a reason for hiding this comment

Uh oh!

tmshort commented Jan 30, 2026

Uh oh!

openshift-ci bot commented Jan 30, 2026

Uh oh!

camilamacedo86 commented Jan 31, 2026

Uh oh!

openshift-ci bot commented Jan 31, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

camilamacedo86 commented Jan 11, 2026 •

edited

Loading

netlify bot commented Jan 11, 2026 •

edited

Loading

codecov bot commented Jan 11, 2026 •

edited

Loading