Add delete_pods_in_phase override to KubernetesPodOperator#61637
Open
SameerMesiah97 wants to merge 1 commit intoapache:mainfrom
Open
Add delete_pods_in_phase override to KubernetesPodOperator#61637SameerMesiah97 wants to merge 1 commit intoapache:mainfrom
SameerMesiah97 wants to merge 1 commit intoapache:mainfrom
Conversation
Add delete_pods_in_phase as a power-user option to delete pods based on their Kubernetes phase. When set, this explicitly overrides on_finish_action and deletes the pod only if its phase matches one of the configured values. Includes input validation and unit tests covering both validation and phase-based deletion behavior.
734928e to
fabae54
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This change introduces a new optional parameter,
delete_pods_in_phase, toKubernetesPodOperatorthat allows pod deletion to be controlled explicitly by Kubernetes pod phase.When
delete_pods_in_phaseis provided, the operator deletes a pod only if its current phase matches one of the configured values. In this mode, pod deletion is determined solely by phase and takes precedence overon_finish_action. This behavior is applied consistently across all pod-cleanup paths, including normal execution, failure handling, and duplicate-label pod resolution.When the override is enabled, the operator emits an informational log message during initialization to make the precedence over
on_finish_actionexplicit in task logs.Rationale
Recent changes to
on_finish_actionhave introduced or proposed additional pod-deletion modes, such asdelete_active_pod(PR #59160) anddelete_non_failed_pod(PR #59413), to support more nuanced cleanup behavior.Rather than continuing to add new high-level
on_finish_actionvalues for each possible combination of pod phases, this change introduces a more general and explicit mechanism for advanced users. Thedelete_pods_in_phaseparameter allows users to specify exactly which Kubernetes pod phases should trigger deletion, avoiding ambiguity and removing the need to debate or encode specific phase combinations into new modes.This approach maximizes configurability while preserving existing semantics and keeping default behavior unchanged.
Documentation
A docstring has been added to
process_pod_deletiondescribing the deletion decision flow and documenting the precedence ofdelete_pods_in_phaseoveron_finish_action.Notes
delete_pods_in_phase, ensuring consistent cleanup behavior when multiple pods share identical task labels.Tests
Added unit tests that:
delete_pods_in_phaseoverrideson_finish_actionwhen determining whether a pod should be deleted.delete_pods_in_phasevalues.Backwards Compatibility
When
delete_pods_in_phaseis not provided, pod deletion behavior remains unchanged and continues to be governed exclusively byon_finish_action. No existing defaults are modified, and no public APIs are removed or altered besides the addition of thedelete_pods_in_phaseparameter.