Skip to content

Fix ImportError when importing SUPERVISOR_COMMS outside task context#61630

Open
andreahlert wants to merge 2 commits intoapache:mainfrom
andreahlert:fix/supervisor-comms-import-error
Open

Fix ImportError when importing SUPERVISOR_COMMS outside task context#61630
andreahlert wants to merge 2 commits intoapache:mainfrom
andreahlert:fix/supervisor-comms-import-error

Conversation

@andreahlert
Copy link
Contributor

@andreahlert andreahlert commented Feb 8, 2026

Summary

Fixes #51816

SUPERVISOR_COMMS was declared as a bare type annotation (SUPERVISOR_COMMS: CommsDecoder[...]) in task_runner.py, which does not create an actual module attribute in Python. This caused ImportError when Variable.get() was called at the top level of a DAG file (outside task execution context), such as during dag.test().

Changes

  • Initialize SUPERVISOR_COMMS to None so the import always succeeds. The existing check in ensure_secrets_backend_loaded() already handles is not None, so the fallback logic is unaffected.
  • Add None guards before .send() calls in _set_variable(), _delete_variable(), and all ExecutionAPISecretsBackend methods to prevent AttributeError on None.
  • Improve error messages when Variable.get() fails outside task context, suggesting alternatives (environment variables, Jinja templates, moving the call inside a task).
  • Add tests for Variable.get/set/delete outside task execution context.

Files changed

File Change
task_runner.py SUPERVISOR_COMMS = None (was bare annotation)
context.py None guards + enhanced error messages
execution_api.py Defensive return None guards
test_variables.py 4 new tests for outside-task-context behavior

Test plan

  • Existing test_variables.py tests pass (no regression)
  • New TestVariableOutsideTaskContext tests pass
  • Variable.get() with env var works outside task context
  • Variable.get() without env var raises helpful error outside task context
  • Variable.set() raises clear error outside task context
  • Variable.delete() raises clear error outside task context

SUPERVISOR_COMMS was declared as a bare type annotation without
assignment, which does not create an actual module attribute in Python.
This caused ImportError when Variable.get() was called at the top level
of a DAG file (outside task execution context), such as during
dag.test().

Initialize SUPERVISOR_COMMS to None so the import always succeeds, add
None guards before .send() calls in _set_variable, _delete_variable,
and ExecutionAPISecretsBackend methods, and provide helpful error
messages suggesting alternatives like environment variables or Jinja
templates.

Closes: apache#51816
Copy link
Contributor

@SameerMesiah97 SameerMesiah97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This fix looks reasonable in isolation, but several provider tests appear to have relied on the previous permissive behavior when SUPERVISOR_COMMS was absent or not initialized. By normalizing it to None, execution-time paths (particularly around connection resolution) are now enforced and fail loudly, which is causing widespread provider CI failures. It seems those tests did not anticipate SUPERVISOR_COMMS being present but None and therefore entering these stricter code paths. It appears that provider tests might have to be loosened (or maybe connection resolution migh have to check for SUPERVISOR_COMMS) to accomodate this change but I am not going to make such a sweeping suggestion unilaterally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ImportError: cannot import name 'SUPERVISOR_COMMS' with dag.test()

2 participants