Added functionality to list variable keys w optional prefix filter via task-sdk

[justinpakzad]

related: #61166

Opened this PR to implement a way to list variable keys (optionally by prefix) via the task SDK. The ask in the issue was to support fetching variables where the keys might not be known beforehand.

I've implemented a list_keys() method in the Variable class with an optional prefix parameter. Example: Variable.list_keys() or Variable.list_keys(prefix="api_")

There were some architectural decisions I was unsure about. For example, whether a method like list_keys() should be part of the Variable class or if it should live elsewhere, given that variable represents a single k/v pair. Also, the existing router dependency expects a key as a path parameter, but that wouldn't work here, meaning I needed to move that to the function level and exclude it from the list keys endpoint. Lastly, there could be a path conflict with the existing /{variable_key:path} route. For now I just used /keys/list, but that's probably not ideal (maybe a separate router?).

Anyways, opening this up for a discussion.

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: [Claude Sonnet 4.5] following the guidelines

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

[potiuk]

The issue is that none of the secrets backend actually suports list_variable_keys, and I am not sure if they should - currently the limitation of Airflow is that it can only show variables in the UI that are defined in DB - and it does not list any of the variables or connections from remote backends - becuase the contract for the secret backend is to get the key from it - none of the "contracts" in those backend specify listing variables or connection.

It's possible that we might want to change it and have some (or all) of the backends to actually support listing keys - but that is a bigger change and I think if we decide to do it - it should at least be discussed in the devlist - where we could make decision about the scope of it and how it would impact exposing keys also in Airflow UI and CLIs.

[justinpakzad]

The issue is that none of the secrets backend actually suports list_variable_keys, and I am not sure if they should - currently the limitation of Airflow is that it can only show variables in the UI that are defined in DB - and it does not list any of the variables or connections from remote backends - becuase the contract for the secret backend is to get the key from it - none of the "contracts" in those backend specify listing variables or connection.

It's possible that we might want to change it and have some (or all) of the backends to actually support listing keys - but that is a bigger change and I think if we decide to do it - it should at least be discussed in the devlist - where we could make decision about the scope of it and how it would impact exposing keys also in Airflow UI and CLIs.

Yea, that makes sense. As I was working on this I realized it's not as simple as just adding a way to "list keys", hence why I stated some of my doubts in the description lol. I guess a discussion on if or how something like this could be supported would be interesting, but yea that might something for the future.