Skip to content

feat: impl handle_child_pushdown_result for UnionExec#20145

Open
haohuaijin wants to merge 8 commits intoapache:mainfrom
haohuaijin:union-filter-pushdown
Open

feat: impl handle_child_pushdown_result for UnionExec#20145
haohuaijin wants to merge 8 commits intoapache:mainfrom
haohuaijin:union-filter-pushdown

Conversation

@haohuaijin
Copy link
Contributor

@haohuaijin haohuaijin commented Feb 4, 2026

Which issue does this PR close?

Rationale for this change

see #20144

What changes are included in this PR?

This PR impl handle_child_pushdown_result for UnionExec, for any case, the filter will always pushdown to UnionExec

UnionExec needs specialized filter pushdown handling when children have heterogeneous pushdown support. Without this, when some children support pushdown and others don't, the default behavior would leave FilterExec above UnionExec, re-applying filters to outputs of all children—including those that already applied the filters via pushdown. This specialized implementation adds FilterExec only to children that don't support pushdown, avoiding redundant filtering and improving performance.

Example: Given Child1 (no pushdown support) and Child2 (has pushdown support)
Default behavior:          This implementation:
FilterExec                 UnionExec
  UnionExec                  FilterExec
    Child1                     Child1
    Child2(filter)           Child2(filter)

Are these changes tested?

yes, add two test cases

Are there any user-facing changes?

@github-actions github-actions bot added core Core DataFusion crate physical-plan Changes to the physical-plan crate labels Feb 4, 2026
@github-actions github-actions bot added the sqllogictest SQL Logic Tests (.slt) label Feb 4, 2026
@haohuaijin haohuaijin requested a review from adriangb February 4, 2026 15:51
@haohuaijin
Copy link
Contributor Author

haohuaijin commented Feb 4, 2026

Thanks for your reviews @adriangb , i add the slt test case for reproduce.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate physical-plan Changes to the physical-plan crate sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pushdown filter though UnionExec when some child support pushdown and some child do not support

2 participants