[paimon] Not bundle paimon in fluss-lake-paimon to avoid class conflict #2531
Conversation
There was a problem hiding this comment.
Pull request overview
This pull request refactors the fluss-lake-paimon module to provide a "thin" JAR that no longer bundles the Paimon dependency. This change addresses the class conflict issues that occurred when users had different versions of Paimon in their environment, as described in issue #2426.
Changes:
- Converted
fluss-lake-paimonto a thin JAR by markingpaimon-bundleandfluss-commondependencies asprovidedscope - Removed the Maven Shade plugin configuration that previously bundled Paimon classes
- Updated build configuration to include
paimon-bundleas a separate JAR in the plugin distribution - Added comprehensive documentation updates explaining version compatibility, migration steps, and new deployment requirements
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| fluss-lake/fluss-lake-paimon/pom.xml | Changed dependency scopes to provided and removed shading configuration for thin JAR approach |
| fluss-dist/pom.xml | Added paimon-bundle as compile dependency for plugin distribution |
| fluss-dist/src/main/assemblies/plugins.xml | Updated assembly to explicitly include paimon-bundle without transitive dependencies |
| website/docs/streaming-lakehouse/integrate-data-lakes/paimon.md | Added version compatibility table and updated prerequisites to include paimon-bundle download |
| website/docs/quickstart/flink.md | Added paimon-bundle download step to the quickstart guide |
| website/docs/maintenance/tiered-storage/lakehouse-storage.md | Added instruction to download and place paimon-bundle JAR |
| website/docs/maintenance/operations/upgrade-notes-0.9.md | Added migration guide for users upgrading from v0.8, with compatibility information and deployment steps |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
luoyuxia
force-pushed
the
make-lake-thin
branch
from
d1aa334 to
ffa06d3
Compare
luoyuxia
force-pushed
the
make-lake-thin
branch
from
ffa06d3 to
6a410ef
Compare
|
|
||
| ##### Prerequisites | ||
| Download the [fluss-lake-paimon-$FLUSS_VERSION$.jar](https://repo1.maven.org/maven2/org/apache/fluss/fluss-lake-paimon/$FLUSS_VERSION$/fluss-lake-paimon-$FLUSS_VERSION$.jar), and place it into `${FLINK_HOME}/lib`. | ||
| Download the [fluss-lake-paimon-$FLUSS_VERSION$.jar](https://repo1.maven.org/maven2/org/apache/fluss/fluss-lake-paimon/$FLUSS_VERSION$/fluss-lake-paimon-$FLUSS_VERSION$.jar) and [paimon-bundle jar](https://repo.maven.apache.org/maven2/org/apache/paimon/paimon-bundle/1.3.1/paimon-bundle-1.3.1.jar), and place it into `${FLINK_HOME}/lib`. |
There was a problem hiding this comment.
One question — I can see that the Paimon version for the downloads are currently hardcoded to the 1.3.1 release. Would it be useful to store this in a separate variable to better support future changes (e.g. PAIMON_VERSION)?
rionmonster
left a comment
rionmonster
left a comment
There was a problem hiding this comment.
Left one comment, which isn't at all required. Otherwise, looks good. Approving!
luoyuxia
changed the title
|
@wuchong Could you please review it when you got some time? |
| "$PAIMON_VERSION$": "1.3.1", | ||
| "$PAIMON_VERSION_SHORT$": "1.3" |
There was a problem hiding this comment.
We shouldn’t hard-code the Paimon version in index.js, as this file is shared across all versioned documentation sites.
For example, if we release Fluss 0.9 with Paimon 1.3.1, and later upgrade to Paimon 1.4.1 in a subsequent Fluss version, the shared index.js would incorrectly update the 0.9 documentation to reference Paimon 1.4.1—even though that combination was never tested and may be incompatible.
To ensure version accuracy and isolation, we should move such version-specific variables into fluss-versions.json, where they can be properly scoped per Fluss release.
There was a problem hiding this comment.
Thank you for the explantion!
3 participants
Purpose
Linked issue: close #2426
Brief change log
paimon-bundiejar influss-lake-paimonTests
verify it in local env manually.
API and Format
Documentation