Packaging: Add basic BUILD metadata for rpm/deb packages #6325

cognifloyd · 2025-03-27T18:08:31Z

This PR is working towards doing packaging via pantsbuild. Eventually, I hope to archive and stop using st2-packages.git.

The pants nfpm backend was activated in #6321. This PR adds an nfpm_deb_package target and an nfpm_rpm_package target with basic metadata like packager, package description, homepage, etc. I added review comments to address each piece of this metadata, where it came from in st2-packages, and what documentation is relevant for this BUILD target field.

st2/packaging/BUILD

Lines 33 to 34 in 2ef489b

    
           nfpm_deb_package( 
        
               name="st2.deb",

st2/packaging/BUILD

Lines 61 to 62 in 2ef489b

    
           nfpm_rpm_package( 
        
               name="st2.rpm",

Beyond this metdata, I also included the deb maintainer scripts and rpm scriptlets--the pre-install, post-install, pre-remove, and post-remove scripts--added in these PRs:

To add them, the scripts have to be in dependencies (deps on other BUILD targets--package deps will be in a later PR), and the script has to be registered in the scripts field:

st2/packaging/BUILD

Lines 35 to 46 in 2ef489b

    
           dependencies=[ 
        
               "./deb/scripts", 
        
           ], 
        
           scripts=dict( 
        
               preinstall="deb/scripts/pre-install.sh", 
        
               postinstall="deb/scripts/post-install.sh", 
        
               preremove="deb/scripts/pre-remove.sh", 
        
               postremove="deb/scripts/post-remove.sh", 
        
               # config="", 
        
               # templates="", 
        
               # rules="", 
        
           ),

st2/packaging/BUILD

Lines 63 to 74 in 2ef489b

    
           dependencies=[ 
        
               "./rpm/scripts", 
        
           ], 
        
           scripts=dict( 
        
               preinstall="rpm/scripts/pre-install.sh", 
        
               postinstall="rpm/scripts/post-install.sh", 
        
               preremove="rpm/scripts/pre-remove.sh", 
        
               postremove="rpm/scripts/post-remove.sh", 
        
               # pretrans="", 
        
               # posttrans="", 
        
               # verify="", 
        
           ),

The scripts field is documented here:

Note that there are 3 more possible deb-specific scripts (config, templates, rules) and 3 more possible rpm-specific scripts (pretrans, posttrans, verify). I left a comment so that we can add those later if we have a reason to do so.

File Triggers

This section is informational. It is related to the deb maintainer scripts registered in this PR, so I mention it in this PR even though this PR does not implement file triggers.

In deb packages, triggers would tell dpkg to run our post-install script when, for example, the python binary is updated which would allow the post-install script to rebuild the venv. However, nFPM currently does not have support for that for rpm. For rpm, the underlying rpmpack library needs to be updated to support adding file triggers. To that end, I filed this feature request:

Feature request: Support for file triggers google/rpmpack#99

As I mentioned in that issue, I can probably work on adding that to rpmpack at some point, but I need to finish the rest of the pants+packaging stuff for st2 first.

…rgets This should also be included in the output_path and possibly other fields.

cognifloyd · 2025-03-27T18:12:05Z

packaging/BUILD

+_pkg_description = """
+StackStorm Event-driven automation
+Package is full standalone st2 installation including all components
+in a pre-built venv.
+"""


From:

deb: https://github.com/StackStorm/st2-packages/blob/d4d2d8dfdf1c88412e5d58635adb87da9c671952/packages/st2/debian/control#L20-L22

rpm:

https://github.com/StackStorm/st2-packages/blob/d4d2d8dfdf1c88412e5d58635adb87da9c671952/packages/st2/rpm/st2.spec#L56

https://github.com/StackStorm/st2-packages/blob/d4d2d8dfdf1c88412e5d58635adb87da9c671952/packages/st2/rpm/st2.spec#L59-L61

See:

https://www.pantsbuild.org/stable/reference/targets/nfpm_deb_package#description

https://www.pantsbuild.org/stable/reference/targets/nfpm_rpm_package#description

Also, the first line of the description is used for summary:

https://www.pantsbuild.org/stable/reference/targets/nfpm_deb_package#summary

https://www.pantsbuild.org/stable/reference/targets/nfpm_rpm_package#summary

cognifloyd · 2025-03-27T18:14:55Z

packaging/BUILD

+_common_pkg_metadata = dict(
+    package_name="st2",
+    description=_pkg_description,
+    homepage="https://stackstorm.com",


From:

deb: https://github.com/StackStorm/st2-packages/blob/d4d2d8dfdf1c88412e5d58635adb87da9c671952/packages/st2/debian/control#L11

rpm: https://github.com/StackStorm/st2-packages/blob/d4d2d8dfdf1c88412e5d58635adb87da9c671952/rpmspec/st2pkg_toptags.spec#L17
(I used stackstorm.com instead of the github link because that makes more sense)

See:

https://www.pantsbuild.org/stable/reference/targets/nfpm_deb_package#homepage

https://www.pantsbuild.org/stable/reference/targets/nfpm_rpm_package#homepage

cognifloyd · 2025-03-27T18:18:18Z

packaging/BUILD

+    # https://jfearn.fedorapeople.org/en-US/RPM/4/html-single/RPM_Guide/index.html#idp3030720
+    license="Apache-2.0",  # TODO: nFPM is putting this under Copyright tag instead of License


From:

deb: https://github.com/StackStorm/st2-packages/blob/d4d2d8dfdf1c88412e5d58635adb87da9c671952/packages/st2/debian/copyright#L14-L28

rpm: https://github.com/StackStorm/st2-packages/blob/d4d2d8dfdf1c88412e5d58635adb87da9c671952/rpmspec/st2pkg_toptags.spec#L16
(I used the SPDX identifier instead of the non-standard "ASL 2.0")

See:

https://www.pantsbuild.org/stable/reference/targets/nfpm_deb_package#license

https://www.pantsbuild.org/stable/reference/targets/nfpm_rpm_package#license

cognifloyd · 2025-03-27T18:21:10Z

packaging/BUILD

+    # arch used to be "any", but that was not correct as the venv has compiled packages.
+    arch="amd64",  # NOTE: parametrize this if adding support for arm64 or other arch.


From:

deb: https://github.com/StackStorm/st2-packages/blob/d4d2d8dfdf1c88412e5d58635adb87da9c671952/packages/st2/debian/control#L16
(I used amd64 instead of any as explained in the comment. I can remove the comment if needed.)

rpm: not defined in st2-packages.git

Also, this uses the ARCH defined in golang which gets translated to the package-specific architecture. So, amd64 becomes x86_64 for rpm, but remains amd64 for deb. This page shows how that translation happens (though the doc is slightly incorrect here as amd64 does not get translated to x86_64 for deb files): https://nfpm.goreleaser.com/goarch-to-pkg/

See:

https://www.pantsbuild.org/stable/reference/targets/nfpm_deb_package#arch

https://www.pantsbuild.org/stable/reference/targets/nfpm_rpm_package#arch

https://nfpm.goreleaser.com/goarch-to-pkg/

cognifloyd · 2025-03-27T18:21:39Z

packaging/BUILD

+    version="",  # injected by pants-plugins/release
+    # arch used to be "any", but that was not correct as the venv has compiled packages.
+    arch="amd64",  # NOTE: parametrize this if adding support for arm64 or other arch.
+    platform="linux",


This is just needed for nfpm. I included it here to be explicit instead of relying on the default value linux. 🤷 We can drop this line if anyone doesn't like it here.

See:

https://www.pantsbuild.org/stable/reference/targets/nfpm_deb_package#platform

https://www.pantsbuild.org/stable/reference/targets/nfpm_rpm_package#platform

cognifloyd · 2025-03-27T21:15:13Z

packaging/BUILD

+        "Vcs-Git": "git://github.com/stackstorm/st2.git",
+        "Vcs-Browser": "https://github.com/stackstorm/st2",
+    },
+    section="python",


From (deb): https://github.com/StackStorm/st2-packages/blob/d4d2d8dfdf1c88412e5d58635adb87da9c671952/packages/st2/debian/control#L2

See: https://www.pantsbuild.org/stable/reference/targets/nfpm_deb_package#section

cognifloyd · 2025-03-27T21:16:15Z

packaging/BUILD

+        "Vcs-Browser": "https://github.com/stackstorm/st2",
+    },
+    section="python",
+    priority="optional",


From (deb): https://github.com/StackStorm/st2-packages/blob/d4d2d8dfdf1c88412e5d58635adb87da9c671952/packages/st2/debian/control#L3

See: https://www.pantsbuild.org/stable/reference/targets/nfpm_deb_package#priority

cognifloyd · 2025-03-27T21:17:05Z

packaging/BUILD

+def _distro(distro_id: str, **kwargs):
+    return parametrize(
+        distro_id,
+        distro_id=distro_id,


Here's the distro_id field added in #6324.

cognifloyd · 2025-03-27T21:21:32Z

packaging/BUILD

+        # posttrans="",
+        # verify="",
+    ),
+    vendor="The StackStorm Project",


This is new. It was not in st2-packages.git. It is not used in Fedora, so we don't have to include it.

See: https://www.pantsbuild.org/stable/reference/targets/nfpm_rpm_package#vendor

cognifloyd · 2025-03-27T21:26:17Z

packaging/BUILD

+    ),
+    vendor="The StackStorm Project",
+    packager=_maintainer,
+    # group="System/Management",  # was only useful for EL 5 and earlier


From (rpm): https://github.com/StackStorm/st2-packages/blob/d4d2d8dfdf1c88412e5d58635adb87da9c671952/rpmspec/st2pkg_toptags.spec#L15

We no longer support EL 5 or earlier, so we should probably just drop this line instead of commenting it out. Right?

Suggested change

# group="System/Management", # was only useful for EL 5 and earlier

See: https://www.pantsbuild.org/stable/reference/targets/nfpm_rpm_package#group

cognifloyd added 5 commits March 27, 2025 10:16

Packaging: add rpm/deb package metadata

a067617

pants-plugins/release: add distro_id field to nfpm rpm/deb package ta…

02ea550

…rgets This should also be included in the output_path and possibly other fields.

pants: refactor rpm/deb package parametrization

403b3ce

Packaging: Include deb maint scripts & rpm scriptlets

b7a4540

update changelog entry

2ef489b

cognifloyd added enhancement st2-packages pantsbuild labels Mar 27, 2025

cognifloyd added this to the pants milestone Mar 27, 2025

cognifloyd self-assigned this Mar 27, 2025

pull-request-size bot added the size/M PR that changes 30-99 lines. Good size to review. label Mar 27, 2025

cognifloyd commented Mar 27, 2025

View reviewed changes

cognifloyd marked this pull request as ready for review March 27, 2025 22:27

cognifloyd requested review from a team, amanda11, guzzijones, nzlosh, rush-skills and winem March 27, 2025 22:28

nzlosh approved these changes Mar 31, 2025

View reviewed changes

cognifloyd requested a review from a team March 31, 2025 17:44

guzzijones approved these changes Apr 2, 2025

View reviewed changes

guzzijones merged commit dc4dd91 into master Apr 2, 2025
85 checks passed

guzzijones deleted the packaging-nfpm_packages branch April 2, 2025 17:45

This was referenced Apr 2, 2025

Packaging: Add systemd files to rpm/deb packages #6326

Merged

Packaging: Add directories and symlinks to deb/rpm packages #6327

Merged

Packaging: Add venv and packs to deb/rpm packages #6328

Merged

cognifloyd mentioned this pull request Apr 11, 2025

Packaging: Add conf files to deb/rpm packages #6330

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Packaging: Add basic BUILD metadata for rpm/deb packages #6325

Packaging: Add basic BUILD metadata for rpm/deb packages #6325

Uh oh!

cognifloyd commented Mar 27, 2025 •

edited

Loading

Uh oh!

cognifloyd Mar 27, 2025

Uh oh!

cognifloyd Mar 27, 2025

Uh oh!

cognifloyd Mar 27, 2025

Uh oh!

cognifloyd Mar 27, 2025 •

edited

Loading

Uh oh!

cognifloyd Mar 27, 2025

Uh oh!

cognifloyd Mar 27, 2025

Uh oh!

cognifloyd Mar 27, 2025

Uh oh!

cognifloyd Mar 27, 2025

Uh oh!

cognifloyd Mar 27, 2025

Uh oh!

cognifloyd Mar 27, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	dependencies=[
	"./deb/scripts",
	],
	scripts=dict(
	preinstall="deb/scripts/pre-install.sh",
	postinstall="deb/scripts/post-install.sh",
	preremove="deb/scripts/pre-remove.sh",
	postremove="deb/scripts/post-remove.sh",
	# config="",
	# templates="",
	# rules="",
	),

	dependencies=[
	"./rpm/scripts",
	],
	scripts=dict(
	preinstall="rpm/scripts/pre-install.sh",
	postinstall="rpm/scripts/post-install.sh",
	preremove="rpm/scripts/pre-remove.sh",
	postremove="rpm/scripts/post-remove.sh",
	# pretrans="",
	# posttrans="",
	# verify="",
	),

		# https://jfearn.fedorapeople.org/en-US/RPM/4/html-single/RPM_Guide/index.html#idp3030720
		license="Apache-2.0", # TODO: nFPM is putting this under Copyright tag instead of License

		# arch used to be "any", but that was not correct as the venv has compiled packages.
		arch="amd64", # NOTE: parametrize this if adding support for arm64 or other arch.

Uh oh!

Packaging: Add basic BUILD metadata for rpm/deb packages #6325

Packaging: Add basic BUILD metadata for rpm/deb packages #6325

Uh oh!

Conversation

cognifloyd commented Mar 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

File Triggers

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cognifloyd Mar 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

cognifloyd commented Mar 27, 2025 •

edited

Loading

cognifloyd Mar 27, 2025 •

edited

Loading