Skip to content

Conversation

@cognifloyd
Copy link
Member

@cognifloyd cognifloyd commented Sep 24, 2024

pymongo 4 removes deprecated names for database connection options that were renamed some time ago. We even have a TODO about updating our usage of the deprecated names in the code:

# NOTE: In pymongo 3.9.0 some of the ssl related arguments have been renamed -
# https://api.mongodb.com/python/current/changelog.html#changes-in-version-3-9-0
# Old names still work, but we should eventually update to new argument names.

One change in particular could not be supported without breaking backwards compatibility: The files for ssl_keyfile and ssl_certfile must be concatenated and passed as one file in the new tls_certificate_key_file option.

Why update to pymongo 4 (in a follow-up PR)? We need to update pymongo to ensure we're using a version that tests with and officially supports our target MongoDB version(s) (we are planning on using MongoDB 7; see #6246 and #6236). The pymongo4 upgrade guide has details on the option naming migration.

Each commit touches one option or aspect of this migration, so it will be useful to review each commit.

Since we're using newer oslo.config now, we can also be more explicit about deprecations when defining the options. So, a few of the commits make use of those newer features to improve our st2.conf.sample file.

In summary, these options were migrated:

  • 3622868 ssl -> tls
  • dc62d11 ssl_keyfile + ssl_certfile -> tls_certificate_key_file (files must be concatenated)
    • Also add tls_certificate_key_file_password
  • 2094ff4 ssl_cert_reqs -> tls_allow_invalid_certificates (from a string opt to a bool opt)
  • e610bdc ssl_ca_certs -> tls_ca_file
  • c962d3c ssl_match_hostnames -> tls_allow_invalid_hostnames (inverted meaning)

Note: In #6246, I initially developed this using mongo's camelCase naming convention in st2.conf. After discussing with @nzlosh I went back to using snake_case to be consistent with the rest of the st2.conf options.

…l_certfile

pymongo 4 will ignore the ssl_keyfile/ssl_certfile options.

For consistency in st2.conf, this uses snake_case not the mongo camelCase option name.

This also adds tls_certificate_key_file_password.
We did not support ssl_pem_passphrase before, so there was nothing to migrate.
This needed to be a different option (instead of just renaming) because
the option type is changing from str+choices to a bool.

For consistency in st2.conf, this uses snake_case not the mongo
camelCase option name.
For consistency in st2.conf, this uses snake_case not the mongo
camelCase option name.
…tname

For consistency in st2.conf, this uses snake_case not the mongo
camelCase option name.
Not sure if this wasn't available before, or why it wasn't used. Try and see.
And use fix the sample default for python_binary to use python3.
@cognifloyd cognifloyd added this to the 3.9.0 milestone Sep 24, 2024
@cognifloyd cognifloyd self-assigned this Sep 24, 2024
@pull-request-size pull-request-size bot added the size/XL PR that changes 500-999 lines. Consider splitting work into several ones that easier to review. label Sep 24, 2024
@cognifloyd cognifloyd marked this pull request as ready for review September 24, 2024 03:37
@cognifloyd cognifloyd requested review from a team and amanda11 September 24, 2024 03:37

result = await Get(
FallibleProcessResult,
ProcessResult,
Copy link
Member Author

@cognifloyd cognifloyd Sep 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change prevents the conf/st2.conf.sample from being replaced if tools/config_gen.py exits with an error. It also makes pants report the error by printing stdout from the process, which includes the traceback.

Copy link
Contributor

@nzlosh nzlosh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job!

@cognifloyd cognifloyd requested a review from a team September 24, 2024 06:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking change enhancement external dependency migrations mongodb size/XL PR that changes 500-999 lines. Consider splitting work into several ones that easier to review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants