chore: unpin urllib3

[bigpick]

Info

Unpins urllib3, since looks like requests is no longer broken, and that was the reason it was originally pinned <2.

Testing

Quick install from git+test detect-secrets command works:

pip install --upgrade "git+https://github.com/bigpick/detect-secrets.git@remove-pinned-urllib3#egg=detect-secrets"
<installs fine other than warning about using legacy setup.py install>

detect-secrets --version
0.13.1+ibm.63.dss

detect-secrets scan --update .secrets.baseline --use-all-plugins .
<runs fine>

detect-secrets audit .secrets.baseline
Nothing to audit!

Proper

py39 via tox (via make test):

  py39: OK (108.58=setup[28.22]+cmd[0.28,35.06,0.87,0.83,43.32] seconds)
  congratulations :) (109.70 seconds)

full output:

Toggle dropdown

make test
tox
py38: skipped because could not find python interpreter with spec(s): py38
py38: SKIP ⚠ in 0.98 seconds
py39: recreate env because python changed executable='/Users/gp/.pyenv/versions/3.9.19/bin/python3.9'->'/usr/local/bin/python3.9'
py39: remove tox env folder /tmp/workdir/.tox/py39
py39: install_deps> python -I -m pip install -r requirements-dev.txt
.pkg: recreate env because python changed executable='/Users/gp/.pyenv/versions/3.9.19/bin/python3.9'->'/usr/local/bin/python3.9'
.pkg: remove tox env folder /tmp/workdir/.tox/.pkg
.pkg: install_requires> python -I -m pip install 'setuptools>=40.8.0' wheel
.pkg: _optional_hooks> python /tmp/workdir/.direnv/python-3.9/lib/python3.9/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_sdist> python /tmp/workdir/.direnv/python-3.9/lib/python3.9/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_wheel> python /tmp/workdir/.direnv/python-3.9/lib/python3.9/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: install_requires_for_build_wheel> python -I -m pip install wheel
.pkg: prepare_metadata_for_build_wheel> python /tmp/workdir/.direnv/python-3.9/lib/python3.9/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: build_sdist> python /tmp/workdir/.direnv/python-3.9/lib/python3.9/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
py39: install_package_deps> python -I -m pip install binaryornot 'boxsdk[jwt]' packaging pyyaml requests tabulate urllib3
py39: install_package> python -I -m pip install --force-reinstall --no-deps /tmp/workdir/.tox/.tmp/package/2/detect_secrets-0.13.1+ibm.63.dss.tar.gz
py39: commands[0]> coverage erase
py39: commands[1]> coverage run -m pytest tests
================================================================================= test session starts ==================================================================================
platform linux -- Python 3.9.19, pytest-8.2.0, pluggy-1.5.0
cachedir: .tox/py39/.pytest_cache
rootdir: /tmp/workdir
collected 1303 items

tests/core/audit_test.py ...........................................                                                                                                             [  3%]
tests/core/baseline_test.py ..............................                                                                                                                       [  5%]
tests/core/bidirectional_iterator_test.py ......                                                                                                                                 [  6%]
tests/core/potential_secret_test.py .......                                                                                                                                      [  6%]
tests/core/report/conditions_test.py ........                                                                                                                                    [  7%]
tests/core/report/output_test.py ...................................                                                                                                             [  9%]
tests/core/report/report_test.py ....................                                                                                                                            [ 11%]
tests/core/secrets_collection_test.py ......................                                                                                                                     [ 13%]
tests/core/usage_test.py ...........                                                                                                                                             [ 13%]
tests/main_test.py ..............................                                                                                                                                [ 16%]
tests/plugins/artifactory_test.py ........................................                                                                                                       [ 19%]
tests/plugins/aws_key_test.py ...............                                                                                                                                    [ 20%]
tests/plugins/azure_storage_key_test.py .                                                                                                                                        [ 20%]
tests/plugins/base_test.py ............                                                                                                                                          [ 21%]
tests/plugins/basic_auth_test.py ......                                                                                                                                          [ 21%]
tests/plugins/box_test.py ....................                                                                                                                                   [ 23%]
tests/plugins/cloudant_test.py .........................                                                                                                                         [ 25%]
tests/plugins/common/filters_test.py ........................                                                                                                                    [ 27%]
tests/plugins/common/initialize_test.py ......                                                                                                                                   [ 27%]
tests/plugins/common/yaml_file_parser_test.py ...                                                                                                                                [ 27%]
tests/plugins/db2_test.py .....................................                                                                                                                  [ 30%]
tests/plugins/gh_enterprise_test.py ..............................................................                                                                               [ 35%]
tests/plugins/github_token_test.py ...                                                                                                                                           [ 35%]
tests/plugins/high_entropy_strings_test.py .....................................................................................                                                 [ 42%]
tests/plugins/ibm_cloud_iam_test.py ...........................................                                                                                                  [ 45%]
tests/plugins/ibm_cos_hmac_test.py .......................................                                                                                                       [ 48%]
tests/plugins/jwt_test.py ..............                                                                                                                                         [ 49%]
tests/plugins/keyword_test.py .................................................................................................................................................. [ 60%]
................................................................................................................................................................................ [ 74%]
................................................................................................................................................................................ [ 87%]
....................                                                                                                                                                             [ 89%]
tests/plugins/mailchimp_key_test.py ......                                                                                                                                       [ 89%]
tests/plugins/npm_test.py ......                                                                                                                                                 [ 90%]
tests/plugins/private_key_test.py ..                                                                                                                                             [ 90%]
tests/plugins/slack_test.py ..........                                                                                                                                           [ 91%]
tests/plugins/softlayer_test.py ...................................................................                                                                              [ 96%]
tests/plugins/square_oauth_test.py .                                                                                                                                             [ 96%]
tests/plugins/stripe_key_test.py ....                                                                                                                                            [ 96%]
tests/plugins/twilio_test.py ..                                                                                                                                                  [ 96%]
tests/pre_commit_hook_test.py .....................                                                                                                                              [ 98%]
tests/util_test.py ...................                                                                                                                                           [100%]

================================================================================ 1303 passed in 34.32s =================================================================================
py39: commands[2]> coverage report --show-missing '--include=tests/*' --fail-under 100
Name    Stmts   Miss Branch BrPart  Cover   Missing
---------------------------------------------------
TOTAL    2548      0    160      0   100%

38 files skipped due to complete coverage.
py39: commands[3]> coverage report --show-missing '--include=detect_secrets/*' --fail-under 97
Name                                               Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------------------------------------------
detect_secrets/core/audit.py                         260     12    108      8    95%   86, 103-104, 209-210, 293->302, 391, 402, 480-481, 600->587, 674, 693-694, 741->740
detect_secrets/core/baseline.py                      122      1     80      1    99%   90
detect_secrets/core/report/report.py                  45      1     26      1    97%   123
detect_secrets/core/secrets_collection.py            161      5     84      9    94%   123->128, 139->129, 271->278, 279-284, 285->287, 378, 421->430, 431-436, 437->exit
detect_secrets/core/usage.py                         211     12     30      0    93%   443-454, 463-471
detect_secrets/main.py                               103      5     58      8    92%   30->33, 37, 76->105, 167->170, 174, 212-216, 219->exit, 239
detect_secrets/plugins/aws.py                         37      1     12      0    98%   32
detect_secrets/plugins/base.py                       138      0     50      1    99%   162->166
detect_secrets/plugins/cloudant.py                    43      1     10      2    94%   72, 124->126
detect_secrets/plugins/common/ini_file_parser.py      62      1     30      2    97%   113->119, 123
detect_secrets/plugins/common/util.py                 34      0     19      1    98%   29->39
detect_secrets/plugins/db2.py                         53      3      8      0    95%   91, 125-141
detect_secrets/plugins/high_entropy_strings.py       186      3     52      0    99%   345, 421, 425
detect_secrets/plugins/jwt.py                         32      1     10      0    98%   21
detect_secrets/plugins/softlayer.py                   33      1     10      2    93%   39, 71->73
detect_secrets/pre_commit_hook.py                    115      0     36      1    99%   33->36
----------------------------------------------------------------------------------------------
TOTAL                                               2457     47    840     36    97%

38 files skipped due to complete coverage.
py39: commands[4]> pre-commit run --all-files --show-diff-on-failure
[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Initializing environment for https://github.com/pycqa/flake8.
[INFO] Initializing environment for https://github.com/asottile/reorder_python_imports.
[INFO] Initializing environment for https://github.com/asottile/add-trailing-comma.
[INFO] Initializing environment for https://github.com/pre-commit/mirrors-autopep8.
[INFO] Initializing environment for https://github.com/ibm/detect-secrets.
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/pycqa/flake8.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/asottile/reorder_python_imports.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/asottile/add-trailing-comma.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/pre-commit/mirrors-autopep8.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for https://github.com/ibm/detect-secrets.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
Check builtin type constructor use.......................................Passed
Check docstring is first.................................................Passed
Debug Statements (Python)................................................Passed
Fix double quoted strings................................................Passed
Fix End of Files.........................................................Passed
Tests should end in _test.py.............................................Passed
Trim Trailing Whitespace.................................................Passed
flake8...................................................................Passed
Reorder python imports...................................................Passed
Add trailing commas......................................................Passed
autopep8.................................................................Passed
Detect secrets...........................................................Passed
  py38: SKIP (0.98 seconds)
  py39: OK (108.58=setup[28.22]+cmd[0.28,35.06,0.87,0.83,43.32] seconds)
  congratulations :) (109.70 seconds)

[NielsKorschinsky]

Hi @bigpick is this PR ready to review?
Thanks for making the effort of creating this PR.
We are getting issues due to the low version and would also like to update without incompatibilities.
(we're also users of this module, not devs).

I believe @williamsbritt is the regular Maintainer of this repository, right? :)

[bigpick]

Hi - yeah, its been ready since the last commit AFAICT

Though as I understand it, the team responsible for maintaining this detect-secrets repo is pretty slammed, so this PR+some other outstanding work/fixes/improvements are going to continue waiting till they have time to review (or help share the duty of maintenance to a broader team - myself and some others have expressed interest)

[bradhvr]

Hello, is there any plan or timeline on getting this PR merged?

[bigpick]

Hello, is there any plan or timeline on getting this PR merged?

Its been made clear by the internal IBM team responsible for managing this repository that they are not interested nor able to maintain this repo, and they have/are actively ignoring any offerings of outside help to support such efforts (for I imagine numerous reasons) from folks like @dnwe and myself

... so, I doubt it.

[Jawahars]

There’s a security advisory (CVE-2024-37891) affecting urllib3. It would be a good idea to upgrade to a safe version to ensure this project remains secure.

[kyle170]

Adding to the party that also (GHSA-pq67-6m6q-mj2v && GHSA-48p4-8xcf-vxj5) due to the pinned version of urllib3