feat: log concurrency configuration at startup for debugging#925
feat: log concurrency configuration at startup for debugging#925devin-ai-integration[bot] wants to merge 3 commits intomainfrom
Conversation
Co-Authored-By: unknown <>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. 💡 Show Tips and TricksTesting This CDK VersionYou can test this version of the CDK using the following: # Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1772184954-log-concurrency-level#egg=airbyte-python-cdk[dev]' --help
# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1772184954-log-concurrency-levelPR Slash CommandsAirbyte Maintainers can execute the following slash commands on your PR:
|
PyTest Results (Fast)597 tests - 3 272 585 ✅ - 3 272 3m 27s ⏱️ - 3m 24s For more details on these failures, see this check. Results for commit 4ea2fe0. ± Comparison against base commit 7f41401. This pull request removes 3272 tests.♻️ This comment has been updated with latest results. |
PyTest Results (Full)3 872 tests 3 859 ✅ 11m 18s ⏱️ For more details on these failures, see this check. Results for commit 4ea2fe0. ♻️ This comment has been updated with latest results. |
|
/prerelease
|
…values Co-Authored-By: unknown <>
Co-Authored-By: unknown <>
|
/prerelease
|
Summary
Adds an info-level log line in
ConcurrentDeclarativeSource.__init__that logs the resolved concurrency configuration at startup. This logs:concurrency_level(the number of worker threads)initial_number_of_partitions_to_generateconcurrency_levelwas defined in the manifest or fell back to the defaultThis is needed to debug a Google Ads connector deadlock in Airbyte Cloud where the connector hangs deterministically when the
num_workersconfig field is absent, but works when explicitly set to3— despite both cases producing identical concurrency levels locally. This log line will confirm the actual concurrency level the connector receives at runtime in the Cloud environment.Review & Testing Checklist for Human
__init__, so should be negligible)Notes