Add switch for running Sonobuoy tests in parallel#1117
Conversation
Signed-off-by: Matthias Büchse <matthias.buechse@alasca.cloud>
| @click.option("-r", "--result_dir_name", "result_dir_name", type=str, default="sonobuoy_results", help="directory name to store results at",) | ||
| @click.option("-c", "--check", "check_name", type=str, default="sonobuoy_executor", help="this MUST be the same name as the id in 'scs-compatible-kaas.yaml'",) | ||
| @click.option("--scs-sonobuoy-config", "scs_sonobuoy_config_yaml", type=click.Path(exists=True), default="kaas/sonobuoy-config.yaml", help="Configuration for Sonobuoy (yaml format)") | ||
| @click.option('--execution-mode', 'mode', type=click.Choice(('serial', 'parallel')), default='serial') |
There was a problem hiding this comment.
| @click.option('--execution-mode', 'mode', type=click.Choice(('serial', 'parallel')), default='serial') | |
| @click.option('--execution-mode', 'mode', type=click.Choice(('serial', 'parallel')), default='parallel') |
There was a problem hiding this comment.
I would like to see the default being parallel. This is what I would expect to be working from services like K8s.
There was a problem hiding this comment.
I think we should vote on that in the SIG because it's a clear deviation from the status quo. If we decide to change the default, we need to do at the top of the file scs-compatible-kaas.yaml, because the default of the CLI tool is overriden by that.
| Replace `SUBJECT` with an arbitrary, but meaningful subject name. Also, please note that the check | ||
| will always use the `current-context` of the kubeconfig and will fail if it isn't set. | ||
|
|
||
| Note: Sonobuoy checks (such as CNCF k8s conformance) are known to run for multiple hours (during which |
There was a problem hiding this comment.
Unrelated to this line, but there is no PR description
There was a problem hiding this comment.
I do write a description if I find the heading isn't sufficient. What info would you say is missing specifically?
| if mode == 'parallel': | ||
| # This is merely a shortcut to simplify commandline in scs-compatible-kaas.yaml | ||
| # For more on parallel execution, see https://github.com/vmware-tanzu/sonobuoy/issues/1435 | ||
| args.append('--e2e-parallel=true') |
There was a problem hiding this comment.
I don't quite understand how to variable replacement/insertion functions. I would have thought that if I enabled parallel execution with -a execution_mode=parallel then I would see --e2e-parallel=true in the debug log, but I only see --execution-mode parallel
DEBUG: running './kaas/sonobuoy_handler/run_sonobuoy.py run -k /home/ubuntu/aurora/admin.conf --scs-sonobuoy-config kaas/scs-sonobuoy-config.yaml -r ./sono-results-0219 -c \'kaas-networking-check\' -a \'--e2e-focus "NetworkPolicy"\' --execution-mode parallel'...
There was a problem hiding this comment.
For run_sonobuoy.py we have a dedicated switch for the execution mode now so we don't need to use parameter passthrough to Sonobuoy any more. What you see in your quote is the command-line for run_sonobuoy.py, not the commandline for sonobuoy -- the latter would show --e2e-parallel=true (not sure if it's ever logged though).
| spec['var_defaults'] = defaults | ||
|
|
||
|
|
||
| def load_spec(document: dict) -> dict: |
There was a problem hiding this comment.
As far as I can tell, there are no tests yet for this tool. I would at least add some tests to make sure that the configured call, e.g.
./scs-compliance-check.py --debug -v -a kubeconfig=/home/ubuntu/aurora/admin.conf -a subject_root=. -s SUBJECT -o report.yaml -a execution_mode=parallel scs-compatible-kaas.yamlresults in the appropriate cmd call.
There was a problem hiding this comment.
I am not going to write tests for this tool at the current state.
2 participants
No description provided.