add standard for end-to-end testing with Tempest

[depressiveRobot]

Caused by #1010.

As discussed in detail in the SIG Std/Cert (see minutes from 2025-07-04 onwards), OpenStack Powered Compute has been deprecated and the repositories have also been deleted. Therefore Tempest should be used as an alternative, which is the official OpenStack integration test suite. This PR updates the documentation accordingly with a step-by-step guide for operators on how to install and run Tempest tests until we will create a separate SCS standard for it. Update: This PR adds a draft standard and a supplement with a step-by-step guide for operators on how to install and run Tempest tests.

Further related issues:

• Modify OpenStack Powered Compute to make it work #990

[depressiveRobot]

@toothstone @berendt @chrisschwa @flyersa @fzakfeld As discussed in the SIG Std/Cert on 2026-01-15, it would be great if you could work together to create instructions on how to install and configure Tempest so that it can be used by all operators for compliance according to SCS-compatible IaaS.

I already created all relevant files for the documentation, please add the instructions to the TODO:

standards/Docs/guides/testing-guide/tempest.md

Line 26 in 040fcc2

_TODO_ provide step-by-step walkthrough here

[toothstone]

I feel like working on this via a PR is rather clunky, maybe we could have the main guide document in a Hedgedoc so everyone can easily contribute?

[berendt]

Tempest installation: https://github.com/osism/container-images/blob/main/tempest/Containerfile
Tempest configuration: https://github.com/osism/ansible-collection-validations/blob/main/roles/tempest/templates/tempest.conf.j2
Tempest tests: https://github.com/osism/testbed/blob/main/environments/openstack/files/tempest/include-scs-compatible-iaas.lst
Tempest command: https://github.com/osism/testbed/blob/main/scripts/check/302-openstack-scs-compatible-iaas.sh#L18-L29

[mbuechse]

I don't understand what's happening here. The goal of #1010 was clear: create a standard that states what we require (and yes, it's only a subset of OPC because OPC was never really feasible, only that no one noticed that) and maybe a supplement that explains how to actually perform the test. What I see now is that a new Docs folder is being created with documents that don't fall under our regular ADR scheme, and this folder is then used by the docs repo (which we control, and so far, we have put docs there), and the YAML file now doesn't reference an authoritative document but some docs page? This is quite irregular.

[toothstone]

Tempest installation: https://github.com/osism/container-images/blob/main/tempest/Containerfile Tempest configuration: https://github.com/osism/ansible-collection-validations/blob/main/roles/tempest/templates/tempest.conf.j2 Tempest tests: https://github.com/osism/testbed/blob/main/environments/openstack/files/tempest/include-scs-compatible-iaas.lst Tempest command: https://github.com/osism/testbed/blob/main/scripts/check/302-openstack-scs-compatible-iaas.sh#L18-L29

My findings, mostly regarding the use of the OSISM tempest role: https://input.scs.community/findings-scs-compliant-yaook?view#Passing-the-OpenStack-Powered-Compute-test-suite-202211

Yaook has merged my feature for loading a list of tests and an example manifest: https://docs.yaook.cloud/user/explanations/examples/tempest.html#specifying-a-list-of-tests-to-load

[mbuechse]

There was a big misunderstanding. Sorry about that. It should be resolved now. This PR is not ready to be merged yet, but I'm formally approving because I want to retract my former "request changes".

[toothstone]

uStack tempest config: https://gitlab.com/uhurutec/stack/ustack/-/blob/672149065564a22fb3f80475e3fe47ebc2850533/example/tempest-tests.yaml

In the last SIG Std&Cert call someone mentioned very briefly that it is possible to use image or flavor names (instead of the _ref values I assume). The official docs don't mention anything like that and I haven't seen it in the wild. If anyone has any pointers please add them here.

[toothstone]

Please adjust the PR title to reflect the change in scope, and maybe the initial description as well (it still states that this PR will not create a new standard).

[chrisschwa]

Just to my interest. Is it really mandatory or "standard" to make all those tempest tests with admin credentials and creation of dynamic users? I do really dislike giving something admin access which is in charge of creating & deleting ressources. We had some really bad experience with the OHM for example..

[mbuechse]

Just to my interest. Is it really mandatory or "standard" to make all those tempest tests with admin credentials and creation of dynamic users? I do really dislike giving something admin access which is in charge of creating & deleting ressources. We had some really bad experience with the OHM for example..

No, it isn't.

[mbuechse]

@chrisschwa It's explicitly stated here:
https://github.com/SovereignCloudStack/standards/blob/98b3279fce178ff0dd2630ba468f427befddf815/Standards/scs-0128-w1-testing-notes.md#test-cases

[toothstone]

Passing this test case requires explicitly disabling swift in the Tempest config if you don't have it, otherwise it will try to check its service type and fail.

[fkr]

I suggest to move this from a 'Draft PR" into a PR that could be merged.
One fact that just became once again apparent during @mbuechse 's talk at the Yaook Hackathon when he presented the certificate scopes: the link to OpenStack powered compute from our certificate scope ends in a 404 (since upstream removed the stuff). This has been obvious for quite some time, I just wanted to highlight this again and why it would be good to move along with this.