-
Notifications
You must be signed in to change notification settings - Fork 65
Handle and test xmllint versions that return a zero exit code even if validation fails #708
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…behavior of bad xmllint which returns a zero exit code even if the validation fails
…urn a zero exit code even if the validation fails; simply logic and update test/unit_tests/test_sdf.py accordingly
| res = validate_xml_file(file, 'suite', schema_version, logger) | ||
| if not res: | ||
| raise CCPPError(f"Invalid suite definition file, '{sdf}'") | ||
| raise CCPPError(f"Invalid suite definition file, '{file}'") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unrelated bug fix uncovered while working on this PR
|
@climbfuji I'm a bit confused about how this new test helps the issue. I thought your intention was to run a test on a known bad XML, and if that returns a zero exit code, use that to determine that the xmllint version is bad? With this test it seems like we're just creating a test that will never fail with a non-zero exit code, unless I'm misunderstanding something. |
No, the intention is to create a fake xmllint version that behaves like the "bad" version that @gold2718 somehow ended up with and then make sure that this version can also be used and produces the correct results. The fake xmllint version is the newly added xmllint wrapper. |
peverwhee
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks @climbfuji !
dustinswales
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These changes look good to me. Thanks!
|
@climbfuji I assume we want to wait for @gold2718 to review this one before merging? |
Yes, this is only five days old. Thanks for checking. |
gold2718
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good (i.e., way better than my version), thanks!
Handle and test xmllint versions that return a zero exit code even if validation fails
scripts/parse_tools/xml_tools.py, handlexmllintversions that return a zero exit code even if the validation fails. Simply logic, remove unused code, and updatetest/unit_tests/test_sdf.pyaccordinglytest/unit_tests/xmllint_wrapper/xmllintwrapper that emulates the behavior of "bad"xmllintwhich returns a zero exit code even if the validation fails, and test in GitHub actionsNotes.
call_commandwas only used in one place (validate XML files) and contained overly complicated logic to deal with optional arguments that were unnecessary. With the need to parse stdout/stderr from the subprocess in thevalidate_xmlfunction, there would have been even more code duplication and unnecessary logic.xmllintbecame a hard requirement, therefore we can remove logic that allows silent failures of the XML validation step whenxmllintcannot be found (silent failures in general are a bad idea imo).User interface changes?: No
Fixes #700
Testing:
unit tests:
pytestis now run in GitHub actions with both "good" and "bad" xmllint versionsmanual testing: ran Python
docteston the updatedxml_tools.pyfile, ranpytestlocally on my laptop with "good" and "bad" xmllint