[test-operator] Expose new SkipRegexList parameter

[kstrenkova]

This change exposes a newly added parameter for Tobiko custom resource, the SkipRegexList. It is a list of name patterns of tests that should be skipped.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign imatza-rh for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• roles/test_operator/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[softwarefactory-project-zuul]

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/ddb4297c6ef0439891d7574515f9548a

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 39m 54s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 28m 15s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 36m 50s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 14m 02s
✔️ cifmw-multinode-tempest SUCCESS in 1h 51m 21s
✔️ cifmw-pod-zuul-files SUCCESS in 4m 57s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 50s
❌ cifmw-pod-pre-commit FAILURE in 8m 40s
✔️ cifmw-molecule-test_operator SUCCESS in 3m 10s

[michburk]

Changes look good, but I notice that people are getting tripped-up by the pre_commit checks when making changes to the test_operator role. Inserting a new variable in the readme, even if you're inserting at the right spot alphabetically, shifts all of the line numbers for all of the github links.

Maybe we should remove the line numbers for the sync script? It's a nice-to-have, but generates a lot of noisy changes and keeps getting in the way.

[kstrenkova]

Hi @michburk 👋 I think we should update the sync script. I am not sure where I can see the output of why it failed, could you guide me to the log? I can create a ticket for it and fix it before merging this PR. It shouldn't be merged before I merge a change in test-operator, so I will put it on hold anyways.

[michburk]

At the bottom of https://logserver.rdoproject.org/614/rdoproject.org/614bc0c57f4847bbb1fa762388a5194e/pre_commit.log it shows:

Sync test_operator README................................................Failed
- hook id: sync-test-operator-readme
- files were modified by this hook

This means the version of the readme submitted with the pr doesn't match the readme after the sync script has been run on it. I think the only way to check the exact issue is by running it locally. On this pr, you can see:

$ gh pr checkout 3841
$ scripts/sync_test_operator_var_readme.py
$ git diff
# there are a bunch of diffs, but they're all similar to this one for example:
-* `cifmw_test_operator_ansibletest_workflow`: (List) A parameter that contains a workflow definition. Default value: See defaults in [main.yml](https://github.com/openstack-k8s-operators/ci-framework/blob/main/roles/test_operator/defaults/main.yml#L244)
+* `cifmw_test_operator_ansibletest_workflow`: (List) A parameter that contains a workflow definition. Default value: See defaults in [main.yml](https://github.com/openstack-k8s-operators/ci-framework/blob/main/roles/test_operator/defaults/main.yml#L246)

the only missing changes are the new line numbers for a bunch of the URLs.

I think it would make sense to remove the logic to link to the exact line numbers.
Also if there's a way we could improve the error message in pre_commit.log when the test_operator sync script fails, that would be good too.

[kstrenkova]

Ack, thank you 😄 I saw the error in the pre_commit.log and was wondering if you have a log with the differences as well. But if you ran it locally, then I will check it myself like that as well. I will also create a ticket for the script fix and assign it to myself. I definitely do agree with your points 👍