Add skupper for BKL and keystone to SKMO job by vakwetu · Pull Request #3836 · openstack-k8s-operators/ci-framework

vakwetu · 2026-04-07T18:54:46Z

[skmo] Add Skupper for cross-region RabbitMQ and Keystone internal routing

Add hook playbooks and configuration to establish Skupper virtual
services for RabbitMQ and Keystone internal endpoints, enabling
cross-region connectivity in the multi-namespace SKMO scenario.

skupper-connector.yaml: query the RabbitMQ TLS secret from the correct
CRD - rabbitmq.openstack.org/v1beta1 (RabbitMq) as used by the
OpenStack infra-operator, not the community rabbitmq.com/v1beta1
(RabbitmqCluster). Add retries to wait for spec.tls.secretName to
be populated before creating the Skupper Connector.

skupper-keystone-connector.yaml: add retries to the KeystoneAPI CR
lookup to wait for spec.tls.api.internal.secretName to be available,
since that field is not populated until Keystone completes TLS setup.

configure-leaf-keystone-internal.yaml: after patching the leaf OSCP
to use the Skupper Keystone virtual service, also create a MetalLB
LoadBalancer Service (keystone-regionone-lb) and a DNSData CR
(keystone-skupper) so that EDPM compute nodes outside the OCP cluster
can resolve and connect to the Keystone auth_url. The Skupper Listener
creates a ClusterIP-only Service that is unreachable from EDPM nodes;
the LoadBalancer Service obtains a MetalLB IP on the leaf internalapi
network and the DNSData entry registers both the short (.svc) and
fully-qualified (.svc.cluster.local) names in the dnsmasq instance
serving those nodes.

openshift-ci · 2026-04-07T18:54:52Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign fultonj for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

softwarefactory-project-zuul · 2026-04-10T19:39:55Z

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/451747df4e864222b9ec4411ae75617d

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 42m 53s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 24m 33s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 28m 47s
❌ cifmw-crc-podified-edpm-baremetal-minor-update RETRY_LIMIT in 32m 49s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 10s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 56s

evallesp

In general LGTM

evallesp · 2026-04-16T08:11:16Z

+      until:
+        - _rabbitmq_cluster.resources | length > 0
+        - _rabbitmq_cluster.resources[0].spec.tls is defined
+        - _rabbitmq_cluster.resources[0].spec.tls.secretName is defined


(blocking) suggestion: I think we should add: _rabbitmq_cluster.resources[0].spec.tls.secretName | length > 0. So now we're checking it's actually populated.

Done in latest commit

evallesp · 2026-04-16T08:22:37Z

+    cifmw_skupper_upstream_install_url: "https://skupper.io/v2/install.yaml"
+    cifmw_skupper_downstream_install_file: ""
+  tasks:
+    - name: Check if Skupper CRD is already present


praise: I like this: can we add this in others playbook? like in listener and connector?

Done in latest commit.

evallesp · 2026-04-16T08:23:36Z

+                    transport_url = rabbit://{{ _username }}:{{ _password }}@{{ cifmw_skupper_listener_host }}:{{ cifmw_skupper_rabbitmq_port }}/?ssl=1
                    [keystone_notifications]
-                    pool_name = barbican-listener-regionTwo
+                    pool_name = barbican-listener-regiontwo


(non-blocking) concern: Is this camelCase removal required?

the name doesn't matter as long as its unique. This parameter isn't referenced anywhere else. Non-camelcase is more consistent across the playbook.

evallesp · 2026-04-16T08:25:29Z

+        - _skupper_crd.resources | length == 0
+        - cifmw_skupper_install_source == 'upstream'
+      ansible.builtin.command:
+        cmd: "oc apply -f {{ cifmw_skupper_upstream_install_url }}"


(blocking) question: I'd like to see kubernetes.core.k8s. Probably in two tasks: one with ansible.builtin.uri to fetch the content and save in a var, and then one of kubernets.core.k8s with the variable.content and state as present.

Done in latest commit

evallesp · 2026-04-16T08:25:37Z

+        - _skupper_crd.resources | length == 0
+        - cifmw_skupper_install_source == 'downstream'
+        - cifmw_skupper_downstream_install_file | length > 0
+      ansible.builtin.command:


(blocking) question: I'd like to see kubernetes.core.k8s.

Done in latest commit

The existing hook_controlplane_config.yml assumed the OpenStackControlPlane (OSCP) was not yet present, which is correct for the component / CRC pipeline. The SKMO integration pipeline runs after kustomize_deploy has already brought up the OSCP, so it needs a different hook that reads the live OSCP, preserves existing CA bundle data, and patches the resource in place. Split into two task files: - hook_controlplane_config.yml (pre-deploy / component pipeline, unchanged) - hook_controlplane_config_postdeploy.yml (post-deploy / SKMO pipeline) Add a new playbook federation-controlplane-config-postdeploy.yml that wraps the new post-deploy task file. The architecture/automation/vars/multi-namespace-skmo.yaml is updated separately to call the new post-deploy playbook. Fix variable names in both task files to use the required cifmw_ prefix so they pass the var-naming[pattern] rule enforced by ansible-lint. Relates-To: OSPCIX-1321 Signed-off-by: Ade Lee <alee@redhat.com> Co-authored-by: Claude Sonnet 4.6 <no-reply@anthropic.com> Made-with: Cursor

softwarefactory-project-zuul · 2026-04-17T04:14:44Z

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://softwarefactory-project.io/zuul/t/rdoproject.org/buildset/faf40921ee674e6c9ef0143267a03148

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 17m 33s
❌ podified-multinode-edpm-deployment-crc FAILURE in 24m 42s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 34m 29s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 04m 18s
✔️ cifmw-pod-zuul-files SUCCESS in 8m 04s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 11m 47s
✔️ cifmw-pod-pre-commit SUCCESS in 10m 18s
✔️ cifmw-molecule-federation SUCCESS in 2m 43s

danpawlik · 2026-04-17T11:08:06Z

+#     MetalLB address-pool name for the leaf internalapi network.  An IP is
+#     auto-assigned from this pool; no static address is required.
+- name: Configure leaf region to use Skupper Keystone internal endpoint
+  hosts: localhost


we used yo use hosts: "{{ cifmw_target_hook_host | default('localhost') }}"

ack. fixed.

…uting Add hook playbooks and configuration to establish Skupper virtual services for RabbitMQ and Keystone internal endpoints, enabling cross-region connectivity in the multi-namespace SKMO scenario. skupper-connector.yaml: query the RabbitMQ TLS secret from the correct CRD - rabbitmq.openstack.org/v1beta1 (RabbitMq) as used by the OpenStack infra-operator, not the community rabbitmq.com/v1beta1 (RabbitmqCluster). Add retries to wait for spec.tls.secretName to be populated before creating the Skupper Connector. skupper-keystone-connector.yaml: add retries to the KeystoneAPI CR lookup to wait for spec.tls.api.internal.secretName to be available, since that field is not populated until Keystone completes TLS setup. configure-leaf-keystone-internal.yaml: after patching the leaf OSCP to use the Skupper Keystone virtual service, also create a MetalLB LoadBalancer Service (keystone-regionone-lb) and a DNSData CR (keystone-skupper) so that EDPM compute nodes outside the OCP cluster can resolve and connect to the Keystone auth_url. The Skupper Listener creates a ClusterIP-only Service that is unreachable from EDPM nodes; the LoadBalancer Service obtains a MetalLB IP on the leaf internalapi network and the DNSData entry registers both the short (.svc) and fully-qualified (.svc.cluster.local) names in the dnsmasq instance serving those nodes. Signed-off-by: Ade Lee <alee@redhat.com> Co-authored-by: Claude <noreply@anthropic.com> Made-with: Cursor

vakwetu · 2026-04-17T15:39:38Z

Note: the prefix check is failing because the earliest commit contains [federation] changes. Once #3847 is merged, then I will rebase and federation role changes will no longer be in this or subsequent PRs.

vakwetu added the do-not-merge/work-in-progress label Apr 7, 2026

vakwetu force-pushed the skmo-skupper-keystone branch from e91b57c to 74abdbd Compare April 7, 2026 18:57

github-actions bot added the Ready For Review label Apr 7, 2026

vakwetu force-pushed the skmo-skupper-keystone branch from 74abdbd to f87154f Compare April 8, 2026 19:48

github-actions bot removed the Ready For Review label Apr 8, 2026

vakwetu force-pushed the skmo-skupper-keystone branch from f87154f to 64dea66 Compare April 8, 2026 20:37

github-actions bot added the Ready For Review label Apr 8, 2026

vakwetu force-pushed the skmo-skupper-keystone branch from 64dea66 to d3f6a8e Compare April 9, 2026 18:56

github-actions bot removed the Ready For Review label Apr 9, 2026

openshift-ci bot removed the do-not-merge/work-in-progress label Apr 9, 2026

vakwetu mentioned this pull request Apr 9, 2026

[skmo] Add Skupper for cross-region RabbitMQ and Keystone openstack-k8s-operators/architecture#734

Open

vakwetu requested a review from dmendiza April 9, 2026 19:25

github-actions bot added the Ready For Review label Apr 9, 2026

vakwetu force-pushed the skmo-skupper-keystone branch from d3f6a8e to 6368ac0 Compare April 10, 2026 17:55

github-actions bot removed the Ready For Review label Apr 10, 2026

vakwetu added the do-not-merge/work-in-progress label Apr 10, 2026

vakwetu force-pushed the skmo-skupper-keystone branch 4 times, most recently from 3502950 to c88378c Compare April 15, 2026 00:24

vakwetu added Ready For Review and removed do-not-merge/work-in-progress labels Apr 15, 2026

vakwetu requested review from abays, evallesp and fultonj April 15, 2026 18:49

evallesp reviewed Apr 16, 2026

View reviewed changes

vakwetu force-pushed the skmo-skupper-keystone branch from c88378c to 89a9ef9 Compare April 16, 2026 18:07

github-actions bot removed the Ready For Review label Apr 16, 2026

vakwetu added the do-not-merge/work-in-progress label Apr 16, 2026

vakwetu force-pushed the skmo-skupper-keystone branch 2 times, most recently from 52c399b to 7b01c17 Compare April 16, 2026 19:04

vakwetu added Ready For Review and removed do-not-merge/work-in-progress labels Apr 17, 2026

vakwetu force-pushed the skmo-skupper-keystone branch from 7b01c17 to 82dfda1 Compare April 17, 2026 01:56

danpawlik reviewed Apr 17, 2026

View reviewed changes

vakwetu force-pushed the skmo-skupper-keystone branch from 82dfda1 to 2190966 Compare April 17, 2026 15:36

Conversation

vakwetu commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci bot commented Apr 7, 2026

Uh oh!

softwarefactory-project-zuul bot commented Apr 10, 2026

Uh oh!

evallesp left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

softwarefactory-project-zuul bot commented Apr 17, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vakwetu commented Apr 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

vakwetu commented Apr 7, 2026 •

edited

Loading