feat: Catalog Integration: Search and Discovery

[dionesiusap]

Feature Request: Mendix Catalog Integration

Summary
Add mxcli catalog search command to discover and list services from Mendix Catalog programmatically.

---

Problem

Mendix Catalog (catalog.mendix.com) is the centralized registry for discovering data sources and services across an organization's landscape. It indexes OData services, REST APIs, SOAP services, and Business Events published by Mendix applications and external systems.

Currently, users must:

1. Manually browse the Catalog web portal to discover available services before consuming them
2. Copy service URLs and metadata paths into Studio Pro or MDL scripts by hand
3. Lack scriptable access for CI/CD pipelines that need to validate service availability or generate reports

This creates friction in development workflows and prevents automation of service discovery in CI/CD pipelines.

---

Proposed Feature (Phase 1)

mxcli catalog search <query> [flags]

Search for Catalog services with filters.

Examples:

# Authenticate first (one-time)
mxcli auth login

# Basic search
mxcli catalog search "customer"

# Filter by service type
mxcli catalog search "customer" --service-type OData

# Production endpoints only
mxcli catalog search "inventory" --production-only

# JSON output for scripting
mxcli catalog search "order" --json | jq '.[] | {name, uuid, type}'

# Pagination
mxcli catalog search "api" --limit 10 --offset 20

Table Output (7 columns, ~120 chars):

NAME                   TYPE    VERSION    APPLICATION           ENVIRONMENT   PROD  UUID
CustomerService        OData   1.2.0      CRM Application       Production    Yes   a7f3c2d1
OrderAPI               REST    2.0.1      E-commerce Platform   Acceptance    No    b8e4d3e2
InventorySync          SOAP    1.0.0      Warehouse System      Test          No    c9f5e4f3

Flags:

• --profile <name> - Auth profile (default: "default")
• --service-type <OData|REST|SOAP> - Filter by protocol
• --production-only - Show only production endpoints
• --owned-only - Show only owned services
• --limit <n> - Results per page (default: 20, max: 100)
• --offset <n> - Pagination offset (default: 0)
• --json - Output as JSON array

mxcli catalog show <uuid> [flags]

Display detailed endpoint metadata including entities, actions, and full contract.

Examples:

# Show endpoint details
mxcli catalog show a7f3c2d1-4b5e-6c7f-8d9e-0a1b2c3d4e5f

# JSON output with full metadata
mxcli catalog show a7f3c2d1 --json

Human-readable output:

Name:         CustomerService
Type:         OData
Version:      1.2.0
Application:  CRM Application
Environment:  Production (EU)
Description:  Manages customer data and relationships

Entities (3):
  - Customer (6 attributes, 2 associations)
  - Order (5 attributes, 1 association)
  - Address (4 attributes)

Actions (2):
  - CalculateDiscount
  - ValidateCustomer

Flags:

• --profile <name> - Auth profile (default: "default")
• --json - Output full JSON response including embedded contract

---

Technical Implementation

API: https://catalog.mendix.com/rest/search/v5/data
OpenAPI Spec: https://docs.mendix.com/openapi-spec/catalog-search_v5.yaml
Auth: Reuses existing internal/auth infrastructure (PAT tokens)
Host whitelisting: catalog.mendix.com already in internal/auth/scheme.go

Files to create:

• internal/catalog/types.go - API request/response structs
• internal/catalog/client.go - HTTP client for catalog.mendix.com/rest/search/v5
• cmd/mxcli/cmd_catalog.go - Cobra command definitions

---

Architecture Discussion: Future Integration

Before implementing, we need to decide where Catalog integration should live to support future features like:

# Future goal: Create OData client from Catalog entry
mxcli catalog create-odata-client <uuid> --into MyModule -p app.mpr

Key Technical Detail: Two-Step API Workflow

The Catalog API requires two endpoint calls to create an OData client:

Step 1: Discovery - GET /data returns search results with UUIDs:

{
  "data": [
    {
      "uuid": "a7f3c2d1-4b5e-6c7f-8d9e-0a1b2c3d4e5f",
      "name": "CustomerService",
      "serviceType": "OData",
      // ... basic metadata
    }
  ]
}

Step 2: Fetch Contract - GET /endpoints/{uuid} returns the full metadata:

{
  "uuid": "a7f3c2d1-4b5e-6c7f-8d9e-0a1b2c3d4e5f",
  "serviceVersion": {
    "contracts": [
      {
        "type": "CSDL",
        "documents": [
          {
            "isPrimary": true,
            "uri": "metadata.xml",
            "contents": "<?xml version=\"1.0\" encoding=\"utf-8\"?><edmx:Edmx>...</edmx:Edmx>"
          }
        ]
      }
    ]
  }
}

Key insight: The metadata is embedded in the JSON response (serviceVersion.contracts[0].documents[0].contents), not at a separate URL. This means:

• Users cannot simply copy a URL and paste into MDL (metadata requires auth to fetch)
• Any client creation feature must handle the API call to extract the embedded contract

Current Architecture

Current state:

• CLI commands (cmd/mxcli/cmd_*.go): Network-aware, auth-aware, can call external APIs
• MDL execution (mdl/executor/*.go): Pure MPR manipulation, no network, no auth
• Existing MDL syntax: CREATE EXTERNAL ENTITIES FROM 'http://...' INTO MyModule (requires public URL)

Desired future workflow:

# Step 1: Discovery via CLI
mxcli catalog search "customer"
# Output: CustomerService (uuid: a7f3c2d1...)

# Step 2: Create OData client from catalog UUID
# Option A: Via CLI command?
mxcli catalog create-odata-client a7f3c2d1 --into MyModule -p app.mpr

# Option B: Via MDL syntax?
CREATE EXTERNAL ENTITIES FROM CATALOG 'a7f3c2d1' INTO MyModule;

---

Core Architectural Question

Phase 1 (this issue) implements CLI-only search: mxcli catalog search "customer"

Phase 2 (future) needs to support: CREATE EXTERNAL ENTITIES FROM CATALOG '<uuid>' INTO MyModule

The question is: Should MDL/REPL/Executor support Catalog integration?

Current State

• MDL/Executor: Pure MPR manipulation, no network, no auth
• CLI commands: Network-aware, auth-aware, can call platform APIs
• Existing pattern: CREATE EXTERNAL ENTITIES FROM 'http://...' requires a public URL

The Challenge

To create an OData client from Catalog, we need to:

1. Authenticate with PAT token
2. Call GET /endpoints/{uuid} (requires auth)
3. Extract embedded metadata from JSON response
4. Parse and create entities in MPR

Option A: MDL/Executor Integration

• Add FROM CATALOG '<uuid>' to MDL grammar
• Executor calls Catalog API internally
• Works in REPL, scripts, and CLI
• Trade-off: Executor becomes network-dependent (auth, latency, errors, offline mode breaks)

Option B: CLI-Only

• mxcli catalog create-odata-client <uuid> --into MyModule -p app.mpr
• CLI fetches metadata, generates MDL, executes
• Trade-off: Cannot use in REPL/scripts, two ways to create clients

Key Questions

1. Should the executor have network access?

   • Currently: Pure MPR manipulation (offline, deterministic, testable)
   • Catalog integration would require: HTTP calls, auth, error handling for network failures
   • Impact: Testing complexity, offline usage, REPL session reliability

2. Should REPL sessions call external APIs?

   • Today: REPL is local-only (reads/writes MPR file)
   • Catalog integration: REPL would need credentials, handle auth failures mid-session
   • User experience: Is network dependency in REPL acceptable?

3. Should MDL syntax support Catalog UUIDs?

   • Yes: FROM CATALOG '<uuid>' alongside FROM 'http://...' (consistent, powerful)
   • No: Keep MDL pure, use CLI commands for Catalog workflows (simpler, explicit)

4. How should credentials flow into executor?

   • Executor takes (reader, ast) parameters today
   • Need: Profile name, PAT token, HTTP client
   • Options: Pass explicitly, use ambient env vars, or keep executor pure

Recommendation

Phase 1 (this PR): Implement mxcli catalog search only. Defer architecture decision.

Phase 2 (future discussion): Decide based on:

• User feedback on Phase 1 workflows
• Prototype both approaches to compare UX/complexity
• Team consensus on whether executor should be network-aware

This issue focuses on Phase 1. Architecture discussion for Phase 2 can happen in a separate issue/proposal once we have real usage data.

---

Proposed Acceptance Criteria (Phase 1)

Search command:

• mxcli catalog search <query> command works with all filter flags
• Table output truncates long values gracefully (7 columns, ~120 chars)
• JSON output includes full endpoint details
• Proper error handling with actionable hints (e.g., "Run: mxcli auth login")

Show command:

• mxcli catalog show <uuid> displays detailed endpoint metadata
• Human-readable format shows entities, actions, environment
• JSON output includes full contract (embedded metadata)
• Proper error handling for invalid UUIDs

Quality:

• Unit tests for client and command layers
• Manual testing against real Catalog API
• CHANGELOG.md updated
• CONTRIBUTING.md compliance (tests, lint, branch naming)

---

Out of Scope (Future PRs)

• Creating OData clients from discovered services (pending architecture decision in Phase 2)
• Interactive search UI (fzf-style picker)

---

References

• OpenAPI spec: https://docs.mendix.com/openapi-spec/catalog-search_v5.yaml
• Catalog docs: https://docs.mendix.com/catalog/
• Platform auth implementation: internal/auth/, cmd/mxcli/cmd_auth.go

[github-actions]

Issue Triage

Summary:
The issue requests a new mxcli catalog search command to programmatically discover and list services from the Mendix Catalog, including examples, flags, and implementation details.

Checklist for feature request completeness

Required information	Present?	Notes
Use case – why the feature is needed, what problem it solves	✅	Described in the “Problem” section (manual browsing, lack of scriptable access, CI/CD friction).
Proposed syntax or behavior (if applicable)	✅	Detailed command syntax, examples, flag list, and expected table/JSON output provided.
Mendix version relevance (if version‑specific)	⚠️	Not explicitly mentioned; the feature appears to be version‑agnostic (works with any Mendix version that has Catalog access). If it is indeed version‑specific, please add the relevant Mendix version(s).

Feedback
The issue is well‑structured and includes a clear use case and concrete proposal. Since the feature does not seem to depend on a particular Mendix version, the missing version relevance is not a blocker. If you later discover that the command only works with certain Mendix releases (e.g., requires a specific platform version for Catalog API compatibility), feel free to add that detail.

Thank you for the thorough feature request! 🙌

---

Automated triage via OpenRouter — workflow source

[dionesiusap]

Naming Collision: "Catalog" Has Two Meanings

During implementation, we identified a potential confusion issue:

The Problem

The word "catalog" is overloaded in mxcli:

1. Mendix Catalog (new in this PR)

   • External service registry at catalog.mendix.com
   • CLI: mxcli catalog search "customer"
   • Purpose: Discover OData/REST/SOAP services across your organization
   • Requires: Platform authentication (PAT token)

2. MDL CATALOG keyword (existing feature)

   • Local project metadata tables in SQLite
   • MDL: SELECT ... FROM CATALOG.entities
   • Purpose: Query your local project structure
   • Requires: REFRESH CATALOG command (no auth)

Why this is problematic:

1. User confusion - "catalog search" could mean either concept
2. Documentation ambiguity - Searching for "catalog" in docs returns mixed results
3. AI agent disambiguation - Even with improved documentation and skills, AI agents may struggle to determine which "catalog" the user is referring to in conversational contexts (e.g., "show me the catalog tables" vs "search the catalog for services")

Current Solution

We've improved documentation with disambiguation warnings in:

• .claude/skills/mendix/catalog-search.md - explicit note at top + comparison table
• .claude/skills/mendix/browse-integrations.md - warning pointing to external catalog docs
• cmd/mxcli/cmd_catalog.go - CLI help text clarifies "(catalog.mendix.com)"
• docs/11-proposals/PROPOSAL_catalog_integration.md - terminology note

User guidance: When working with AI agents, users should explicitly say "Mendix Catalog" (or "catalog.mendix.com") when referring to the external service registry, and "MDL CATALOG" (or "CATALOG tables") when referring to local project metadata.

Discussion

We didn't rename the CLI command because:

• mxcli catalog clearly maps to the Mendix product name "Mendix Catalog"
• Renaming would require inventing a new term (e.g., mxcli registry, mxcli services)
• The MDL CATALOG keyword is well-established and cannot change

However, I wanted to flag this for team discussion. Is this acceptable, or should we consider:

• Alternative CLI command names?
• More explicit naming in the CLI (e.g., mxcli mendix-catalog or mxcli catalog-registry)?
• Additional tooling to detect ambiguous usage?

Thoughts welcome!