Skip to content

feat: API Server - Pipeline runs - Mirror Pipeline Run Annotations when creating a new pipeline run#216

Open
yuechao-qin wants to merge 1 commit intomasterfrom
ycq/feat-mirror-pipeline-run-annotations
Open

feat: API Server - Pipeline runs - Mirror Pipeline Run Annotations when creating a new pipeline run#216
yuechao-qin wants to merge 1 commit intomasterfrom
ycq/feat-mirror-pipeline-run-annotations

Conversation

@yuechao-qin
Copy link
Copy Markdown
Collaborator

@yuechao-qin yuechao-qin commented Apr 19, 2026

TL;DR

Closes https://github.com/Shopify/oasis-backend/issues/221

User-provided annotations are now mirrored into the PipelineRunAnnotation table at pipeline run creation time, and set_annotation now applies truncation and None-to-empty-string coercion via a shared helper.

What changed?

  • Introduced _mirror_single_annotation, a helper that centralizes annotation write logic: guards against system-key prefix collisions, coerces None values to empty strings, truncates values to the VARCHAR max length, and upserts the row.
  • Introduced _mirror_pipeline_run_annotations, which iterates over a user-supplied annotations dict and delegates each entry to _mirror_single_annotation.
  • _mirror_system_annotations now accepts an optional annotations dict and calls _mirror_pipeline_run_annotations so user annotations are persisted alongside system annotations at creation time.
  • create passes annotations through to _mirror_system_annotations.
  • set_annotation now delegates to _mirror_single_annotation instead of directly constructing and merging a PipelineRunAnnotation, gaining truncation and coercion for free.
  • The previously documented known failure for set_annotation overflowing the VARCHAR column is resolved — long values are now silently truncated rather than raising an IntegrityError.

How to test?

Run the updated test suite:

uv run pytest tests/test_api_server_sql.py

Why make this change?

Previously, user-provided annotations passed to create were not written to the PipelineRunAnnotation table, making them invisible to filter queries. Additionally, set_annotation lacked the same truncation and coercion guards applied to system annotations, causing potential IntegrityError failures on long values. Centralizing annotation write logic into _mirror_single_annotation ensures consistent behavior across all annotation write paths.

Copy link
Copy Markdown
Collaborator Author

@yuechao-qin yuechao-qin marked this pull request as ready for review April 19, 2026 10:37
@yuechao-qin yuechao-qin requested a review from Ark-kun as a code owner April 19, 2026 10:37
return value[:max_len]


def _mirror_single_annotation(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_mirror_single_**pipeline_run**_annotation

@Ark-kun Ark-kun changed the title feat: Mirror Pipeline Run Annotations feat: API Server - Pipeline runs - Mirror Pipeline Run Annotations when creating a new pipeline run Apr 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants