Skip to content

refactor!: partition ModelOutputThunk execution metadata into Generat…#908

Open
ajbozarth wants to merge 2 commits intogenerative-computing:mainfrom
ajbozarth:refactor/mot-field-partitioning
Open

refactor!: partition ModelOutputThunk execution metadata into Generat…#908
ajbozarth wants to merge 2 commits intogenerative-computing:mainfrom
ajbozarth:refactor/mot-field-partitioning

Conversation

@ajbozarth
Copy link
Copy Markdown
Contributor

@ajbozarth ajbozarth commented Apr 22, 2026

Misc PR

Type of PR

  • Bug Fix
  • New Feature
  • Documentation
  • Other

Description

Groups the five flat execution-metadata fields (usage, model, provider,
ttfb_ms, streaming) from ModelOutputThunk into a single GenerationMetadata
dataclass accessed as mot.generation. This was motivated by __init__,
_copy_from, __copy__, and __deepcopy__ growing a new line with every
telemetry field addition — the sub-object keeps those methods lean as the
metadata surface grows and gives callers a typed namespace.

⚠️ Breaking change: mot.usage, mot.model, mot.provider, mot.ttfb_ms,
and mot.streaming are removed — use mot.generation.* instead.

All five fields were added ~5 weeks ago as part of the same telemetry epic, so
this is a hard break with no deprecated aliases. All callers in mellea/,
cli/, test/, and docs/ have been updated in this PR.

Changed files:

  • mellea/core/base.py — new GenerationMetadata dataclass; ModelOutputThunk replaces 5 flat fields with self.generation
  • mellea/core/__init__.py — exports GenerationMetadata
  • mellea/backends/{openai,litellm,ollama,watsonx,huggingface}.pymot.*mot.generation.* in post_processing()
  • mellea/telemetry/metrics_plugins.py — updated all 4 plugins to read via mot.generation
  • mellea/helpers/openai_compatible_helpers.pybuild_completion_usage updated to read output.generation.usage
  • test/backends/test_{ollama,openai_ollama,litellm_ollama,watsonx,huggingface}.py — assertion updates
  • test/cli/test_{serve,serve_streaming,serve_errors}.py — fixture/setup updates
  • test/core/test_base.py — new copy semantics tests for GenerationMetadata
  • test/telemetry/test_metrics_plugins.py — updated helpers to use GenerationMetadata constructor
  • AGENTS.md, docs/docs/evaluation-and-observability/metrics.md, docs/examples/telemetry/metrics_example.py — updated field references

Testing

  • Tests added to the respective file if code was changed
  • New code has 100% coverage if code as added
  • Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

  • AI coding assistants used

@github-actions github-actions Bot added the enhancement New feature or request label Apr 22, 2026
@github-actions
Copy link
Copy Markdown
Contributor

The PR description has been updated. Please fill out the template for your PR to be reviewed.

@ajbozarth ajbozarth self-assigned this Apr 22, 2026
@ajbozarth ajbozarth marked this pull request as ready for review April 22, 2026 21:13
@ajbozarth ajbozarth requested review from a team, jakelorocco and nrfulton as code owners April 22, 2026 21:13
…ionMetadata

Group the five flat telemetry fields (usage, model, provider, ttfb_ms,
streaming) into a new GenerationMetadata dataclass accessed as mot.generation.
This keeps __init__, _copy_from, __copy__, and __deepcopy__ lean as the
metadata surface grows, and gives type-safe access to all five fields under
a single namespace.

BREAKING CHANGE: mot.usage, mot.model, mot.provider, mot.ttfb_ms, and
mot.streaming are removed. Access them as mot.generation.usage,
mot.generation.model, mot.generation.provider, mot.generation.ttfb_ms,
and mot.generation.streaming respectively.

Assisted-by: Claude Code
Signed-off-by: Alex Bozarth <ajbozart@us.ibm.com>
@ajbozarth ajbozarth force-pushed the refactor/mot-field-partitioning branch from b15e069 to 45b687d Compare April 22, 2026 22:00
@ajbozarth
Copy link
Copy Markdown
Contributor Author

@planetf1 Cross-reference for #891 implementation: this PR removed the five flat execution-metadata fields (usage, model, provider, ttfb_ms, streaming) from ModelOutputThunk with no deprecated aliases. They now live on mot.generation.*.

The streaming path introduced by #891 will interact with at least mot.generation.streaming and mot.generation.ttfb_ms, so the sub-issue should use the new paths. No merge conflicts expected — the file sets are disjoint — but worth noting before #898#901 work begins.

Minor: sub-issue #901 cites mellea/core/base.py:434-436 for the single-consumer constraint comment. Line numbers may have shifted due to changes in this PR — worth verifying when implementing.

@planetf1
Copy link
Copy Markdown
Contributor

planetf1 commented Apr 23, 2026

@ajbozarth thanks. One other note I found -- __copy__ now shares the GenerationMetadata instance by reference rather than copying each scalar independently, so post-copy mutations to one affect the other. No production code shallow-copies a MOT today, but #901 (stream_with_chunking) could introduce that path. If needed, copied.generation = copy(self.generation) is a one-line fix — happy to pick that up in the #901 PR rather than actioning it here speculatively. Will also make things easier if this is merged as I can design/build on top of it!

Comment thread mellea/helpers/openai_compatible_helpers.py
- Copy GenerationMetadata in __copy__ rather than sharing the instance
  by reference, so post-copy mutations don't bleed across both objects
- Tighten build_completion_usage signature from Any to ModelOutputThunk

Assisted-by: Claude Code
Signed-off-by: Alex Bozarth <ajbozart@us.ibm.com>
@ajbozarth
Copy link
Copy Markdown
Contributor Author

One other note I found -- copy now shares the GenerationMetadata instance by reference rather than copying each scalar independently, so post-copy mutations to one affect the other.

Fixed in 756d1a3

@planetf1 your review should all be addressed now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ModelOutputThunk field refactor: partition fields into sub-structures

2 participants