feat: reference contextual signal encoder with model descriptor

[esteininger]

Summary

The scoring service consumes embeddings but the spec has no reference for producing them. This adds a contextual signal encoder — content in, ORTB-compatible embedding out — using the model metadata fields already defined in embedding_format.schema.json (version, embedding_space_id, metric).

No new schema — this is a reference implementation of what the spec already describes.

Public lane / private lane

Ships with sentence-transformers as the "public lane" — a standardized open-source model any party can use. The EmbeddingProvider interface supports private-lane providers. The spec metadata fields travel with every embedding regardless of which lane produced it.

Output format

Uses fields from specs/v1.0/embedding_format.schema.json:

{
  "ext": {
    "ver": "1.0",
    "vector": [0.042, -0.118, "..."],
    "model": "all-MiniLM-L6-v2",
    "dimension": 384,
    "type": "context",
    "version": "2.0.0",
    "embedding_space_id": "aa://spaces/contextual/sentence-transformers/minilm-l6-v2",
    "metric": "cosine"
  }
}

Output plugs directly into POST /score as a user.data[] entry.

Designed as a foundation for benchmarking and re-embedding

The pluggable provider interface and spec-defined metadata fields support two follow-on capabilities:

1. Benchmarking — run multiple providers against the same test corpus, compare results. The interface already supports swapping encoders; the spec metadata makes results reproducible.

2. Re-embedding / upgrade path — when a new model is adopted, embedding_space_id + version identify which vectors need migration. A migration tool re-encodes through a new provider, outputs to a new space, and both run in parallel. The scoring service partitions by model name, so old and new coexist.

The provider interface is minimal (2 methods) so any embedding backend can be dropped in without changing the encoder, wire format, or scoring integration.

Changes

• POST /encode endpoint (text or URL input)
• Uses spec-defined metadata fields (no custom schema)
• EmbeddingProvider interface for pluggable public/private backends
• sentence-transformers default provider
• 9 tests, Dockerfile

Test plan

• pytest tests/ -v — 9 passing
• Review field alignment with embedding_format.schema.json