Skip to content

fix(ai-evals): split compound pipeline delegation assertion into two#199

Open
orionnye wants to merge 1 commit intomainfrom
cursor/split-pipeline-eval-assertion
Open

fix(ai-evals): split compound pipeline delegation assertion into two#199
orionnye wants to merge 1 commit intomainfrom
cursor/split-pipeline-eval-assertion

Conversation

@orionnye
Copy link
Copy Markdown

Summary

  • Split the compound assertion "should build a self-contained Task prompt with the pipeline file path and return expectation" into two separate assertions: one for file path inclusion, one for return expectation
  • The original assertion was failing at 2/4 runs — splitting it makes it easier to diagnose which part is actually failing

Test plan

  • Run pipeline-skill-test.sudo eval and verify both new assertions are evaluated independently

Made with Cursor

The "self-contained Task prompt with pipeline file path and return
expectation" assertion was failing at 2/4 runs. Splitting it into
separate assertions for file path inclusion and return expectation
makes failures more diagnosable.

Made-with: Cursor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant