Draft
Conversation
Add an opt-in debug trace profiling endpoint that breaks transaction replay into timing phases and surfaces historical store lookup costs. Extend the store tracer to record iterator samples and per-operation timing so SS-heavy traces show which keys and scans dominated the replay. Made-with: Cursor
Thread request-scoped read tracing through the SS stack so debug trace profiles can attribute latency to MVCC and Pebble internals like iterator creation, Last, SeekLT, and NextPrefix. Surface the low-level stats alongside the existing store trace so it is clear whether historical lookups are dominated by SS wrapper logic or the underlying Pebble read path. Made-with: Cursor
Instrument the inner getMVCCSlice path so debug trace profiling attributes historical GET latency across iterator creation, Last, key reads, version decoding, value reads, cloning, and iterator close. This makes the dominant historical lookup path explicit instead of burying it inside a single coarse iterator bucket. Made-with: Cursor
Add a seidb command that runs debug_traceTransactionProfile across a block range, saves raw results, and generates a summarized report. Relax the release-branch trace test so it still validates the new endpoint while tolerating backend differences in the older v6.4 test setup. Made-with: Cursor
Polish the generated trace profile report into a dashboard with summary cards, top takeaways, and clearer hotspot tables for phases, low-level ops, store ops, slowest transactions, and slowest blocks. Keep the same underlying analytics while making results easier to scan during live tracing investigations. Made-with: Cursor
Speed up debug trace historical reads by caching repeated per-request lookups and reusing one Pebble iterator per store through a request-scoped snapshot instead of creating a new iterator for every point read. Keep the existing profiling surface and trace cleanup intact so the before/after impact remains measurable from debug_traceTransactionProfile. Made-with: Cursor
Add a versionless latest-value index for recent historical reads and fall back to MVCC history only when needed. Cache replayed block state across trace requests so tracing multiple transactions in the same block no longer replays from tx zero every time. Made-with: Cursor
Switch MVCC version encoding to descending order so historical reads can seek forward to the first visible version instead of creating bounded reverse iterators. Update forward and reverse iterator semantics plus prune behavior so the new ordering stays correct under iteration, pruning, and trace profiling. Made-with: Cursor
|
The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).
|
Sort Pebble batch writes by comparer order so the new latest-value index and inverted MVCC layout do not trip write-order errors. Tighten inverted-order iteration, old-version reads, and prune behavior so the DB path stays correct while preserving the new trace speedup path. Made-with: Cursor
Codecov Report❌ Patch coverage is
❌ Your patch check has failed because the patch coverage (33.33%) is below the target coverage (50.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## release/v6.4 #3252 +/- ##
================================================
- Coverage 58.37% 58.36% -0.01%
================================================
Files 2091 2091
Lines 172983 172763 -220
================================================
- Hits 100972 100832 -140
+ Misses 63009 62944 -65
+ Partials 9002 8987 -15
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Write the latest-version metadata key outside the comparer-sorted Pebble data batch so inverted MVCC ordering and latest-value indexing do not conflict with metadata key ordering. Keep the read-path optimizations intact while removing the runtime batch ordering failure. Made-with: Cursor
Strip the latest-value side index entirely to eliminate the write-ordering conflict between s/l: index keys and s/k: MVCC data keys under the custom comparer. Keep the inverted MVCC ordering, request cache, reusable iterators, and replay cache which are the main performance improvements. Made-with: Cursor
Re-add the latest-value side index for fast recent reads. The actual root cause of the write-ordering error was empty/malformed keys from the state sync import path, not the index itself. Guard the import path and batch writers against empty storeKey or empty key to prevent the collision. Made-with: Cursor
The s/l: latest-value index keys are fundamentally incompatible with the custom MVCC comparer during Pebble compaction. Remove the index entirely. Keep inverted MVCC ordering, request cache, reusable iterators, and replay cache. Made-with: Cursor
Store a per-key "latest value" pointer at a reserved sentinel MVCC version (math.MaxInt64) so the custom comparer sees a fully-formed MVCC key and never the plain-shape keys that triggered the state-sync ordering bug. Readers do a single bloom-filter accelerated db.Get on the sentinel key before falling back to the MVCC scan path. Prune, RawIterate, and iteration all explicitly skip the sentinel so it never surfaces as real data. No new prefix, no new DB, no migration. Made-with: Cursor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe your changes and provide context
Testing performed to validate your change