Skip --checkpoint file statements with --resume#1853
Open
rolandwalker wants to merge 1 commit intomainfrom
Open
Skip --checkpoint file statements with --resume#1853rolandwalker wants to merge 1 commit intomainfrom
--checkpoint file statements with --resume#1853rolandwalker wants to merge 1 commit intomainfrom
Conversation
In --batch mode, when the batch input script is not STDIN, and when --checkpoint is also given, --resume causes mycli to replay the checkpoint file, looking for leading matching statements, and skip execution of batch statements already present in the checkpoint file. Motivation: resumption of interrupted batch scripts. The number of statements in the checkpoint file must be fewer than the number of statements in the batch script, and form a leading match, or mycli will exit without executing anything. Once execution is picked up again from the midpoint of the --batch script, we continue to append _new_ statements to the checkpoint file, after each statement is successfully executed. That behavior is unchanged. This allows the checkpoint file to be used again if the batch script is interrupted multiple times. The --progress bar and included ETA calculation account for the statements replayed from the checkpoint file, and show corrected views. Further work could include creating a [batch] section in myclirc and adding a default value for resumption, with a --no-resume option. Note: some SQL statements change server/session state or start transactions. But _any_ successful statement will be checkpointed and then not executed upon resumption in --resume mode. It is incumbent on the user to account for such state when resuming from a checkpoint.
rolandwalker
force-pushed
the
RW/batch-replay-checkpoint-file
branch
from
90f4540 to
483bfbf
Compare
scottnemes
reviewed
Apr 18, 2026
| click.secho('--progress is only compatible with a plain file.', err=True, fg='red') | ||
| return 1 | ||
| try: | ||
| completed_statement_count = replay_checkpoint_file(cli_args.batch, cli_args.checkpoint, cli_args.resume) |
Contributor
There was a problem hiding this comment.
Looks like currently the batch file is getting opened three times; one in this new replay_checkpoint_file function, again on line 122, and again on line 172. Would it be possible to combine those somehow? Maybe returning something from replay_checkpoint_file to re-use.
scottnemes
reviewed
Apr 18, 2026
| if cli_args.password is None and os.environ.get("MYSQL_PWD") is not None: | ||
| cli_args.password = os.environ.get("MYSQL_PWD") | ||
|
|
||
| if cli_args.resume and not cli_args.checkpoint: |
Contributor
There was a problem hiding this comment.
Should you also check if --batch mode is given with --resume? I.e. could use --execute and --resume currently.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
In
--batchmode, when the batch input script is not STDIN, and when--checkpointis also given,--resumecauses mycli to replay the checkpoint file, looking for leading matching statements, and skip execution of batch statements already present in the checkpoint file.Motivation: resumption of interrupted batch scripts.
The number of statements in the checkpoint file must be fewer than the number of statements in the batch script, and form a leading match, or mycli will exit without executing anything.
Once execution is picked up again from the midpoint of the
--batchscript, we continue to append new statements to the checkpoint file, after each statement is successfully executed. That behavior is unchanged. This allows the checkpoint file to be used again if the batch script is interrupted multiple times.The
--progressbar and included ETA calculation account for the statements replayed from the checkpoint file, and show corrected views.Further work could include creating a
[batch]section in myclirc and adding a default value for resumption, with a--no-resumeoption at the CLI.Note: some SQL statements change server/session state or start transactions. But any successful statement will be checkpointed and then not executed upon resumption in
--resumemode. It is incumbent on the user to account for such state when resuming from a checkpoint.This PR represents a major goal in a series of PRs to strengthen batch execution, some of which are:
--throttleoption for batch mode pauses #1460--checkpointfile for batch input #1466--batchoption as an alternative to STDIN #1723--progressprogress-bar option #1731--progressand--checkpointstrictly by-statement #1753Checklist
changelog.mdfile.AUTHORSfile (or it's already there).