feat: Add support for new GCP and Azure features [COMP-1463, COMP-1464]#606
Merged
cristianrcv merged 14 commits intomasterfrom Apr 16, 2026
Merged
Conversation
This was referenced Apr 10, 2026
- WIF (Workload Identity Federation) support for Google credentials with --mode=workload-identity, --service-account-email, --workload-identity-provider, and --token-audience options - Network tags (--network-tags) with VPC requirement and GCP format validation, plus --network and --subnetwork options - Machine type selection: --head-job-machine-type (single) and --compute-jobs-machine-type (comma-separated list with wildcard support), mutually exclusive with instance templates - Boot disk image (--boot-disk-image) with format validation for projects/*/global/images/*, family paths, and batch-* short names - Fusion Snapshots (--fusion-snapshots) toggle requiring Fusion v2 All features include CLI validation matching backend/frontend rules. Code will compile once tower-java-sdk is bumped to include the new fields on GoogleSecurityKeys and GoogleBatchConfig. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Upgrade tower-java-sdk from 1.114.0 to 1.133.0 to include new fields on GoogleSecurityKeys and GoogleBatchConfig - Add bash integration test script for all COMP-1463 features Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add Azure Entra (service principal) credentials support (add/update) - Add managed identity options (head/pool client IDs and resource IDs) for both Forge and Manual Azure Batch compute environments - Add VNet/subnet support (--subnet-id) for private network isolation - Add dual pool mode (--dual-pool) with separate head and worker pool configuration for Forge mode, and --worker-pool for Manual mode - Add job max wall clock time option (--job-max-wall-clock-time) - Replace JobCleanupPolicy enum with 3 boolean toggles: --delete-jobs-on-completion, --delete-tasks-on-completion, --terminate-jobs-on-completion Note: requires tower-java-sdk bump to >= 1.133.0 for new model fields. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Bump tower-java-sdk from 1.114.0 to 1.133.0 for new Azure Batch model fields (managed identity, subnet, dual pool, wall clock time, cleanup toggles) - Add bash integration test script for all new Azure Batch CLI features Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…DK 1.133.0 bump Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix AzBatchConfig.deleteJobsOnCompletion mock response: changed from string "on_success" to boolean true after field type change in SDK - Switch request body matchers from exact string to json() partial matching for pipeline and label mocks to handle new serialized fields in WorkflowLaunchRequest, AssociatePipelineLabelsRequest, and CreateLabelRequest Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…eflection Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
fda15a9 to
07e1f8b
Compare
…aalVM reflection Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
stefanoboriero
approved these changes
Apr 13, 2026
Simplify test coverage to Forge flows, fix ws_flag ordering so it comes after the subcommand, auto-fill /subnets/default when SUBNET_ID is a VNet ID, and extract cleanup into a standalone clean-azure-batch-test-resources.sh that wipes all test-* compute envs and credentials. [COMP-1464] [test]
… GraalVM reflection Register AzBatchForgePlatform inner classes (HeadPoolOptions, WorkerPoolOptions, ManagedIdentityOptions), AzBatchManualPlatform.ManagedIdentityOptions, Azure Entra credential commands/providers, ManageCmd for pipeline versions, plus additional SDK 1.133.0 model classes (Role DTOs, GitHubAppSecurityKeys, DataStudio/Dataset update requests, AzBatchPoolConfig, ValidateUrl) exposed by the native Azure Batch integration tests. [COMP-1464] [graalvm]
…ed-identity-head-client-id
Align the head-job managed identity client ID flag with its resource-id counterpart so all head/pool option pairs share the same naming scheme (--managed-identity-{head,pool}-{client-id,resource-id}). Applies to both forge and manual AzBatch platforms; AzCloud is left untouched because it has no head/pool split.
Addresses Stefano's review comment on PR #606.
[COMP-1464]
…DK field Reverting 459650f. The CLI should mirror the API field names; the SDK model AzBatchConfig has managedIdentityClientId (head) alongside managedIdentityHeadResourceId, managedIdentityPoolClientId, managedIdentityPoolResourceId. Renaming the CLI flag to add a 'head' prefix broke that 1:1 mapping. Added a note on the Option description acknowledging the asymmetry is inherited from the SDK. Audited all other COMP-1464 flags against the tower-java-sdk 1.133.0 AzBatchConfig, AzBatchForgeConfig, AzBatchPoolConfig and AzureEntraKeys models — remaining flags already match their API counterparts (only inverted-boolean idioms like --no-auto-scale / --preserve-resources differ, and those predate COMP-1464). [COMP-1464]
…ed-identity-head-client-id Restoring the rename originally applied in 459650f (reverted in d059503). Per reviewer discussion on PR #606, the CLI-side head/pool symmetry (--managed-identity-{head,pool}-{client-id,resource-id}) is clearer for users than strictly mirroring the SDK's asymmetric field names, so prefer the consistent CLI naming even though the underlying API keeps managedIdentityClientId for the head job. [COMP-1464]
…ror messages IllegalArgumentException was not handled by the CLI error handler, causing full stack traces to be printed for validation errors. Switching to TowerRuntimeException produces clean one-line ERROR messages instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Contributor
GCP QA Test Results (2026-04-15)Target: WIF Credentials
Google Batch Compute Environment
Bug FixDuring QA, validation errors were printing full Java stack traces instead of clean error messages. Fixed in 6b4db1f by replacing |
jonmarti
approved these changes
Apr 15, 2026
ramonamela
approved these changes
Apr 15, 2026
georgi-seqera
added a commit
that referenced
this pull request
Apr 17, 2026
* feat: add new Google Batch CE and credential features [COMP-1463]
- WIF (Workload Identity Federation) support for Google credentials
with --mode=workload-identity, --service-account-email,
--workload-identity-provider, and --token-audience options
- Network tags (--network-tags) with VPC requirement and GCP format
validation, plus --network and --subnetwork options
- Machine type selection: --head-job-machine-type (single) and
--compute-jobs-machine-type (comma-separated list with wildcard
support), mutually exclusive with instance templates
- Boot disk image (--boot-disk-image) with format validation for
projects/*/global/images/*, family paths, and batch-* short names
- Fusion Snapshots (--fusion-snapshots) toggle requiring Fusion v2
All features include CLI validation matching backend/frontend rules.
Code will compile once tower-java-sdk is bumped to include the new
fields on GoogleSecurityKeys and GoogleBatchConfig.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: bump tower-java-sdk to 1.133.0 and add integration test script
- Upgrade tower-java-sdk from 1.114.0 to 1.133.0 to include new fields
on GoogleSecurityKeys and GoogleBatchConfig
- Add bash integration test script for all COMP-1463 features
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: update CLI to support new Azure Batch features [COMP-1464]
- Add Azure Entra (service principal) credentials support (add/update)
- Add managed identity options (head/pool client IDs and resource IDs)
for both Forge and Manual Azure Batch compute environments
- Add VNet/subnet support (--subnet-id) for private network isolation
- Add dual pool mode (--dual-pool) with separate head and worker pool
configuration for Forge mode, and --worker-pool for Manual mode
- Add job max wall clock time option (--job-max-wall-clock-time)
- Replace JobCleanupPolicy enum with 3 boolean toggles:
--delete-jobs-on-completion, --delete-tasks-on-completion,
--terminate-jobs-on-completion
Note: requires tower-java-sdk bump to >= 1.133.0 for new model fields.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: bump tower-java-sdk to 1.133.0 and add integration test script
- Bump tower-java-sdk from 1.114.0 to 1.133.0 for new Azure Batch
model fields (managed identity, subnet, dual pool, wall clock time,
cleanup toggles)
- Add bash integration test script for all new Azure Batch CLI features
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: pass resolveSymlink param to generateDownloadUrlDataLink after SDK 1.133.0 bump
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: update tests for SDK 1.133.0 serialization changes
- Fix AzBatchConfig.deleteJobsOnCompletion mock response: changed from
string "on_success" to boolean true after field type change in SDK
- Switch request body matchers from exact string to json() partial
matching for pipeline and label mocks to handle new serialized fields
in WorkflowLaunchRequest, AssociatePipelineLabelsRequest, and
CreateLabelRequest
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: register new SDK 1.133.0 model methods and classes for GraalVM reflection
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: register DataStudioStartRequest name JsonNullable methods for GraalVM reflection
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test(azure-batch): refine Azure Batch integration test script
Simplify test coverage to Forge flows, fix ws_flag ordering so it comes after the subcommand, auto-fill /subnets/default when SUBNET_ID is a VNet ID, and extract cleanup into a standalone clean-azure-batch-test-resources.sh that wipes all test-* compute envs and credentials.
[COMP-1464] [test]
* fix: register Azure Batch forge options and additional SDK models for GraalVM reflection
Register AzBatchForgePlatform inner classes (HeadPoolOptions, WorkerPoolOptions, ManagedIdentityOptions), AzBatchManualPlatform.ManagedIdentityOptions, Azure Entra credential commands/providers, ManageCmd for pipeline versions, plus additional SDK 1.133.0 model classes (Role DTOs, GitHubAppSecurityKeys, DataStudio/Dataset update requests, AzBatchPoolConfig, ValidateUrl) exposed by the native Azure Batch integration tests.
[COMP-1464] [graalvm]
* refactor(azure-batch): rename --managed-identity-client-id to --managed-identity-head-client-id
Align the head-job managed identity client ID flag with its resource-id counterpart so all head/pool option pairs share the same naming scheme (--managed-identity-{head,pool}-{client-id,resource-id}). Applies to both forge and manual AzBatch platforms; AzCloud is left untouched because it has no head/pool split.
Addresses Stefano's review comment on PR #606.
[COMP-1464]
* revert(azure-batch): restore --managed-identity-client-id to mirror SDK field
Reverting 459650f. The CLI should mirror the API field names; the SDK model AzBatchConfig has managedIdentityClientId (head) alongside managedIdentityHeadResourceId, managedIdentityPoolClientId, managedIdentityPoolResourceId. Renaming the CLI flag to add a 'head' prefix broke that 1:1 mapping. Added a note on the Option description acknowledging the asymmetry is inherited from the SDK.
Audited all other COMP-1464 flags against the tower-java-sdk 1.133.0 AzBatchConfig, AzBatchForgeConfig, AzBatchPoolConfig and AzureEntraKeys models — remaining flags already match their API counterparts (only inverted-boolean idioms like --no-auto-scale / --preserve-resources differ, and those predate COMP-1464).
[COMP-1464]
* refactor(azure-batch): rename --managed-identity-client-id to --managed-identity-head-client-id
Restoring the rename originally applied in 459650f (reverted in d059503). Per reviewer discussion on PR #606, the CLI-side head/pool symmetry (--managed-identity-{head,pool}-{client-id,resource-id}) is clearer for users than strictly mirroring the SDK's asymmetric field names, so prefer the consistent CLI naming even though the underlying API keeps managedIdentityClientId for the head job.
[COMP-1464]
* feat: add a command to update studio details
---------
Co-authored-by: ramonamela <25862624+ramonamela@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Cristian Ramon-Cortes <cristian.ramon-cortes@seqera.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary GCP
Closes COMP-1463 — Make new platform features available via CLI.
--mode=workload-identitywith--service-account-email,--workload-identity-provider,--token-audienceoptions for Google credentials (PR #10346)--network-tags(comma-separated) with--network/--subnetworkoptions and GCP format validation (PR #10416)--head-job-machine-type(single) and--compute-jobs-machine-type(comma-separated, wildcard support), mutually exclusive with instance templates (PRs #10488, #10532)--boot-disk-imagewith format validation for project paths, family paths, and batch short names (PR #10484)--fusion-snapshotstoggle requiring--fusion-v2(PR #10751)All features include CLI-side validation matching backend/frontend rules.
Summary Azure
tw credentials add azure-entra)--subnet-id) for private network isolation--dual-pool) with separate head and worker pool configuration for Forge, and--worker-poolfor Manual mode--job-max-wall-clock-time)JobCleanupPolicyenum with 3 boolean toggles:--delete-jobs-on-completion,--delete-tasks-on-completion,--terminate-jobs-on-completionNote: Requires tower-java-sdk bump to >= 1.133.0 for new model fields. Code will not compile until the SDK is updated.
Platform PRs implemented