Skip to content

Metal backend: Materialize non-packed tensor views in reinterpret_tensor#19033

Merged
manuelcandales merged 1 commit intomainfrom
manuel/metal-materialize-non-packed-views
Apr 22, 2026
Merged

Metal backend: Materialize non-packed tensor views in reinterpret_tensor#19033
manuelcandales merged 1 commit intomainfrom
manuel/metal-materialize-non-packed-views

Conversation

@manuelcandales
Copy link
Copy Markdown
Contributor

AOTI generates reinterpret_tensor views with non-packed strides (e.g. chunk/split for RoPE rotation) that have holes in memory. ExecuTorch's make_tensor_ptr requires densely packed layouts.

When aoti_torch__reinterpret_tensor encounters non-packed strides, allocate a new contiguous Metal buffer and copy elements using strided access from the source.

Authored with Claude.

Copilot AI review requested due to automatic review settings April 21, 2026 19:48
@pytorch-bot
Copy link
Copy Markdown

pytorch-bot Bot commented Apr 21, 2026

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19033

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 Cancelled Job, 2 Unrelated Failures

As of commit cf71b8e with merge base 1d37abd (image):

CANCELLED JOB - The following job was cancelled. Please retry:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 21, 2026
@manuelcandales manuelcandales removed the request for review from shoumikhin April 21, 2026 19:48
@github-actions
Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@manuelcandales manuelcandales force-pushed the manuel/metal-materialize-non-packed-views branch from 6a23ab6 to 0f6aae2 Compare April 21, 2026 19:51
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the Metal AOTI runtime shim to handle reinterpret_tensor views that have non-packed (holey) strides by materializing them into a newly allocated contiguous Metal buffer, aligning with ExecuTorch tensor construction requirements.

Changes:

  • Added is_packed_strides() to detect when a strided view has “holes” (storage extent > numel).
  • Added materialize_packed() to allocate a new Metal buffer and copy elements from the strided source view into a packed layout.
  • Updated aoti_torch__reinterpret_tensor to materialize non-packed views, compute contiguous strides for the new buffer, and update ownership/refcount bookkeeping accordingly.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread backends/apple/metal/runtime/shims/memory.cpp Outdated
Comment thread backends/apple/metal/runtime/shims/memory.cpp
Comment thread backends/apple/metal/runtime/shims/memory.cpp
AOTI generates reinterpret_tensor views with non-packed strides (e.g.
chunk/split for RoPE rotation) that have holes in memory. ExecuTorch's
make_tensor_ptr requires densely packed layouts.

When aoti_torch__reinterpret_tensor encounters non-packed strides,
allocate a new contiguous Metal buffer and copy elements using strided
access from the source.

Authored with Claude.
@manuelcandales manuelcandales force-pushed the manuel/metal-materialize-non-packed-views branch from 0f6aae2 to cf71b8e Compare April 21, 2026 20:35
Copy link
Copy Markdown
Contributor

@metascroy metascroy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stamping, but maybe consider adopting SlimTensor to address this

@manuelcandales manuelcandales merged commit 6be4fb5 into main Apr 22, 2026
200 of 203 checks passed
@manuelcandales manuelcandales deleted the manuel/metal-materialize-non-packed-views branch April 22, 2026 03:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants