Known issue: cuda.pathfinder cannot be imported in PEP 517 in-tree build backends without a workaround

[rwgk]

Problem

Packages that use cuda as a namespace package (e.g. cuda-bindings, cuda-core) and define an in-tree build backend via backend-path=["."] in pyproject.toml cannot import cuda.pathfinder at build time, even when cuda-pathfinder is listed in build-system.requires and correctly installed in the build environment.

Cause

When a PEP 517 build frontend (pip install, python -m build) loads an in-tree build backend, it prepends the project directory to sys.path. At that point, import cuda resolves the cuda namespace package to only the project's cuda/ subdirectory. Python caches a _NamespacePath for cuda in sys.modules. The cuda/pathfinder/ tree installed under site-packages is never merged into that namespace, so import cuda.pathfinder raises ModuleNotFoundError.

pkgutil.extend_path does not help: _NamespacePath._recalculate() re-derives its path list from the import machinery's finders, discarding manual additions.

Workaround

Replace the _NamespacePath with a plain list that includes the site-packages/cuda/ directory:

import os
import sys

def _import_get_cuda_path_or_home():
    """Import get_cuda_path_or_home, working around PEP 517 namespace shadowing.

    See https://github.com/NVIDIA/cuda-python/issues/1824 for why this helper is needed.
    """
    try:
        import cuda.pathfinder
    except ModuleNotFoundError as exc:
        if exc.name not in ("cuda", "cuda.pathfinder"):
            raise
        try:
            import cuda
        except ModuleNotFoundError:
            cuda = None

        for p in sys.path:
            sp_cuda = os.path.join(p, "cuda")
            if os.path.isdir(os.path.join(sp_cuda, "pathfinder")):
                cuda.__path__ = list(cuda.__path__) + [sp_cuda]
                break
        else:
            raise ModuleNotFoundError(
                "cuda-pathfinder is not installed in the build environment. "
                "Ensure 'cuda-pathfinder>=1.5' is in build-system.requires."
            )
        import cuda.pathfinder

    return cuda.pathfinder.get_cuda_path_or_home

This pattern is used in both cuda_bindings/build_hooks.py and cuda_core/build_hooks.py. Any other package that uses cuda as a namespace and needs cuda.pathfinder at build time will need a similar workaround.

References

• EXPERIMENT — NOT FOR MERGING — cuda-pathfinder in build-system.requires #1803 — original investigation with diagnostic CI output on linux-64, linux-aarch64, and win-64
• Use cuda-pathfinder in build-system.requires #1817 — PR that introduced the workaround into cuda-bindings and cuda-core

[rwgk]

Standing issue, to document a known issue.

[cpcloud]

I'm not following at all why an issue was opened and then immediately closed. Wouldn't #1817 close this issue?

[rwgk]

I wanted to leave the "Standing issue, to document a known issue." message.