Skip to content

rules_python not able to build offline from vendor only directoryΒ #3707

@amfern

Description

@amfern

🐞 bug report

Description

rules_python isn't able to build things completely offline, and still downloads python dependencies even through everything supposed to be available in the --vendor_dir to support complete offline build.

πŸ”¬ Minimal Reproduction

git clone git@github.com:bazel-contrib/rules_python.git

# prepare vendor dir
docker run -e USER="$(id -u)" -u="$(id -u)" -v $HOME/.netrc:/.netrc -v $(pwd):/src/workspace -w /src/workspace/examples/bzlmod gcr.io/bazel-public/bazel:8.4.2 --output_user_root=/tmp/build_output vendor --vendor_dir=my_vendor_dir //...

# run build offline using only the data from my_vendor_dir
docker run --network none -e USER="$(id -u)" -u="$(id -u)" -v $HOME/.netrc:/.netrc -v $(pwd):/src/workspace -w /src/workspace/examples/bzlmod gcr.io/bazel-public/bazel:8.4.2 --output_user_root=/tmp/build_output build --lockfile_mode=error --vendor_dir=my_vendor_dir //...

πŸ”₯ Exception or Error


Extracting Bazel installation...
Starting local Bazel server and connecting to it...
Computing main repo mapping:
WARNING: For repository 'bazel_skylib', the root module requires module version bazel_skylib@1.7.1, but got bazel_skylib@1.8.2 in the resolved dependency graph.
WARNING: For repository 'platforms', the root module requires module version platforms@0.0.4, but got platforms@1.0.0 in the resolved dependency graph.
WARNING: For repository 'rules_shell', the root module requires module version rules_shell@0.3.0, but got rules_shell@0.6.1 in the resolved dependency graph.
Loading:
Loading: 0 packages loaded
WARNING: Download from https://pypi.org/simple/ failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException Unknown host: pypi.org
ERROR: /tmp/build_output_offline/6a4437257408fba6059e0f41368f396b/external/rules_python~/python/private/pypi/simpleapi_download.bzl:258:39: Traceback (most recent call last):
        File "/tmp/build_output_offline/6a4437257408fba6059e0f41368f396b/external/rules_python~/python/private/pypi/extension.bzl", line 378, column 25, in _pip_impl
                mods = parse_modules(
        File "/tmp/build_output_offline/6a4437257408fba6059e0f41368f396b/external/rules_python~/python/private/pypi/extension.bzl", line 267, column 30, in parse_modules
                builder.pip_parse(
        File "/tmp/build_output_offline/6a4437257408fba6059e0f41368f396b/external/rules_python~/python/private/pypi/hub_builder.bzl", line 60, column 47, in lambda
                pip_parse = lambda *a, **k: _pip_parse(self, *a, **k),
        File "/tmp/build_output_offline/6a4437257408fba6059e0f41368f396b/external/rules_python~/python/private/pypi/hub_builder.bzl", line 193, column 22, in _pip_parse
                _create_whl_repos(
        File "/tmp/build_output_offline/6a4437257408fba6059e0f41368f396b/external/rules_python~/python/private/pypi/hub_builder.bzl", line 542, column 50, in _create_whl_repos
                requirements_by_platform = parse_requirements(
        File "/tmp/build_output_offline/6a4437257408fba6059e0f41368f396b/external/rules_python~/python/private/pypi/parse_requirements.bzl", line 196, column 36, in parse_requirements
                index_urls = get_index_urls(
        File "/tmp/build_output_offline/6a4437257408fba6059e0f41368f396b/external/rules_python~/python/private/pypi/hub_builder.bzl", line 381, column 144, in lambda
                self._get_index_urls[python_version] = lambda ctx, distributions, *, index_url = None, extra_index_urls = None: self._simpleapi_download_fn(
        File "/tmp/build_output_offline/6a4437257408fba6059e0f41368f396b/external/rules_python~/python/private/pypi/simpleapi_download.bzl", line 92, column 31, in simpleapi_download
                dist_urls = _get_dist_urls(
        File "/tmp/build_output_offline/6a4437257408fba6059e0f41368f396b/external/rules_python~/python/private/pypi/simpleapi_download.bzl", line 164, column 43, in _get_dist_urls
                results[index_url] = download.wait()
        File "/tmp/build_output_offline/6a4437257408fba6059e0f41368f396b/external/rules_python~/python/private/pypi/simpleapi_download.bzl", line 258, column 39, in lambda
                result = download.wait(),
Error in wait: java.io.IOException: Error downloading [https://pypi.org/simple/] to /tmp/build_output_offline/6a4437257408fba6059e0f41368f396b/modextwd/rules_python~~pip/https___pypi_org_simple.html: Unknown host: pypi.org
ERROR: error evaluating module extension pip in @@rules_python~//python/extensions:pip.bzl
INFO: Elapsed time: 7.499s
INFO: 0 processes.
ERROR: Build did NOT complete successfully
FAILED:

🌍 Your Environment

**Operating System: **

  
ubuntu 24.04
  

Output of bazel version:

  
bazel 8.4.2
  

Rules_python version:

  
https://github.com/bazel-contrib/rules_python/commit/c33ba2383b7ce23f92fd97fbaa58132341dec2ca
  

Anything else relevant?

There is a test for building from vendor, however it allows network access which makes it ineffective.

- "bazel vendor --vendor_dir=./vendor //..."
- "bazel build --vendor_dir=./vendor //..."
- "rm -rf ./vendor"

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions