Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Building cython with mach-nix stopped working, with no change in nixos version #538

Open
henrycrutcher opened this issue Feb 15, 2023 · 1 comment
Open

Building cython with mach-nix stopped working, with no change in nixos version #538

henrycrutcher opened this issue Feb 15, 2023 · 1 comment

Comments

@henrycrutcher
Copy link

With this file...
let
customizePython = import ./customizePython.nix;
nixpkgs = import {
overlays = [ customizePython ];
};
mach-nix = import (builtins.fetchGit {
url = "https://github.com/DavHau/mach-nix/";
ref = "master";
}) {
pkgs=nixpkgs;
};
myCython = (mach-nix.buildPythonPackage {
src="https://github.com/cython/cython/archive/refs/tags/3.0.0a11.tar.gz";
patches = [ ./cython-memoryview.patch ./cython-cygdb.patch ];
python="python310";
});
mypython = mach-nix.mkPython {
requirements = ''
sh
'';
packagesExtra = with nixpkgs ; [
myCython
llvmPackages_13.clang
hello
];
python = "python310";
providers.torch = "nixpkgs";
providers.blosc = "nixpkgs";
providers.c-blosc = "nixpkgs";
providers.tables = "nixpkgs";
_.tables.buildInputs.add = [ myCython ];
overridesPost = [(curr: prev: {
torch = prev.pytorchWithCuda.override {
cudaSupport = true;
};
numba = prev.numba.override {
cudaSupport = true;
};
cython = myCython;
})];
};
in
nixpkgs.buildEnv {
name="eyeserver";
paths = [
mypython
(nixpkgs.gdb.override{python3=mypython;})
];
}
I get the errors pasted below, which indicate that setuptools cannot be found. This is on nixos 22.11. I suspect this issue has nothing to do with cython, as the error message complains of no setuptools, which seems fundamental to python packaging (I'm not an expert here). Any workarounds

replacing old 'eyeserver'
installing 'eyeserver'
trace:
automatically detected pname: 'Cython'
trace:
automatically detected version: '3.0.0a11'
trace:
automatically detected requirements of Cython:3.0.0a11 22.11.2568.c43f676c938:

building '/nix/store/5a45lyax54lplga18pz7vnfn105z768h-python3.9-distlib-0.3.7.dev.0.drv'...
building '/nix/store/2y40cpgsw8ssk0s7w24p2v08w8iqccrr-python3.9-resolvelib-0.8.1.drv'...
building '/nix/store/w91pw3rkjk8kwqsbwkdikny26b4xfcz0-python3.9-tree-format-0.1.2.drv'...
Sourcing python-remove-tests-dir-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing pip-build-hook
Using pipBuildPhase
Using pipShellHook
Sourcing pip-install-hook
Using pipInstallPhase
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing python-namespaces-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-tests-dir-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing setuptools-build-hook
Using setuptoolsBuildPhase
Using setuptoolsShellHook
Sourcing pip-install-hook
Using pipInstallPhase
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing python-namespaces-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-tests-dir-hook
Sourcing python-catch-conflicts-hook.sh
Sourcing python-remove-bin-bytecode-hook.sh
Sourcing setuptools-build-hook
Using setuptoolsBuildPhase
Using setuptoolsShellHook
Sourcing pip-install-hook
Using pipInstallPhase
Sourcing python-imports-check-hook.sh
Using pythonImportsCheckPhase
Sourcing python-namespaces-hook
Sourcing python-catch-conflicts-hook.sh
unpacking sources
unpacking source archive /nix/store/ibrcwzm3wikip0v049ilxfjifwamr2ma-61534987ec5e4b3bdd346de8ffcc7aeaeb132572.zip
unpacking sources
unpacking source archive /nix/store/1lwdh43nlfskcb7589ainrzyisvcdr01-resolvelib-0.8.1.tar.gz
source root is resolvelib-0.8.1
unpacking sources
unpacking source archive /nix/store/nm09xrwsvq9h68fy188vn4bpg100g21k-tree-format-0.1.2.tar.gz
source root is tree-format-0.1.2
setting SOURCE_DATE_EPOCH to timestamp 1633986422 of file resolvelib-0.8.1/setup.cfg
patching sources
setting SOURCE_DATE_EPOCH to timestamp 1509954038 of file tree-format-0.1.2/tree_format.egg-info/top_level.txt
configuring
patching sources
configuring
no configure script, doing nothing
source root is distlib-61534987ec5e4b3bdd346de8ffcc7aeaeb132572
building
Executing setuptoolsBuildPhase
no configure script, doing nothing
building
setting SOURCE_DATE_EPOCH to timestamp 1667246889 of file distlib-61534987ec5e4b3bdd346de8ffcc7aeaeb132572/tox.ini
Executing setuptoolsBuildPhase
patching sources
configuring
no configure script, doing nothing
building
Executing pipBuildPhase
Creating a wheel...
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/tree_format
copying tree_format/_text.py -> build/lib/tree_format
copying tree_format/init.py -> build/lib/tree_format
creating build/lib/tree_format/tests
copying tree_format/tests/init.py -> build/lib/tree_format/tests
copying tree_format/tests/test_text.py -> build/lib/tree_format/tests
/nix/store/rny5442dzxz8aamj1r9k7ns4lqwqwc38-python3.9-setuptools-65.3.0/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
installing to build/bdist.linux-x86_64/wheel
running install
running bdist_wheel
running install_lib
running build
running build_py
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/tree_format
copying build/lib/tree_format/_text.py -> build/bdist.linux-x86_64/wheel/tree_format
copying build/lib/tree_format/init.py -> build/bdist.linux-x86_64/wheel/tree_format
creating build/bdist.linux-x86_64/wheel/tree_format/tests
copying build/lib/tree_format/tests/init.py -> build/bdist.linux-x86_64/wheel/tree_format/tests
copying build/lib/tree_format/tests/test_text.py -> build/bdist.linux-x86_64/wheel/tree_format/tests
running install_egg_info
creating build
creating build/lib
creating build/lib/resolvelib
copying src/resolvelib/init.py -> build/lib/resolvelib
copying src/resolvelib/providers.py -> build/lib/resolvelib
copying src/resolvelib/reporters.py -> build/lib/resolvelib
copying src/resolvelib/resolvers.py -> build/lib/resolvelib
copying src/resolvelib/structs.py -> build/lib/resolvelib
creating build/lib/resolvelib/compat
copying src/resolvelib/compat/init.py -> build/lib/resolvelib/compat
copying src/resolvelib/compat/collections_abc.py -> build/lib/resolvelib/compat
running egg_info
running egg_info
writing src/resolvelib.egg-info/PKG-INFO
writing dependency_links to src/resolvelib.egg-info/dependency_links.txt
writing tree_format.egg-info/PKG-INFO
writing requirements to src/resolvelib.egg-info/requires.txt
writing top-level names to src/resolvelib.egg-info/top_level.txt
writing dependency_links to tree_format.egg-info/dependency_links.txt
writing requirements to tree_format.egg-info/requires.txt
writing top-level names to tree_format.egg-info/top_level.txt
reading manifest file 'src/resolvelib.egg-info/SOURCES.txt'
reading manifest file 'tree_format.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
writing manifest file 'tree_format.egg-info/SOURCES.txt'
Copying tree_format.egg-info to build/bdist.linux-x86_64/wheel/tree_format-0.1.2-py3.9.egg-info
writing manifest file 'src/resolvelib.egg-info/SOURCES.txt'
running install_scripts
copying src/resolvelib/init.pyi -> build/lib/resolvelib
copying src/resolvelib/providers.pyi -> build/lib/resolvelib
copying src/resolvelib/py.typed -> build/lib/resolvelib
copying src/resolvelib/reporters.pyi -> build/lib/resolvelib
copying src/resolvelib/resolvers.pyi -> build/lib/resolvelib
copying src/resolvelib/structs.pyi -> build/lib/resolvelib
/nix/store/rny5442dzxz8aamj1r9k7ns4lqwqwc38-python3.9-setuptools-65.3.0/lib/python3.9/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
warnings.warn(
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/resolvelib
copying build/lib/resolvelib/init.py -> build/bdist.linux-x86_64/wheel/resolvelib
copying build/lib/resolvelib/providers.py -> build/bdist.linux-x86_64/wheel/resolvelib
copying build/lib/resolvelib/reporters.py -> build/bdist.linux-x86_64/wheel/resolvelib
copying build/lib/resolvelib/resolvers.py -> build/bdist.linux-x86_64/wheel/resolvelib
copying build/lib/resolvelib/structs.py -> build/bdist.linux-x86_64/wheel/resolvelib
creating build/bdist.linux-x86_64/wheel/resolvelib/compat
copying build/lib/resolvelib/compat/init.py -> build/bdist.linux-x86_64/wheel/resolvelib/compat
copying build/lib/resolvelib/compat/collections_abc.py -> build/bdist.linux-x86_64/wheel/resolvelib/compat
copying build/lib/resolvelib/init.pyi -> build/bdist.linux-x86_64/wheel/resolvelib
copying build/lib/resolvelib/providers.pyi -> build/bdist.linux-x86_64/wheel/resolvelib
copying build/lib/resolvelib/py.typed -> build/bdist.linux-x86_64/wheel/resolvelib
copying build/lib/resolvelib/reporters.pyi -> build/bdist.linux-x86_64/wheel/resolvelib
copying build/lib/resolvelib/resolvers.pyi -> build/bdist.linux-x86_64/wheel/resolvelib
copying build/lib/resolvelib/structs.pyi -> build/bdist.linux-x86_64/wheel/resolvelib
running install_egg_info
Copying src/resolvelib.egg-info to build/bdist.linux-x86_64/wheel/resolvelib-0.8.1-py3.9.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/tree_format-0.1.2.dist-info/WHEEL
creating 'dist/tree_format-0.1.2-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'tree_format/init.py'
adding 'tree_format/_text.py'
adding 'tree_format/tests/init.py'
adding 'tree_format/tests/test_text.py'
adding 'tree_format-0.1.2.dist-info/METADATA'
adding 'tree_format-0.1.2.dist-info/WHEEL'
adding 'tree_format-0.1.2.dist-info/top_level.txt'
adding 'tree_format-0.1.2.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
adding license file "LICENSE" (matched pattern "LICEN[CS]E*")
creating build/bdist.linux-x86_64/wheel/resolvelib-0.8.1.dist-info/WHEEL
creating 'dist/resolvelib-0.8.1-py2.py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'resolvelib/init.py'
adding 'resolvelib/init.pyi'
adding 'resolvelib/providers.py'
adding 'resolvelib/providers.pyi'
adding 'resolvelib/py.typed'
adding 'resolvelib/reporters.py'
adding 'resolvelib/reporters.pyi'
adding 'resolvelib/resolvers.py'
adding 'resolvelib/resolvers.pyi'
adding 'resolvelib/structs.py'
adding 'resolvelib/structs.pyi'
adding 'resolvelib/compat/init.py'
adding 'resolvelib/compat/collections_abc.py'
adding 'resolvelib-0.8.1.dist-info/LICENSE'
adding 'resolvelib-0.8.1.dist-info/METADATA'
adding 'resolvelib-0.8.1.dist-info/WHEEL'
adding 'resolvelib-0.8.1.dist-info/top_level.txt'
adding 'resolvelib-0.8.1.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Finished executing setuptoolsBuildPhase
installing
Executing pipInstallPhase
/build/tree-format-0.1.2/dist /build/tree-format-0.1.2
Finished executing setuptoolsBuildPhase
installing
Executing pipInstallPhase
/build/resolvelib-0.8.1/dist /build/resolvelib-0.8.1
WARNING: The directory '/homeless-shelter/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
Processing /build/distlib-61534987ec5e4b3bdd346de8ffcc7aeaeb132572
Running command Preparing metadata (pyproject.toml)
Preparing metadata (pyproject.toml) ... done
ERROR: Exception:
Traceback (most recent call last):
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
status = run_func(*args)
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
return func(self, options, args)
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_internal/commands/wheel.py", line 145, in run
requirement_set = resolver.resolve(reqs, check_supported_wheels=True)
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 73, in resolve
collected = self.factory.collect_root_requirements(root_reqs)
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 491, in collect_root_requirements
req = self._make_requirement_from_install_req(
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 453, in _make_requirement_from_install_req
cand = self._make_candidate_from_link(
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
self._link_candidate_cache[link] = LinkCandidate(
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 297, in init
super().init(
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 162, in init
self.dist = self._prepare()
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 231, in _prepare
dist = self._prepare_distribution()
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 308, in _prepare_distribution
return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 438, in prepare_linked_requirement
return self._prepare_linked_requirement(req, parallel_builds)
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 524, in _prepare_linked_requirement
dist = _get_prepared_distribution(
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 68, in _get_prepared_distribution
abstract_dist.prepare_distribution_metadata(
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_internal/distributions/sdist.py", line 61, in prepare_distribution_metadata
self.req.prepare_metadata()
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_internal/req/req_install.py", line 533, in prepare_metadata
self.metadata_directory = generate_metadata(
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_internal/operations/build/metadata.py", line 35, in generate_metadata
distinfo_dir = backend.prepare_metadata_for_build_wheel(metadata_dir)
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_internal/utils/misc.py", line 706, in prepare_metadata_for_build_wheel
return super().prepare_metadata_for_build_wheel(
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_vendor/pep517/wrappers.py", line 188, in prepare_metadata_for_build_wheel
return self._call_hook('prepare_metadata_for_build_wheel', {
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_vendor/pep517/wrappers.py", line 332, in _call_hook
raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
File "/nix/store/chr8vckllnvair2agf39alw2q0ygnk4b-python3.9-pip-22.2.2/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 89, in _build_backend
obj = import_module(mod_path)
File "/nix/store/hf4gib42cbnxz3fqvbh6wai51lix19sp-python3-3.9.16/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 972, in _find_and_load_unlocked
File "", line 228, in _call_with_frames_removed
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'setuptools'

error: builder for '/nix/store/5a45lyax54lplga18pz7vnfn105z768h-python3.9-distlib-0.3.7.dev.0.drv' failed with exit code 2;
last 10 log lines:
> File "", line 1030, in _gcd_import
> File "", line 1007, in _find_and_load
> File "", line 972, in _find_and_load_unlocked
> File "", line 228, in _call_with_frames_removed
> File "", line 1030, in _gcd_import
> File "", line 1007, in _find_and_load
> File "", line 984, in _find_and_load_unlocked
> ModuleNotFoundError: No module named 'setuptools'
>
>
For full logs, run 'nix log /nix/store/5a45lyax54lplga18pz7vnfn105z768h-python3.9-distlib-0.3.7.dev.0.drv'.
error: 1 dependencies of derivation '/nix/store/kyymqwkz6z0vdr34p2ivwdzwh31zamx1-python3-3.9.16-env.drv' failed to build
error: 1 dependencies of derivation '/nix/store/ap2a4cikxgc1mwjrzn1i7jba56dykhbp-mach_nix_file.drv' failed to build
(use '--show-trace' to show detailed ___location information)
@abathur
Copy link

abathur commented Feb 15, 2023

This looks like the same immediate issue as #524. (It would be good to get that problem fixed, and I imagine dav will be happy to merge a PR that does so.)

The broader issue is that you probably shouldn't just be using mach-nix from master if you expect it to be stable.

I've been running a project just fine with mach-nix pinned in my lock.toml as:

[mach-nix]
ref = "3.5.0"
rev = "7e14360bde07dcae32e5e24f366c83272f52923f"
sha256 = "185qf6d5xg8qk1hb1y0b5gggr71vdz8v9d5ga4zg7dmcb1aypxcg"

Your project will likely build if you pin the rev. If it doesn't, you might need to experiment with versions--if it has been a long time since you built the project, even 3.5.0 may have had some breaking changes relative to whatever rev master was at when you last built the project).

As an aside, you can format code for easier reading this:

```nix
mach-nix = import (builtins.fetchGit {
    url = "https://github.com/DavHau/mach-nix/";
    ref = "master";
}) {
    pkgs=nixpkgs;
};
```

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@henrycrutcher @abathur