Compile error installing package (fabric2==2.6.0)

[dmac-au]

Description

I am still getting an error when installing fabric2==2.6.0 with pip 25.1.1 on Debian 11 running Python 3.9.2. I thought it would be fixed by #13359 but it is still happening.

It seems to be caused by the invoke 1.7.3 package, and when I specifically install that package first, it installs fine and I no longer get the error when installing fabric2.

Also even though this error occurs, pip freeze shows that invoke has been installed and running pip install fabric2==2.6.0 then works.

The error occurs installing 2.6.0, 2.7.0 and 2.7.1. Installation proceeds fine in pip 25.0.1.

Expected behavior

Package installs as expected.

pip version

25.1.1

Python version

3.9.2

OS

Debian 11 (Bullseye)

How to Reproduce

1. Install python3-venv and python3-pip on Debian 11
2. python3 -m venv venv && source venv/bin/activate
3. pip install --upgrade pip
4. pip install fabric2==2.6.0
5. Error occurs

Output

(venv) reivernet@debian11-uni:~/test_pip25.1.1$ pip install fabric2==2.6.0
Collecting fabric2==2.6.0
  Using cached fabric2-2.6.0-py2.py3-none-any.whl.metadata (2.5 kB)
Collecting invoke<2.0,>=1.3 (from fabric2==2.6.0)
  Using cached invoke-1.7.3-py3-none-any.whl.metadata (3.3 kB)
Collecting paramiko>=2.4 (from fabric2==2.6.0)
  Using cached paramiko-3.5.1-py3-none-any.whl.metadata (4.6 kB)
Collecting pathlib2 (from fabric2==2.6.0)
  Using cached pathlib2-2.3.7.post1-py2.py3-none-any.whl.metadata (3.5 kB)
Collecting bcrypt>=3.2 (from paramiko>=2.4->fabric2==2.6.0)
  Using cached bcrypt-4.3.0-cp39-abi3-manylinux_2_28_x86_64.whl.metadata (10 kB)
Collecting cryptography>=3.3 (from paramiko>=2.4->fabric2==2.6.0)
  Using cached cryptography-45.0.2-cp37-abi3-manylinux_2_28_x86_64.whl.metadata (5.7 kB)
Collecting pynacl>=1.5 (from paramiko>=2.4->fabric2==2.6.0)
  Using cached PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata (8.6 kB)
Collecting cffi>=1.14 (from cryptography>=3.3->paramiko>=2.4->fabric2==2.6.0)
  Using cached cffi-1.17.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting pycparser (from cffi>=1.14->cryptography>=3.3->paramiko>=2.4->fabric2==2.6.0)
  Using cached pycparser-2.22-py3-none-any.whl.metadata (943 bytes)
Collecting six (from pathlib2->fabric2==2.6.0)
  Using cached six-1.17.0-py2.py3-none-any.whl.metadata (1.7 kB)
Using cached fabric2-2.6.0-py2.py3-none-any.whl (53 kB)
Using cached invoke-1.7.3-py3-none-any.whl (216 kB)
Using cached paramiko-3.5.1-py3-none-any.whl (227 kB)
Using cached bcrypt-4.3.0-cp39-abi3-manylinux_2_28_x86_64.whl (284 kB)
Using cached cryptography-45.0.2-cp37-abi3-manylinux_2_28_x86_64.whl (4.5 MB)
Using cached cffi-1.17.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (445 kB)
Using cached PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
Using cached pathlib2-2.3.7.post1-py2.py3-none-any.whl (18 kB)
Using cached pycparser-2.22-py3-none-any.whl (117 kB)
Using cached six-1.17.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: invoke, six, pycparser, bcrypt, pathlib2, cffi, pynacl, cryptography, paramiko, fabric2
ERROR: Exception:
Traceback (most recent call last):
  File "/usr/lib/python3.9/py_compile.py", line 144, in compile
    code = loader.source_to_code(source_bytes, dfile or file,
  File "<frozen importlib._bootstrap_external>", line 853, in source_to_code
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/home/reivernet/test_pip25.1.1/venv/lib/python3.9/site-packages/invoke/vendor/yaml2/constructor.py", line 173
    except TypeError, exc:
                    ^
SyntaxError: invalid syntax

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.9/compileall.py", line 238, in compile_file
    ok = py_compile.compile(fullname, cfile, dfile, True,
  File "/usr/lib/python3.9/py_compile.py", line 150, in compile
    raise py_exc
py_compile.PyCompileError:   File "/home/reivernet/test_pip25.1.1/venv/lib/python3.9/site-packages/invoke/vendor/yaml2/constructor.py", line 173
    except TypeError, exc:
                    ^
SyntaxError: invalid syntax


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/reivernet/test_pip25.1.1/venv/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 105, in _run_wrapper
    status = _inner_run()
  File "/home/reivernet/test_pip25.1.1/venv/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 96, in _inner_run
    return self.run(options, args)
  File "/home/reivernet/test_pip25.1.1/venv/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 68, in wrapper
    return func(self, options, args)
  File "/home/reivernet/test_pip25.1.1/venv/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 459, in run
    installed = install_given_reqs(
  File "/home/reivernet/test_pip25.1.1/venv/lib/python3.9/site-packages/pip/_internal/req/__init__.py", line 83, in install_given_reqs
    requirement.install(
  File "/home/reivernet/test_pip25.1.1/venv/lib/python3.9/site-packages/pip/_internal/req/req_install.py", line 867, in install
    install_wheel(
  File "/home/reivernet/test_pip25.1.1/venv/lib/python3.9/site-packages/pip/_internal/operations/install/wheel.py", line 729, in install_wheel
    _install_wheel(
  File "/home/reivernet/test_pip25.1.1/venv/lib/python3.9/site-packages/pip/_internal/operations/install/wheel.py", line 615, in _install_wheel
    success = compileall.compile_file(path, force=True, quiet=True)
  File "/usr/lib/python3.9/compileall.py", line 255, in compile_file
    msg = err.msg.encode(sys.stdout.encoding,
TypeError: encode() argument 'encoding' must be str, not None

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[notatallshaw]

I can't reproduce, but I'm using my own installed Python 3.9 not the Debian version. Can you give the exact Python version? i.e. python -V output. (I see you did, I will test with 3.9.2 shortly).

The issue appears to be when compiling the bytecode of invoke, normally pip calls compileall.compile_file(path, force=True, quiet=True) and with those flags any errors are ignored. This makes me think there is a bug in the exact Python version or Python distribution you are using.

[notatallshaw]

I think this is caused by python/cpython#88832, and so I assume you can fix this by upgrading to Python 3.9.7+ (it is certainly fixed on more recent versions of Python 3.9).

Also because of the nature of the error you can run pip install fabric2==2.6.0 twice and you won't see the error on the second time.

@ichard26 I can reproduce this on Python 3.9.2 and it appears to be caused by the same bug that motivated your fix in #13364, but is related to when bytecode compiling fails, and your PR does not seem to catch this case, the library in question is vendoring Python 2 code. This error happens in 25.1, and 25.1.1, but not in 25.0.

[dmac-au]

Thanks for looking @notatallshaw. I will see if there is a reason the latest python package on Bullseye is 3.9.2 (probably because it's LTS) and see if the maintainers can update it.

If it's possible to fix it on this end it would also be appreciated, in case the Debian side fails. I don't want to have to maintain non distro packaged python versions on 700+ machines If I can avoid it.

Thanks again

[notatallshaw]

To set expectations, the pip 25.1 release cycle is closed, and this is unlikely to be urgent enough to reopen. Pip releases on a quarterly basis so the next release (25.2) will be in ~2-3 months. Also pip is a volunteer project, so it will depend on someone submitting an acceptable PR, we have no paid or full time resources.

So if workarounds aren't viable for you (upgrade Python, use --no-compile, install twice), and you are tied to the LTS version of Python that Debian distributes I recommend you stick with the LTS version of pip that Debian distributes and use Debian's packaging to upgrade pip, not pip to upgrade pip.