assertion when using --no-cache-dir in 19.0

[snstanton]

Environment

• pip version: 19.0
• Python version: 3.6.7
• OS: Linux 50de819ca3ba 4.9.125-linuxkit No instructions for how to install pip #1 SMP Fri Sep 7 08:20:28 UTC 2018 x86_64 GNU/Linux

Running in a dockerfile.

Description

The following command works with pip 18.1 and fails with 19.0.

pip3 install --no-cache-dir --upgrade -r requirements.txt

With 19.0, it fails with the following exception:

Exception:
Traceback (most recent call last):
  File "/Users/scotts/.virtualenvs/python3/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 176, in main
    status = self.run(options, args)
  File "/Users/scotts/.virtualenvs/python3/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 346, in run
    session=session, autobuilding=True
  File "/Users/scotts/.virtualenvs/python3/lib/python3.6/site-packages/pip/_internal/wheel.py", line 848, in build
    assert building_is_possible
AssertionError

Removing the --no-cache-dir flag causes the install to succeed.
requirements.txt

[Mohitsharma44]

Same thing happening with:
Python v3.6.8
pip version 18.1
on
Ubuntu:latest image.

@snstanton what base image are you using? I'm seeing a similar issue on pip v18.1 as well

[gst]

I've got the exact same issue.
on my side it seems it doesn't matter which package/distribution I try to install

[dan-blanchard]

I'm seeing this even without --no-cache-dir set. It happens for all packages I try to install, even if they're already installed.

• pip version: 19.0
• Python version: 3.6.0
• OS: Ubuntu 14.04.4 LTS (GNU/Linux 3.13.0-91-generic x86_64)

I should note that in my case I'm seeing it when running pip with a combination of sudo -H and bash -l -c.

$ sudo -H bash -l -c  "/data/virtualenvs/events_beta/bin/pip install hypothesis"
Looking in indexes: https://pypi.org/simple, http://pypi.lan.cogtree.com/cogtree/simple/
Collecting hypothesis
  Downloading http://pypi.lan.cogtree.com/cogtree/simple/hypothesis/hypothesis-4.1.0-py3-none-any.whl (238kB)
    100% |████████████████████████████████| 245kB 120.5MB/s
Requirement already satisfied: attrs>=16.0.0 in /data/virtualenvs/events_beta/lib/python3.6/site-packages (from hypothesis) (18.2.0)
Exception:
Traceback (most recent call last):
  File "/data/virtualenvs/events_beta/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 176, in main
    status = self.run(options, args)
  File "/data/virtualenvs/events_beta/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 346, in run
    session=session, autobuilding=True
  File "/data/virtualenvs/events_beta/lib/python3.6/site-packages/pip/_internal/wheel.py", line 848, in build
    assert building_is_possible
AssertionError

Running the same commands without -l on my bash -c, or without bash -l -c involved at all, it all works fine.

$ sudo -H bash -c  "/data/virtualenvs/events_beta/bin/pip install hypothesis"
Collecting hypothesis
  Downloading https://files.pythonhosted.org/packages/89/7b/d6206dcde963139daa03a1d85b0c3428cb3ebf2ae8de3244b14a63e22680/hypothesis-4.1.0.tar.gz (180kB)
    100% |████████████████████████████████| 184kB 33.7MB/s
Requirement already satisfied: attrs>=16.0.0 in /data/virtualenvs/events_beta/lib/python3.6/site-packages (from hypothesis) (18.2.0)
Building wheels for collected packages: hypothesis
  Building wheel for hypothesis (setup.py) ... done
  Stored in directory: /root/.cache/pip/wheels/10/12/eb/4ab734432e8466d545c8501f531458845b45e8c4427d5367f9
Successfully built hypothesis
Installing collected packages: hypothesis
Successfully installed hypothesis-4.1.0

Fascinatingly, if I run the same command without sudo or bash involved at all, it still fails with the same error, so it seems like it's some weird permissions issue maybe.

[tgs]

Another workaround for some situations

For people who hit this bug because virtualenv is automatically installing the latest version of pip, you can work around it by giving virtualenv the --no-download option, or setting VIRTUALENV_NO_DOWNLOAD=1.

But be aware that this may give you a very old version of pip, depending on the last time you upgraded virtualenv.

This also works with tox: VIRTUALENV_NO_DOWNLOAD=1 tox.

[gst]

for what it's worth : it also fails with same error if package is already installed:

gregory.starck@canon:~/tmp$ ./venv/bin/pip install --no-cache-dir six ; echo $?
Looking in indexes: http://pypi:3141/root/ax/+simple/
Requirement already satisfied: six in ./venv/lib/python3.6/site-packages (1.12.0)
Exception:
Traceback (most recent call last):
  File "/home/gregory.starck/tmp/venv/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 176, in main
    status = self.run(options, args)
  File "/home/gregory.starck/tmp/venv/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 346, in run
    session=session, autobuilding=True
  File "/home/gregory.starck/tmp/venv/lib/python3.6/site-packages/pip/_internal/wheel.py", line 848, in build
    assert building_is_possible
AssertionError
2
gregory.starck@canon:~/tmp$

[cherls]

Ran into the same issue. Ended up pinning the pip version as a fix for now.

pip install --upgrade pip==18.1

[D0han]

Issue is with failing assert, so setting env PYTHONOPTIMIZE=1 (or with parameter -O) makes this error go away.
Just tested it.
This workaround works because python optimizes code removing all asserts as one of the things.
Do not go for =2 (or -OO), as this removes docstrings and other tracebacks will appear - some code wants to operate on them.

[vinnyfuria]

It looks like someone knew this might end up being an issue (source):

        # TODO: This check fails if --no-cache-dir is set. And yet we
        #       might be able to build into the ephemeral cache, surely?
        building_is_possible = self._wheel_dir or (
            autobuilding and self.wheel_cache.cache_dir
        )
        assert building_is_possible

[burkestar]

#5884 looks like this is a related change that could have caused this?

[burkestar]

Seems like the pip maintainers should rollback the recent 19 release to address this breaking change?
19.0 release notes: https://github.com/pypa/pip/blob/master/NEWS.rst#190-2019-01-22

UPDATE: not trying to cast aspersions here, was just suggesting as one way to quickly unblock people affected by this seeing as the release had just happened. Rolling forward with a hotfix works too. I appreciate the hard work of the community that supports this mission critical tooling, and agree with the sentiments below about postmortems to learn from mistakes and prevent future issues. Meanwhile, we are doing the same internally, which means a liberal amount of hardpinning pip versions in all the places :)

[cjerdonek]

The PR adding the TODO comment also has this comment in reply: https://github.com/pypa/pip/pull/5743/files#r215832743

Based on that comment and also the commenter above saying that passing PYTHONOPTIMIZE=1 makes the error go away, it seems like simply removing the assertion might be the correct fix (independent of the question of rolling back).