-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Release v2023.7.23 not resolving all platform hashes when pulling from a custom index #5827
Comments
@matteius I followed the steps from #5793 (comment) to install your version. I confirmed the version is set to
I'm still seeing the same issue where the Pipfile.lock only has hashes for my OS. For our custom repo we are setting |
@jermbop I see, using a private index -- question, is there a recent prior version of pipenv that was resolving all hashes for a private index? |
I am also seeing this with version |
I was able to get this sorted on the
|
I seem to have your latest commit -> I'm still seeing the same behavior where my Pipfile.lock only has hashes for my OS. |
@jerempy Assuming your private index has an index page that can match the file names with the version being resolved, I would think it should work -- but you are also using |
https://pipenv-fork.readthedocs.io/en/latest/advanced.html#using-a-pypi-mirror To my understanding I can either run |
@jermbop thats a different docs link, I think it would be this one: https://pipenv.pypa.io/en/latest/indexes/#using-a-pypi-mirror
Yes that is correct, but my suspicion is that is also where there may still be a bug. Since I do not have access to your private pypi mirror, its harder for me to test this which is why I am asking if you can try to not use the pypi-mirror in testing my branch and instead change the Pipfile or lockfile source URL to be the mirror and run a couple checks. This will help me determine:
On my branch there should be good output with -v or --verbose that should show where its trying to obtain the hashes, if there is an error there that would also be useful to know, or maybe its skipping that because the source isn't in the lockfile directly. Its not really going to be too useful for me to test on main/current release because my future looking branch is likely to get merged soon, and so I'll want to find the path forward from starting from that code. Nothing stood out to me comparing Just to reiterate, I think there is a bug and it should be fixed, but it somewhat non-trivial and so some help pin-pointing it will help me do a better job with it. |
@matteius I removed Pipfile
Pipfile.lock
```
{
"_meta": {
"hash": {
"sha256": "f4733ffb52d998353d78a421bddc36b09d51a95a6779f203cf18c248734cd676"
},
"pipfile-spec": 6,
"requires": {
"python_version": "3.10"
},
"sources": [
{
"name": "pypi",
"url": "https://aws:{ourToken}@{ourCustomDomain}.d.codeartifact.us-east-2.amazonaws.com/pypi/python/simple/",
"verify_ssl": true
}
]
},
"default": {
"cffi": {
"hashes": [
"sha256:39d39875251ca8f612b6f33e6b1195af86d1b3e60086068be9cc053aa4376e21",
"sha256:d400bfb9a37b1351253cb402671cea7e89bdecc294e8016a707f6d1d8ac934f9"
],
"version": "==1.15.1"
},
"cryptography": {
"hashes": [
"sha256:652627a055cb52a84f8c448185922241dd5217443ca194d5739b44612c5e6507",
"sha256:6d192741113ef5e30d89dcb5b956ef4e1578f304708701b8b73d38e3e1461f34",
"sha256:8f09daa483aedea50d249ef98ed500569841d6498aa9c9f4b0531b9964658922"
],
"index": "pypi",
"markers": "python_version >= '3.7'",
"version": "==41.0.3"
},
"pycparser": {
"hashes": [
"sha256:8ee45429555515e1f6b185e78100aea234072576aa43ab53aefcae078162fca9",
"sha256:e644fdec12f7872f86c58ff790da456218b10f863970249516d60a5eaca77206"
],
"version": "==2.21"
}
},
"develop": {}
}
```
Verbose output of pipenv install -v --dev
```
pipenv install -v --dev
Using python: 3.10
Path to python: /Users/myusername/.pyenv/versions/3.10.2/bin/python3
Creating a virtualenv for this project...
Pipfile: /Users/myusername/workspace/test-cryptograph-all-platform-hashes/Pipfile
Using /Users/myusername/.pyenv/versions/3.10.2/bin/python3 (3.10.2) to create virtualenv...
⠇ Creating virtual environment...created virtual environment CPython3.10.2.final.0-64 in 474ms
creator CPython3Posix(dest=/Users/myusername/.local/share/virtualenvs/test-cryptograph-all-platform-hashe-eEkz4Ddq, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/myusername/Library/Application Support/virtualenv)
added seed packages: pip==23.2.1, setuptools==68.0.0, wheel==0.41.1
activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
✔ Successfully created virtual environment! Collecting cffi==1.15.1 (from -r /var/folders/6n/cgk3c7pn3kg97b5md65vw7bw0000gp/T/pipenv-idg8eukk-requirements/pipenv-jngmouuh-hashed-reqs.txt (line 1)) Using cached https://aws:{ourToken}@{ourCustomDomain}.d.codeartifact.us-east-2.amazonaws.com/pypi/python/simple/cffi/1.15.1/cffi-1.15.1-cp310-cp310-macosx_10_9_x86_64.whl (179 Collecting cryptography==41.0.3 (from -r /var/folders/6n/cgk3c7pn3kg97b5md65vw7bw0000gp/T/pipenv-idg8eukk-requirements/pipenv-jngmouuh-hashed-reqs.txt (line 2)) Using cached Collecting pycparser==2.21 (from -r /var/folders/6n/cgk3c7pn3kg97b5md65vw7bw0000gp/T/pipenv-idg8eukk-requirements/pipenv-jngmouuh-hashed-reqs.txt (line 3)) Using cached https://aws:{ourToken}@{ourCustomDomain}.d.codeartifact.us-east-2.amazonaws.com/pypi/python/simple/pycparser/2.21/pycparser-2.21-py2.py3-none-any.whl (118 kB) Installing collected packages: cffi, pycparser, cryptography Successfully installed cffi-1.15.1 cryptography-41.0.3 pycparser-2.21 Install Phase: Editable Requirements Installing dependencies from Pipfile.lock (4cd676)...
|
@jermbop thanks for the output, while There is logic to get the hash from the url of the package link from the index page if its available. Are you able to determine if the hashes are in the your index links (usually as query parameters at the end of the URLs). Also, are you sure that your private mirror has multiple wheels of the same package version for different platforms that it would be able to obtain? If not, it may be possible it was actually getting those extra hashes from pypi before ... really hard to make a determination yet. |
Thanks @jermbop -- that does indicate the URLs have the sha256 hashes as query parameters -- we can dig in more if its still broken, but I cut release |
@matteius Just tested with |
@jermbop I may have some time next weekend given its calling for rain here. If you click on my github profile, I have my email you can send me and I can invite you to a python developers slack group ... |
This is related also to my bug #5894 |
@matteius I tested changes from your branch |
|
Thanks!!
…On Thu, Sep 7, 2023, 10:38 AM Matt Davis ***@***.***> wrote:
2023.9.7 included this change.
—
Reply to this email directly, view it on GitHub
<#5827 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AC4M4INBFYBKVPZVKLQ2TMLXZHS7VANCNFSM6AAAAAA3NIY24E>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Issue description
After upgrading to 2023.7.23 my Pipfile.lock is not populated with all platform hashes when a custom repo is configured. Only the hashes for my OS platform are generated in Pipfile.lock. When I disabled the custom repo, all platform hashes are generated. I have not seen this behavior with earlier versions.
Expected result
I would expect all platform hashes to be generated in Pipfile.lock
Actual result
Hashes in Pipfile.lock are only for my current OS, where I'm running
pipenv install
Steps to replicate
Provide the steps to replicate (which usually at least includes the commands and the Pipfile).
pipenv install
$ pipenv --support
Pipenv version:
'2023.7.23'
Pipenv location:
'/Users/foo/.pyenv/versions/3.10.2/lib/python3.10/site-packages/pipenv'
Python location:
'/Users/foo/.pyenv/versions/3.10.2/bin/python3.10'
OS Name:
'posix'
User pip version:
'21.2.4'
user Python installations found:
PEP 508 Information:
System environment variables:
TERM_PROGRAM
PYENV_ROOT
TERM
SHELL
TMPDIR
TERM_PROGRAM_VERSION
AWS_CREDENTIAL_EXPIRATION
PIP_INDEX_URL
AWS_SESSION_TOKEN
TERM_SESSION_ID
AWS_VAULT
PYENV_VERSION
WORKSPACE
ZSH
USER
COMMAND_MODE
CODEARTIFACT_AUTH_TOKEN
SSH_AUTH_SOCK
PYENV_DIR
__CF_USER_TEXT_ENCODING
PAGER
LSCOLORS
PATH
LaunchInstanceID
AWS_DEFAULT_REGION
COMPOSE_FILES_DIR
__CFBundleIdentifier
PWD
AWS_SECRET_ACCESS_KEY
LANG
ITERM_PROFILE
AWS_REGION
PYENV_HOOK_PATH
XPC_FLAGS
RBENV_SHELL
PIPENV_PYPI_MIRROR
XPC_SERVICE_NAME
AWS_ACCESS_KEY_ID
PYENV_SHELL
SHLVL
HOME
COLORFGBG
LC_TERMINAL_VERSION
ITERM_SESSION_ID
LESS
LOGNAME
LC_TERMINAL
SECURITYSESSIONID
SQLITE_EXEMPT_PATH_FROM_VNODE_GUARDS
ASDF_HASHICORP_TERRAFORM_VERSION_FILE
COLORTERM
PYTHONDONTWRITEBYTECODE
PIP_DISABLE_PIP_VERSION_CHECK
PYTHONFINDER_IGNORE_UNSUPPORTED
Pipenv–specific environment variables:
PIPENV_PYPI_MIRROR
:https://aws:....codeartifact.us-east-2.amazonaws.com/pypi/foo-python/simple/
Debug–specific environment variables:
PATH
:/Users/foo/.pyenv/versions/3.8.7/bin:/Users/foo/.pyenv/versions/3.9.13/bin:/Users/foo/.pyenv/versions/3.10.2/bin:/Users/foo/.pyenv/versions/3.9.10/bin:/Users/foo/.pyenv/versions/3.8.10/bin:/Users/foo/.pyenv/versions/3.10.2/bin:/usr/local/Cellar/pyenv/2.3.16/libexec:/usr/local/Cellar/pyenv/2.3.16/plugins/python-build/bin:/Users/foo/.pyenv/shims:/Users/foo/bin:/Users/foo/.pyenv/bin:/Users/foo/.rbenv/shims:/Users/foo/bin:/Users/foo/.pyenv/bin:/Users/foo/.rbenv/shims:/Users/foo/.pyenv/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Users/foo/.asdf/shims:/Users/foo/.asdf/shims
SHELL
:/bin/zsh
LANG
:en_US.UTF-8
Contents of
Pipfile
('/Users/foo/workspace/foo-test-cryptograph-all-platform-hashes/Pipfile'):Contents of
Pipfile.lock
('/Users/foo/workspace/foo-test-cryptograph-all-platform-hashes/Pipfile.lock'):The text was updated successfully, but these errors were encountered: