Can not read files from ssh repos #5829

KantorSerhiy · 2023-08-15T13:56:54Z

Issue description

Hey,
I have some problem with installing .zip files from ssh-repos.Maybe you can help me.

When I try to install from the public repository - everything works fine ->
pipenv install https://github.com/KantorSerhiy/test/releases/download/1.0.0/test.zip

but when the repository is private - I get the following ->
pipenv.patched.pip._internal.exceptions.NetworkConnectionError: 404 Client Error: Not Found for url: https://github.com/KantorSerhiy/test/releases/download/1.0.0/test.zip

I tried to do something like this ->
test = {url = "https://[email protected]/KantorSerhiy/test/releases/download/1.0.0/test.zip"},
but it doesn't work for me=(

Expected result

Everything will be installed and will work.

Actual result

pipenv.patched.pip._internal.exceptions.NetworkConnectionError: 404 Client Error: Not Found for url: https://github.com/KantorSerhiy/test/releases/download/1.0.0/test.zip

$ pipenv --support

Pipenv version: '2023.7.23'

Pipenv location: '/Users/skantor/.local/share/virtualenvs/work-E2tBrjiz/lib/python3.9/site-packages/pipenv'

Python location: '/Users/skantor/.local/share/virtualenvs/work-E2tBrjiz/bin/python'

OS Name: 'posix'

User pip version: '23.1.2'

user Python installations found:

PEP 508 Information:
{'implementation_name': 'cpython',
 'implementation_version': '3.9.17',
 'os_name': 'posix',
 'platform_machine': 'arm64',
 'platform_python_implementation': 'CPython',
 'platform_release': '22.5.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 22.5.0: Thu Jun  8 22:21:34 PDT '
                     '2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T8112',
 'python_full_version': '3.9.17',
 'python_version': '3.9',
 'sys_platform': 'darwin'}
System environment variables:

PATH

__CFBundleIdentifier

SHELL

TERM

USER

FIG_JETBRAINS_SHELL_INTEGRATION

TMPDIR

COMMAND_MODE

TERMINAL_EMULATOR

SSH_AUTH_SOCK

XPC_FLAGS

TERM_SESSION_ID

__CF_USER_TEXT_ENCODING

LOGNAME

LC_CTYPE

XPC_SERVICE_NAME

HOME

SHLVL

PWD

OLDPWD

VIRTUAL_ENV

PS1

_

PYTHONDONTWRITEBYTECODE

PIP_DISABLE_PIP_VERSION_CHECK

PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

PATH: /Users/skantor/.local/share/virtualenvs/work-E2tBrjiz/bin:/opt/homebrew/bin:/Library/Frameworks/Python.framework/Versions/3.11/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin

SHELL: /bin/zsh

PWD: /Users/skantor/work

VIRTUAL_ENV: /Users/skantor/.local/share/virtualenvs/work

Tasks

Give feedback

No tasks being tracked yet.

Options

The text was updated successfully, but these errors were encountered:

matteius · 2023-08-15T22:35:38Z

@KantorSerhiy I just checked this against #5793 and I believe that branch fixes the issue you describe. However if you could check as well that would be helpful.

KantorSerhiy · 2023-08-16T10:21:25Z

@matteius i just tried,I installed the latest from your draft-no-reqlib branch and performed
pipenv install https://github.com/KantorSerhiy/test/releases/download/1.0.0/test.zip
and I got the same error:

Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
Installing https://github.com/KantorSerhiy/test/releases/download/1.0.0/test.zip...
Resolving https://github.com/KantorSerhiy/test/releases/download/1.0.0/test.zip...
⠹ Installing...CRITICAL:pipenv.patched.pip._internal.network.download:HTTP error 404 while getting https://github.com/KantorSerhiy/test/releases/download/1.0.0/test.zip
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/bin/pipenv", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/cli/options.py", line 58, in main
    return super().main(*args, **kwargs, windows_expand_args=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/click/decorators.py", line 84, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/cli/command.py", line 233, in install
    do_install(
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/routines/install.py", line 221, in do_install
    pkg_requirement = Requirement.from_line(pkg_line)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2467, in from_line
    parsed_line.set_setup_info(parsed_line.get_setup_info())
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 791, in get_setup_info
    setup_info = SetupInfo.from_ireq(self.ireq, subdir=self.subdirectory)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1697, in from_ireq
    unpack_url(
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1837, in unpack_url
    file = get_http_url(
           ^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1802, in get_http_url
    from_path, content_type = download(link, temp_dir.path)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/patched/pip/_internal/network/download.py", line 134, in __call__
    resp = _http_get_download(self._session, link)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/patched/pip/_internal/network/download.py", line 118, in _http_get_download
    raise_for_status(resp)
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/patched/pip/_internal/network/utils.py", line 54, in raise_for_status
    raise NetworkConnectionError(http_error_msg, response=resp)
pipenv.patched.pip._internal.exceptions.NetworkConnectionError: 404 Client Error: Not Found for url: https://github.com/KantorSerhiy/test/releases/download/1.0.0/test.zip

unfortunately it doesn't work for me =(
maybe you still have guesses about this?

(if I manually click on the link( https://github.com/KantorSerhiy/test/releases/download/1.0.0/test.zip) - the download process begins)

matteius · 2023-08-17T01:33:04Z

@KantorSerhiy I think there is some confusion on the expectations here vcs+ssh:// dependencies work (at least in my branch I referenced) but they have to be the actual repository and not a link to the zip file -- a link to the zip file url would just be an installable remote file URI, but it should not require (and will not work) with the git+ssh:// prefix. But you should be able to do pipenv install git+ssh://${GITHUB_TOKEN}@github.com/KantorSerhiy/[email protected] (at leas ton the draft-no-reqslib branch) and it should manage it as installable vcs dependency. I hope this helps!

KantorSerhiy · 2023-08-17T14:24:46Z

this one is a bit not what I expect..

I will reveal a little more details:
I have a private repository that has a certain project structure. With the help of github actions, I form the structure I need(package) and immediately make a release. In the release files, I have a file with the extension .zip that I created during actions.
The main problem is that I can't install this file (.zip) into my project as a package (again, if it's a private repository. If it's public, then everything is fine).
I don't want to create a separate branch for this, I don't see the need, I have everything already packed in .zip

matteius · 2023-08-18T01:54:56Z

@KantorSerhiy Is there a regular pip install line format that works for this example of a private github repo zip file?

KantorSerhiy · 2023-08-18T10:49:09Z

no, that's why I wrote to you)

matteius · 2023-08-18T13:47:36Z

@KantorSerhiy based on our architecture that heavily utilizes pip, if it cannot be installed that way through pip it won't be installable that way through pipenv. I think the git+ssh:// repository install would fit your needs though -- it doesn't have to be a branch, it can be a specific commit hash you tag to.

More details on the forward looking architecture (which is mostly inline with the existing): #5818

KantorSerhiy · 2023-08-18T14:03:07Z

thank you for the answers.

I am attaching a screenshot.
the first is a private repository, the second I changed to public.

I don't understand why this is so and how it works.
what is the reason that the dependency is installed from the public repository without problems, but not from the private one.

P.S
same for
pipenv install https://github.com/KantorSerhiy/logica-test/releases/download/1.0.0/logica.zip
after the command above :
pipfile -> logica = {file = "https://github.com/KantorSerhiy/logica-test/releases/download/1.0.0/logica.zip"}

pipfile.lock ->

        "logica": {
            "editable": true,
            "file": "https://github.com/KantorSerhiy/logica-test/releases/download/1.0.0/logica.zip",
            "hashes": [
                "sha256:9e0ade2b10d4462401cc576c158eb5e6f6232ee8bf72281f49ab7a27ade95c1b"
            ],
            "markers": "python_version >= '3.0'"

matteius · 2023-08-18T14:11:54Z

what is the reason that the dependency is installed from the public repository without problems, but not from the private one.

There is no mechanism to try and authenticate the https url from within pip for the file, but there is a mechanism for working with private ssh vcs repositories. It will require you unlock the ssh key and then will be able to clone the repository from a certain commit or branch. I am not aware of an analogous way to install from a private git repository using just a file. You would have to publish a package to a private antifactory that supports keyring to do something like that, which is really something different from that.

matteius added Type: Question ❔ This is a question or a request for support. triage labels Aug 19, 2023

matteius closed this as not planned Won't fix, can't repro, duplicate, stale Aug 31, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can not read files from ssh repos #5829

Can not read files from ssh repos #5829

KantorSerhiy commented Aug 15, 2023 •

edited

Loading

Tasks

matteius commented Aug 15, 2023

KantorSerhiy commented Aug 16, 2023 •

edited

Loading

matteius commented Aug 17, 2023 •

edited

Loading

KantorSerhiy commented Aug 17, 2023

matteius commented Aug 18, 2023

KantorSerhiy commented Aug 18, 2023

matteius commented Aug 18, 2023

KantorSerhiy commented Aug 18, 2023

matteius commented Aug 18, 2023 •

edited

Loading

Can not read files from ssh repos #5829

Can not read files from ssh repos #5829

Comments

KantorSerhiy commented Aug 15, 2023 • edited Loading

Issue description

Expected result

Actual result

Tasks

matteius commented Aug 15, 2023

KantorSerhiy commented Aug 16, 2023 • edited Loading

matteius commented Aug 17, 2023 • edited Loading

KantorSerhiy commented Aug 17, 2023

matteius commented Aug 18, 2023

KantorSerhiy commented Aug 18, 2023

matteius commented Aug 18, 2023

KantorSerhiy commented Aug 18, 2023

matteius commented Aug 18, 2023 • edited Loading

KantorSerhiy commented Aug 15, 2023 •

edited

Loading

KantorSerhiy commented Aug 16, 2023 •

edited

Loading

matteius commented Aug 17, 2023 •

edited

Loading

matteius commented Aug 18, 2023 •

edited

Loading