Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Caching does not work when using an internal package repo #984

Open
3 tasks
screig opened this issue Dec 13, 2024 · 3 comments
Open
3 tasks

Caching does not work when using an internal package repo #984

screig opened this issue Dec 13, 2024 · 3 comments
Assignees
Labels
bug Something isn't working

Comments

@screig
Copy link

screig commented Dec 13, 2024

Description:
The caching feature wont work if one is pip installing from another python package repository.

Action version:
actions/setup-python@v5

Platform:

  • [X ] Ubuntu
  • macOS
  • Windows

Runner type:

  • [X ] Hosted
  • Self-hosted
    Current runner version: '2.321.0'
    Runner Image
    Image: ubuntu-22.04
    Version: 20241201.1.0

Tools version:
I think it applies across Python versions

Repro steps:
The code is on our internal repo so I can share a link, here is the yaml

First we set up Python, and specify that we wish to cache.

      - uses: actions/setup-python@v5
        with:
          python-version: '3.9'
          cache: 'pip'
          cache-dependency-path: requirements*.txt

Next I install two sets of requirements

      - name: Install dependencies f
        run: |
          echo "Python version is : ${{ matrix.python-version }}"
          pip --version
          pip install -r requirements_for_testing.txt        
          pip install -r requirements.txt --extra-index-url https://${ado_token}@OUR_INETERNAL_ADO_PACKAGE REPO/_packaging/sepypi/pypi/simple/
          pip --version
          pip list

requirements_for_testing.txt contains packages that we can get from pypi. Now here the caching works fine.

image

Next we need to install requirements that include our internal packages on our internal ADO package repo.

pip install -r requirements.txt --extra-index-url https://${ado_token}@OUR_INETERNAL_ADO_PACKAGE REPO/_packaging/sepypi/pypi/simple/

Here the caching never works.

image

Expected behaviour:
I would expect it to cache.

Actual behaviour:
Its not caching...

@screig screig added bug Something isn't working needs triage labels Dec 13, 2024
@jommeke22f
Copy link

jommeke22f commented Dec 15, 2024

Beschrijving: De cache functie werkt niet als pip wordt geïnstalleerd vanuit een andere Psychopathologie.

Actievere: actions/setup-python@voor

Platform:

  • Ubuntu-versie
  • macOS
  • Vensters

Type loper:

  • [X ] Gehost
  • Zelf gehost
    Huidige runner-versie: '2.321.0'
    Runner-afbeelding
    Afbeelding: ubuntu-22.04
    Versie: 20241201.1.0

Tools-versie: Ik denk dat het van toepassing is op alle Python-versies

Repro stappen: De code staat in onze interne repository, dus ik kan een link delen, hier is de yam

Eerst stellen we Python in en geven aan dat we willen cachen.

      - uses: actions/setup-python@vele
        with:
          python-version: '3.9'
          cache: 'pip'
          cache-dependency-path: requirements*.txt

Vervolgens installeer ik twee sets vereisten

      - name: Install dependencies f
        run: |
          echo "Python version is : ${{ matrix.python-version }}"
          pip --version
          pip install -r requirements_for_testing.txt        
          pip install -r requirements.txt --extra-index-url https://${ado_token}@OUR_INETERNAL_ADO_PACKAGE REPO/_packaging/Seppe/Pipi/simple/
          pip --version
          pip list

requirements_for_testing.txt bevat pakketten die we van papier kunnen krijgen. Hier werkt de caching prima.

afbeelding

Vervolgens moeten we de vereisten installeren die onze interne pakketten bevatten in onze interne Doktersassistente.

pip install -r requirements.txt --extra-index-url https://${ado_token}@OUR_INETERNAL_ADO_PACKAGE REPO/_packaging/Seppe/ppi/simple/

Hier werkt de caching nooit.

afbeelding

Verwacht gedrag: Ik verwacht dat het cachet.

Actueel gedrag: Er wordt niet gecached...

@gowridurgad
Copy link
Contributor

Hello @screig,
Thank you for creating this issue. We will investigate it and provide feedback as soon as we have some updates.

@lmvysakh lmvysakh self-assigned this Dec 16, 2024
@lmvysakh
Copy link

Hy, After investigating we found the following issues were causing the caching to fail, such as secret handling, syntax error in the "pip install -r requirements.txt --extra-index-url https://${ado_token}@OUR_INETERNAL_ADO_PACKAGE REPO/_packaging/sepypi/pypi/simple/" which was an invalid format for pip install.

So, the initial fix was made in the following way:

pip install -r requirements.txt --extra-index-url https://${{secrets.ado_token}}@OUR_INETERNAL_ADO_PACKAGE_REPO.git@main#egg=simple&subdirectory=_packaging/sepypi/pypi/simple

Caching was working when we tried it this way but there some deprecation happening while using --extra-index-url , so we fixed it using the following:

pip install -r requirements.txt --extra-index-url https://${{ secrets.ado_token }}@https://github.com

pip install git+@OUR_INETERNAL_ADO_PACKAGE_REPO.git@main#egg=simple&subdirectory=_packaging/sepypi/pypi/simple

So, we have addressed the following issues while trying to fix the caching problem.
1)Ensured that secrets are correctly referenced using ${{ secrets.ado_token }} in the workflow file to avoid "bad substitution" errors.
2)Used the git+https syntax to install packages directly from the repository.

If these workarounds do not resolve the issue, please provide a link to the build or the public repository to help us further investigate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants