Separate auth for REQ_URL and *.url

[BadLiveware]

Issue abstract

Right now REQ_USERNAME and REQ_PASSWORD is used for authenticating for both the REQ_URL and any configmaps/secrets with *.url key, I believe it would be good for these to be handled separately.

Our setup

We are using this from the grafana helm chart, with auth turned on on grafana we need to supply auth in order to hit the reload API on grafana for the different resources, and so we set REQ_USERNAME, REQ_PASSWORD.

We do however also download dashboards from grafana.com, e.g. https://grafana.com/api/dashboards/13329/revisions/latest/download, using the *.url notation. As of right now this will use the same configured auth REQ_USERNAME and REQ_PASSWORD as is used to hit the grafana reload api, since these are local accounts on our own self-hosted instances, this will not work.

Example solution

I think omitting auth by default for *.url and since secrets are an option to use, and basic auth is used, setting them in the value of the *.url, e.g. username:[email protected]/api/dashboards/13329/revisions/latest/download, would solve this issue.

Example request log:

{"time": "2023-06-19T10:01:40.386431+00:00", "msg": "GET request sent to https://grafana.com/api/dashboards/13329/revisions/latest/download. Response: 401 Unauthorized {\n  \"code\": \"InvalidCredentials\",\n  \"message\": \"Invalid password\",\n  \"requestId\": \"7ea4528f-e2b9-4279-a523-53a0bcf2acbb\"\n}", "level": "DEBUG"}

[smbambling]

I'm hitting a similar issue after testing an upgrade of Kube-Prometheus-Stack, I now can't pull from my public repositories using BitBucket as its trying to supply auth creds during the pull.

[smbambling]

Looking at the Grafana Helm chart it appears this commit breaks the ability to use *.url notation as it will alway set the REQ_USERNAME and REQ_PASSWORD env variables. The in turn triggers the fetch to use authorization which breaks pulling from some sources.

grafana/helm-charts@3bcfc47

[github-actions]

This issue has been automatically marked as stale because it has not had any activity in the last 60 days. Thank you for your contributions.

[aslafy-z]

I used kustomize to locally download the file as a workaround:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
configMapGenerator:
- name: tempo-operational-grafana-dashboard
  files:
  - tempo-operational-dashboard.json=https://raw.githubusercontent.com/grafana/tempo/v2.5.0/operations/tempo-mixin-compiled/dashboards/tempo-operational.json
  options:
    disableNameSuffixHash: true
    labels:
      grafana_dashboard: "1"

[github-actions]

This issue has been automatically marked as stale because it has not had any activity in the last 60 days. Thank you for your contributions.

[aslafy-z]

Not stale

[ChristianGeie]

First of all, thank you for your report.

To fully understand the problem: Setup the environment variable REQ_USERNAME + REQ_PASSWORD and using this variable to trigger the corresponding REQ_URL works without any issues.

1. In case you want to download something ending with *.url in a config map, the function _get_file_data_and_name() https://github.com/kiwigrid/k8s-sidecar/blob/master/src/resources.py#L67 where called. Out of this function the request() is in use, which also use REQ_USERNAME + REQ_PASSWORD as basic auth and this ends in a 401.

2. It's not entirely clear to me that the credentials are also used to pull images. Apparently I misunderstood you here.

[aslafy-z]

These credentials are also used to login at the local Grafana API to call for a reload. This issue proposes another mechanism so different credentials can be used for both options.