Extension fails to register new environment variables added to .env file pointed to by the Python: Env File setting, rendering the extension useless

[bitadmiral]

Expected behavior

After I add a new environment variable into my .env file pointed to by the Python: Env File setting, and then reference that variable in a YAML file, I should not see problems reported in my db_project.yml file or in the Problems pane of VS Code.

Actual behavior

After I add a new variable to my .env file, which is pointed to by Python: Env File setting, and after I reference this new variable in a YAML file, I see "problem" notifications in my dbt_project.yml file as well as in the Problems pane of VS Code.
The problem state is:
Env var required but not provided: <ENVIRONMENT_VARIABLE_NAME>
Here is a screenshot:

The variable is actually detected by the extension when using the command "Print environment variables", but the errors don't go away:

Unfortunately, sometimes the "Print environment variables" command fails to display anything in the terminal. It opens but stays blank like this:

To work around that display issue, navigate to the Output pane to view the Log - dbt output, which shows the environment variable is indeed set as above.

Steps To Reproduce

1. Check that the Python: Env File setting is pointing at my .env file, as below (default value):
   "python.envFile": "${workspaceFolder}/.env"
2. Set a new variable in the .env file, for example:
   
3. Reference this new variable in a YAML file as below,

version: 2
sources:
  # See:  https://github.com/duckdb/dbt-duckdb#reading-from-external-files
  - name: socially_determined
    meta: 
      external_location: "read_csv(['{{ env_var('MEDEGRAPH_AUX_SOURCES_ROOT') }}/{name}/*.parquet'])"

4. Wait about a minute for the errors to show up in the IDE. The dbt_project.yml tab will turn red, and the Problems pane will show the error: "Env var required but not provided: 'MEDEGRAPH_AUX_SOURCES_ROOT'"
5. In the VS Code command palette, run command dbt Power User: Print Environment variables.
6. Observe that the environment variable in question is present in the output and set to the intended value.
7. There is no way to make the errors go away.
8. Open any model and try to run the command Validate SQL:
   
9. Observe the error popup "unable to build project":
   
10. In a new terminal window scoped to the same dbt project, run the command dbt build
11. Observe that the project can actually be built without any errors:

PS C:\Users...> dbt build
05:14:23 Running with dbt=1.8.4
05:14:28 Registered adapter: duckdb=1.8.2
05:14:29 Found 2 models, 6 sources, 410 macros
05:14:29
05:14:29 Concurrency: 1 threads (target='dev')
05:14:29
05:14:29 1 of 2 START sql table model main.enrollment ................................... [RUN]
05:14:29 1 of 2 OK created sql table model main.enrollment .............................. [OK in 0.19s]
05:14:29 2 of 2 START sql table model main.v_members .................................... [RUN]
05:14:29 2 of 2 OK created sql table model main.v_members ............................... [OK in 0.14s]
05:14:30
05:14:30 Finished running 2 table models in 0 hours 0 minutes and 0.87 seconds (0.87s).
05:14:30
05:14:30 Completed successfully
05:14:30
05:14:30 Done. PASS=2 WARN=0 ERROR=0 SKIP=0 TOTAL=2

Log output/Screenshots

I believe the extension is failing to detect changes in the .env file, and therefore not registering any new environment variables added to that file. Therefore, it is impossible to use any dbt Power User commands in the command palette.

Operating System

Windows 11

dbt version

1.8.4

dbt Adapter

duckdb

dbt Power User version

0.43.2

Are you willing to submit PR?

• Yes I am willing to submit a PR!

[anandgupta42]

@verticauser810
Re: After I add a new variable to my .env file, which is pointed to by Python: Env File setting, and after I reference this new variable in a YAML file, I see "problem" notifications in my dbt_project.yml file as well as in the Problems pane of VS Code.

Did you restart vscode after setting the environment variable as mentioned in the document?

Re: Unfortunately, sometimes the "Print environment variables" command fails to display anything in the terminal. It opens but stays blank like this:

This is a known problem with some of the OS and you will have run the command twice.

Docs: https://docs.myaltimate.com/setup/optConfig/#listing-environment-variables-detected-by-the-extension

[bitadmiral]

Thank you @anandgupta42 for the tip on getting the environment variables printed in the terminal.

I did read that note in the documentation about setting environment variables in the shell setup files, but that should apply only for environment variables set in shell setup files. (Likewise, if I was setting environment variables in Windows using the Environment Variables control panel or in my PowerShell setup script, then I would expect to be required to restart VS Code.) That makes sense one would need to restart VS Code when setting variables in those shell setup files, but it doesn't make sense when setting variables in a Python .env file. This is because .env files are not read at the shell level but at the application level (VS Code). Many other VS Code operations will automatically respect environment variables set in the Python .env files, such as opening a new terminal within VS Code, running tasks, debug configurations, etc., and none of those require restarting VS Code. The dbt Power User extension shouldn't require this either because it gives the impression the extension isn't working as soon as you start using it, which is frustrating and a time-waster, and I really want more people to like this extension.

Also, the instructions are a bit confusing because they mention restarting VS Code in the same paragraph that leads with the sentence about the Linux shell setup files. Therefore, it's not clear to the reader that it needs to be done every time you modify your Python .env file, which can be quite often.