Rscript not found

[jaskozi]

I have a problem when I want to commit after activating precommit with use_precommit()

I have the latest version, did the autoupdate, everything described here: https://lorenzwalthert.github.io/precommit/articles/precommit.html

My .pre-commit-config.yaml is the default one created after executing use_precommit()

The log file is as follows:

pre-commit version: 2.20.0
git --version: git version 2.33.0.windows.2
sys.version:
    3.9.7 | packaged by conda-forge | (default, Sep 23 2021, 07:24:41) [MSC v.1916 64 bit (AMD64)]
sys.executable: C:\Users\username\AppData\Local\MAMBAF~1\python.exe
os.name: nt
sys.platform: win32

### error information

An unexpected error has occurred: CalledProcessError: command: ('C:/PROGRA~1/R/R-41~1.0\\bin\\Rscript', '--vanilla', '-e', '    options(install.packages.compile.from.source = "never")\n                prefix_dir <- \'C:\\\\Users\\\\username\\\\.cache\\\\pre-commit\\\\repoz11tusy_\'\n            options(\n                repos = c(CRAN = "https://cran.rstudio.com"),\n                renv.consent = TRUE\n            )\n            source("renv/activate.R")\n            renv::restore()\n            activate_statement <- paste0(\n              \'suppressWarnings({\',\n              \'old <- setwd("\', getwd(), \'"); \',\n              \'source("renv/activate.R"); \',\n              \'setwd(old); \',\n              \'renv::load("\', getwd(), \'");})\'\n            )\n            writeLines(activate_statement, \'activate.R\')\n            is_package <- tryCatch(\n              {\n                  path_desc <- file.path(prefix_dir, \'DESCRIPTION\')\n                  suppressWarnings(desc <- read.dcf(path_desc))\n                  "Package" %in% colnames(desc)\n              },\n              error = function(...) FALSE\n            )\n            if (is_package) {\n                renv::install(prefix_dir)\n            }\n            \n    ')
return code: 1
expected return code: 0
stdout:
    Executable `C:/PROGRA~1/R/R-41~1.0\bin\Rscript` not found
stderr: (none)


Traceback (most recent call last):
  File "C:\Users\username\AppData\Roaming\Python\Python39\site-packages\pre_commit\error_handler.py", line 73, in error_handler
    yield
  File "C:\Users\username\AppData\Roaming\Python\Python39\site-packages\pre_commit\main.py", line 358, in main
    return hook_impl(
  File "C:\Users\username\AppData\Roaming\Python\Python39\site-packages\pre_commit\commands\hook_impl.py", line 254, in hook_impl
    return retv | run(config, store, ns)
  File "C:\Users\username\AppData\Roaming\Python\Python39\site-packages\pre_commit\commands\run.py", line 424, in run
    install_hook_envs(to_install, store)
  File "C:\Users\username\AppData\Roaming\Python\Python39\site-packages\pre_commit\repository.py", line 223, in install_hook_envs
    _hook_install(hook)
  File "C:\Users\username\AppData\Roaming\Python\Python39\site-packages\pre_commit\repository.py", line 79, in _hook_install
    lang.install_environment(
  File "C:\Users\username\AppData\Roaming\Python\Python39\site-packages\pre_commit\languages\r.py", line 139, in install_environment
    cmd_output_b(
  File "C:\Users\username\AppData\Roaming\Python\Python39\site-packages\pre_commit\util.py", line 146, in cmd_output_b
    raise CalledProcessError(returncode, cmd, retcode, stdout_b, stderr_b)
pre_commit.util.CalledProcessError: command: ('C:/PROGRA~1/R/R-41~1.0\\bin\\Rscript', '--vanilla', '-e', '    options(install.packages.compile.from.source = "never")\n                prefix_dir <- \'C:\\\\Users\\\\username\\\\.cache\\\\pre-commit\\\\repoz11tusy_\'\n            options(\n                repos = c(CRAN = "https://cran.rstudio.com"),\n                renv.consent = TRUE\n            )\n            source("renv/activate.R")\n            renv::restore()\n            activate_statement <- paste0(\n              \'suppressWarnings({\',\n              \'old <- setwd("\', getwd(), \'"); \',\n              \'source("renv/activate.R"); \',\n              \'setwd(old); \',\n              \'renv::load("\', getwd(), \'");})\'\n            )\n            writeLines(activate_statement, \'activate.R\')\n            is_package <- tryCatch(\n              {\n                  path_desc <- file.path(prefix_dir, \'DESCRIPTION\')\n                  suppressWarnings(desc <- read.dcf(path_desc))\n                  "Package" %in% colnames(desc)\n              },\n              error = function(...) FALSE\n            )\n            if (is_package) {\n                renv::install(prefix_dir)\n            }\n            \n    ')
return code: 1
expected return code: 0
stdout:
    Executable `C:/PROGRA~1/R/R-41~1.0\bin\Rscript` not found
stderr: (none)

The executable Rscript in the named folder is present, I checked manually
Thank you for your time!

[lorenzwalthert]

Did you run pre-commit clean?

[jaskozi]

Before filing a bug

• I have installed the latest dev version of {precommit} with remotes::install_github('lorenzwalthert/precommit')
• I have installed the latest hook revisions (update with precommit::autoupdate())
• I have installed the latest release of the upstream Python framework pre-comit as described under the update instructions.

Describe the bug
I did run pre-commit clean, cleared the file in .git/hooks/, also tried reinstalling, and I also did try via pip and conda, still got the same message. I also created a new blank project, wanted to commit a readme.md, got the same error message

Additional context

• My operating system is: Windows 10
• My project uses {renv}.

[lorenzwalthert]

To me, this looks suspicious:

pre_commit.util.CalledProcessError: command: ('C:/PROGRA~1/R/R-41~1.0\\bin\\Rscript', '--vanilla', '-e', '    ...

I.e. is pre-commit shortening the path to the executable somewhere and then you have a string with a bunch of ~ in it?

[lorenzwalthert]

Plus the direction of the slashes changes: First forward, than backward slashes...

[jaskozi]

Yes it is, because the actual path is C:\Program Files\R\R-4.1.0\bin, and I don't understand why is it shortening

[jaskozi]

A small update:

The problem is only present if I use this repo. If I remove it from the hooks, I can use precommit without problems.

Hopefully this helps.

[lorenzwalthert]

What is this repo? Do you mean if you put github.com/lorenzwalthert/precommit into your .pre-commit-config.yaml?

[lorenzwalthert]

Just to confirm? Are you using {renv}, as indicated in the issue description? Are you using pre-commit from inside RStudio or from where?

[jaskozi]

What is this repo? Do you mean if you put github.com/lorenzwalthert/precommit into your .pre-commit-config.yaml?

Yes exactly. And I am using renv.

[lorenzwalthert]

From inside RStudio?

[jaskozi]

From inside RStudio?

Yes, from inside RStudio.

[lorenzwalthert]

Can you try outside of RStudio $ git commit ...?

[jaskozi]

Can you try outside of RStudio $ git commit ...?

Same error.

[lorenzwalthert]

Can you set the environment variable R_HOME to the full path (C:\Program Files\R\R-4.1.0) and try again? In unix, I would

export R_HOME=/path/to/r/home

Because pre-commit then appends "/bin/Rscript" to R_HOME when running the hooks.

[lorenzwalthert]

If that works, we know that your env variable R_HOME is wrongly set.

[jaskozi]

If that works, we know that your env variable R_HOME is wrongly set.

I've set it, re-installed everything, still not working

[lorenzwalthert]

So the error message is the same?

[lorenzwalthert]

I think your problem is not specific to this repo, but more of an upstream pre-commit problem. I would like to help you debugging the problem but I don't have a windows machine available. I recommend you create a minimal reproducible example in a repo that does not use {renv} and post it upstream.

[lorenzwalthert]

Maybe also relevant:

• https://stackoverflow.com/questions/38242324/how-can-i-set-r-home-and-or-r-home-correctly
• https://stackoverflow.com/questions/11420689/how-to-get-long-file-system-path-from-python-on-windows

I.e. maybe a path containing ~ is valid, but maybe shouldn't be manipulated as we do with os.path.join().

[lorenzwalthert]

alternatively, can you unset R_HOME env variable and see if git commit works?

set FOOBAR=

No need to re-install anything. Because then, you should just get Rscript as the executable, not the whole path.

[jaskozi]

Yes, the error message is still the same when I set R_HOME, as well when I unset it. Also, in R, global options, R version, I have selected the path without the ~, still didn't help.

Thank you for trying to help

[lorenzwalthert]

@EzicJ someone else raised the issue and I am confident that we'll solve it over in pre-commit/pre-commit#2599.

[lorenzwalthert]

@EzicJ version 2.21.0 of pre-commit was released, which should contain a fix for this issue. I suggest you install that one and see if the problem persists.

[jaskozi]

Thank you for the information, unfortunately, after updating to v2.21.0 (and also clearing cache, re-installing, trying in different projects) now when I want to include the R pre-commit repo in the yaml, this is the error message

### version information


pre-commit version: 2.21.0
git --version: git version 2.33.0.windows.2
sys.version:
    3.9.7 | packaged by conda-forge | (default, Sep 23 2021, 07:24:41) [MSC v.1916 64 bit (AMD64)]
sys.executable: C:\Users\<username>\AppData\Local\mambaforge\python.exe
os.name: nt
sys.platform: win32


### error information


An unexpected error has occurred: CalledProcessError: command: ('C:/PROGRA~1/R/R-41~1.0\\bin\\Rscript.exe', '--vanilla', '-e', '    options(install.packages.compile.from.source = "never", pkgType = "binary")\n                prefix_dir <- \'C:\\\\Users\\\\<username>\\\\.cache\\\\pre-commit\\\\repodcp9_orq\'\n            options(\n                repos = c(CRAN = "https://cran.rstudio.com"),\n                renv.consent = TRUE\n            )\n            source("renv/activate.R")\n            renv::restore()\n            activate_statement <- paste0(\n              \'suppressWarnings({\',\n              \'old <- setwd("\', getwd(), \'"); \',\n              \'source("renv/activate.R"); \',\n              \'setwd(old); \',\n              \'renv::load("\', getwd(), \'");})\'\n            )\n            writeLines(activate_statement, \'activate.R\')\n            is_package <- tryCatch(\n              {\n                  path_desc <- file.path(prefix_dir, \'DESCRIPTION\')\n                  suppressWarnings(desc <- read.dcf(path_desc))\n                  "Package" %in% colnames(desc)\n              },\n              error = function(...) FALSE\n            )\n            if (is_package) {\n                renv::install(prefix_dir)\n            }\n            \n    ')
return code: 1
stdout: (none)
stderr:
    During startup - Warning message:
    Setting LC_CTYPE=C.UTF-8 failed 
    Error in options(install.packages.compile.from.source = never, pkgType = binary) : 
      object 'never' not found
    Execution halted



Traceback (most recent call last):
  File "C:\Users\<username>\AppData\Local\mambaforge\lib\site-packages\pre_commit\error_handler.py", line 73, in error_handler
    yield
  File "C:\Users\<username>\AppData\Local\mambaforge\lib\site-packages\pre_commit\main.py", line 366, in main
    return hook_impl(
  File "C:\Users\<username>\AppData\Local\mambaforge\lib\site-packages\pre_commit\commands\hook_impl.py", line 254, in hook_impl
    return retv | run(config, store, ns)
  File "C:\Users\<username>\AppData\Local\mambaforge\lib\site-packages\pre_commit\commands\run.py", line 428, in run
    install_hook_envs(to_install, store)
  File "C:\Users\<username>\AppData\Local\mambaforge\lib\site-packages\pre_commit\repository.py", line 223, in install_hook_envs
    _hook_install(hook)
  File "C:\Users\<username>\AppData\Local\mambaforge\lib\site-packages\pre_commit\repository.py", line 79, in _hook_install
    lang.install_environment(
  File "C:\Users\<username>\AppData\Local\mambaforge\lib\site-packages\pre_commit\languages\r.py", line 140, in install_environment
    cmd_output_b(
  File "C:\Users\<username>\AppData\Local\mambaforge\lib\site-packages\pre_commit\util.py", line 143, in cmd_output_b
    raise CalledProcessError(returncode, cmd, stdout_b, stderr_b)
pre_commit.util.CalledProcessError: command: ('C:/PROGRA~1/R/R-41~1.0\\bin\\Rscript.exe', '--vanilla', '-e', '    options(install.packages.compile.from.source = "never", pkgType = "binary")\n                prefix_dir <- \'C:\\\\Users\\\\<username>\\\\.cache\\\\pre-commit\\\\repodcp9_orq\'\n            options(\n                repos = c(CRAN = "https://cran.rstudio.com"),\n                renv.consent = TRUE\n            )\n            source("renv/activate.R")\n            renv::restore()\n            activate_statement <- paste0(\n              \'suppressWarnings({\',\n              \'old <- setwd("\', getwd(), \'"); \',\n              \'source("renv/activate.R"); \',\n              \'setwd(old); \',\n              \'renv::load("\', getwd(), \'");})\'\n            )\n            writeLines(activate_statement, \'activate.R\')\n            is_package <- tryCatch(\n              {\n                  path_desc <- file.path(prefix_dir, \'DESCRIPTION\')\n                  suppressWarnings(desc <- read.dcf(path_desc))\n                  "Package" %in% colnames(desc)\n              },\n              error = function(...) FALSE\n            )\n            if (is_package) {\n                renv::install(prefix_dir)\n            }\n            \n    ')
return code: 1
stdout: (none)
stderr:
    During startup - Warning message:
    Setting LC_CTYPE=C.UTF-8 failed 
    Error in options(install.packages.compile.from.source = never, pkgType = binary) : 
      object 'never' not found
    Execution halted

[lorenzwalthert]

Ok I don't get that. Because clearly, in the source code, the word never is in quotes. Maybe still some problem with the slashes in the executable ('C:/PROGRA~1/R/R-41~1.0\\bin\\Rscript.exe')? Like can you try

C:/PROGRA~1/R/R-41~1.0\\bin\\Rscript.exe -e "print('this')"
# vs 
C:/PROGRA~1/R/R-41~1.0/bin/Rscript.exe -e "print('this')"
# vs 
C:\\PROGRA~1\\R\\R-41~1.0\\bin\\Rscript.exe -e "print('this')"

Just to see if it's worth re-raising that in the upstream repo?

[wtimmerman-fitp]

Is anyone else still having this issue? Hope i'm not hijacking the issue thread, but seems the same. After doing the precommit setup and trying to commit a minor commit, i run into the issues described below.

I have:

• Done a fresh install of precommit (R package) from CRAN on R 4.2.2. Using renv. Also installed current main branch from github (0.3.2.9008) and got same errors below.
• Done a fresh install of pre-commit 3.2.0, which includes the earlier R fixes here (Adapt extension of R executable depending on os pre-commit/pre-commit#2605) and (Rscript not found in Windows pre-commit/pre-commit#2599). I get warnings about needing to add things to PATH on that install, but i don't think that's the issue here. On Windows, used pip3 install pre-commit --user outside a conda environment.
• Ran precommitt:use_precommit() and precommit::autoupdate()
• Used the basic .pre-commit-config.yaml without modification.
• Updated RHOME in my project-level renviron to change from short path to long/fully specified path (same error with and without this).
• Committed not from RStudio, but just via sourcetree GUI. Error message below. Inserted a browser() in code to test.

I'm pretty stumped now--is there anything else I can try or more info i can provide to help on debugging? Happy to play guinea pig, though I'm very new to pre-commit.

`

git -c diff.mnemonicprefix=false -c core.quotepath=false --no-optional-locks commit -q -F C:\Users\myname\AppData\Local\Temp\eu0ueh5n.mvf
style-files..............................................................Failed

• hook id: style-files
• exit code: 1

Executable Rscript not found

spell-check..........................................(no files to check)Skipped

lintr....................................................................Failed

• hook id: lintr
• exit code: 1

Executable Rscript not found

readme-rmd-rendered..................................(no files to check)Skipped

parsable-R...............................................................Failed

• hook id: parsable-R
• exit code: 1

Executable Rscript not found

no-browser-statement.....................................................Failed

• hook id: no-browser-statement
• exit code: 1

Executable Rscript not found

no-debug-statement.......................................................Failed

• hook id: no-debug-statement
• exit code: 1

Executable Rscript not found

check for added large files..............................................Passed
fix end of files.........................................................Passed
check pre-commit.ci config...........................(no files to check)Skipped
Don't commit common R artifacts......................(no files to check)Skipped

Completed with errors, see above.
`

[lorenzwalthert]

Thanks @wtimmerman-fitp for offering to play guinea pig. Appreciate it. This seems an annoying problem indeed.

Updated RHOME in my project-level renviron to change from short path to long/fully specified path (same error with and without this).

.Renviron is not used when to pre-commit invokes the hooks.

I get warnings about needing to add things to PATH on that install, but i don't think that's the issue here.

Can you post the warnings here anyways? Because as your error message says, the executable RScript is not found, which can be because it's not on the path or for other reasons. Did you ensure that your latest pre-commit installation is first on the path? And if yes, did you pre-commit install in the repo you want to use (to ensure that .git/hooks/pre-commit is populated by the pre-commit framework)?

Do you use the hooks inside a project that uses {renv} or not?

[wtimmerman-fitp]

Long story short, same errors appearing.

Responding on your points:

Updated RHOME in my project-level renviron to change from short path to long/fully specified path (same error with and without this).

.Renviron is not used when to pre-commit invokes the hooks.

🤦‍♂️ yes that makes sense.

I get warnings about needing to add things to PATH on that install, but i don't think that's the issue here.
Can you post the warnings here anyways?

I reinstalled into a random conda env just to get all of the original path warnings to show up, here they are (note that in practice i am suing python 3.9.1 outside a conda env). Bolded what looks relevant. WHen i googled about this earlier i didn't see anything about changing PATH in the docs or online, so i let it be without monkeying with it. Maybe something for the docs for newbies.

WARNING: The script nodeenv.exe is installed in 'C:\Users\myname\AppData\Roaming\Python\Python37\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The script identify-cli.exe is installed in 'C:\Users\myname\AppData\Roaming\Python\Python37\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The script virtualenv.exe is installed in 'C:\Users\myname\AppData\Roaming\Python\Python37\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
WARNING: The scripts pre-commit-validate-config.exe, pre-commit-validate-manifest.exe and pre-commit.exe are installed in 'C:\Users\myname\AppData\Roaming\Python\Python37\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

Did you ensure that your latest pre-commit installation is first on the path? And if yes, did you pre-commit install in the repo you want to use (to ensure that .git/hooks/pre-commit is populated by the pre-commit framework)?
Anyway, i put the Python scripts path into path. Then i ran

C:\Users\myname\Documents\projects_local\myrepo>pre-commit install

and got

pre-commit installed at .git\hooks\pre-commit

I didn't put it first on path, but given that the install works, seems okay.

Do you use the hooks inside a project that uses {renv} or not?

Yes, indeed.

Not sure if i hallucinated you asking, but yes, using Windows.

[lorenzwalthert]

Thanks. And where is your Rscript executable located? I.e. on Unix, I would do which Rscript? Because that's the one we need on the path, as per the docs.

[lorenzwalthert]

Not sure if i hallucinated you asking, but yes, using Windows.

I did but edited after I realised your output shows that already. 🙃

[wtimmerman-fitp]

Bingo!

parsable-R...............................................................Passed
no-browser-statement.....................................................Failed
- hook id: no-browser-statement
- exit code: 1

During startup - Warning message:
Setting LC_CTYPE=C.UTF-8 failed 
Error: File `myfile.R` contains a `browser()` statement.
Execution halted

Looks like it was an R path issue.
On where RScript exec is: For this project i'm using R 4.2.2, but have others. It previously wasn't on my PATH (and indeed which Rscript in bash didn't return anything), but added as C:\Program Files\R\R-4.2.2. This is the same as what i have for R_HOME in Sys.getenv("R_HOME"), which returns the shortened C:/PROGRA~1/R/R-4.2.2 . I hadn't realized that the pre-commit.com docs mention that R needed to be on the path, i had been relying on https://lorenzwalthert.github.io/precommit/articles/precommit.html .

I still get some other errors when running the hooks, but I think I see what the issue is on those. I think I might suggest some updates to the R documentation in a few ways for bumbling people like me, will file as a separate issue or pull request later this week.

Not sure if this solves @EzicJ 's original issue, but I think I'm in okay shape now. Thank you again for the quick response and help, really looking forward to using this.

[lorenzwalthert]

Cool. Yes, if you could help improving the docs with your experience, I'd be all ears for it. Looking forward to a PR and thanks for the persistence.

[lorenzwalthert]

@wtimmerman-fitp if you are still interested in help updating the docs, I'd really appreciate it.

[wtimmerman-fitp]

Ah, thank you for the nudge and reminder. The perils of people on the internet offering help. I actually do have time for this this week, will keep you updated.

[lorenzwalthert]

The quoting issue about “never not found”should be fixed with the latest pre-commit release. Can you install that release, pre-commit clean and pre-commit install again and see if it works now? Rscript needs to be on the PATH in any case.

[lorenzwalthert]

I close this since it has become quite convoluted. Please open a new issue if you still find a problem. @wtimmerman-fitp last reminder for your docs improvement suggestion regarding 'R has to be on the path'.