Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Silent REGRESSION starting 1.8.10: backing up either files or database #355

Open
raph-topo opened this issue Sep 6, 2024 · 4 comments
Open

Silent REGRESSION starting 1.8.10: backing up either files or database #355

raph-topo opened this issue Sep 6, 2024 · 4 comments

Comments

@raph-topo
Copy link

What I'm trying to do and why

docker-borgmatic 1.8.10 stopped backing up files while no configuration file was changed (i.e. the container update from 1.8.9 to 1.8.10 alone has this effect).

This only concerns Borgmatic configurations which do include mysql_databases: or sqlite_databases: (and potentially all/other database hooks, but not tested).

When commenting that block out, files do show up with --list & do back up. Using --dry-dun makes files show up as well.

It was narrowed down in https://projects.torsion.org/borgmatic-collective/borgmatic/issues/903 to not concern upstream Borgmatic but this container definition.

Steps to reproduce

# docker-compose.yml
  borgmatic:
    image: b3vis/borgmatic
    restart: always
    volumes:
      # Sources
      - nextcloud-data:/mnt/source/nextcloud-data:ro
      - nextcloud-config:/mnt/source/nextcloud-config:ro
      - nextcloud-themes:/mnt/source/nextcloud-themes:ro
      - nextcloud-custom_apps:/mnt/source/nextcloud-custom_apps:ro
      # Borgmatic
      - ./borgmatic.d:/etc/borgmatic.d:ro
      - borg-cache:/root/.cache/borg
      # SSH
      - $BORGMATIC_SSH_KEY:/root/.ssh/id_ed25519:ro
    environment:
      TZ: $TZ
    networks:
      - default
# config.yml

repositories:
  - path: ssh://[email protected]/./repo
    label: borgbase

working_directory: /mnt/source
source_directories:
  - .

mysql_databases:
  - name: nextcloud
    hostname: mariadb
    username: root
    password: wwwwwwwww
    options: --skip-comments

archive_name_format: yyyyyyyy@{now}
encryption_passphrase: wwwwwwwww
compression: lz4
match_archives: yyyyyyyy@*

retries: 2
retry_wait: 5
ssh_command: ssh -oStrictHostKeyChecking=accept-new

keep_within: 2m # 2 months
keep_monthly: 24
keep_yearly: 10

checks:
  - name: repository
  - name: archives
check_last: 1

healthchecks:
  ping_url: https://vvvvvvvvvvvvvvv/ping/zzzzzzzzzzzzzzzzzzzzz/backup-ci-yyyyyyyy
  ping_body_limit: 0

before_actions:
  - echo "`date` - Starting Action"
after_actions:
  - echo "`date` - Finished Action"
before_backup:
  - echo "`date` - Starting Backup"
after_backup:
  - echo "`date` - Finished Backup"
before_prune:
  - echo "`date` - Starting Prune"
after_prune:
  - echo "`date` - Finished Prune"
before_compact:
  - echo "`date` - Starting Compact"
after_compact:
  - echo "`date` - Finished Compact"
before_check:
  - echo "`date` - Starting Check"
after_check:
  - echo "`date` - Finished Check"
before_extract:
  - echo "`date` - Starting Extract"
after_extract:
  - echo "`date` - Finished Extract"

on_error:
  - echo "`date` - Borg Error"

Actual behavior

Variant "database"

# borgmatic --version

1.8.13
# borgmatic create --list --verbosity 2

/etc/borgmatic.d/config.yml: No commands to run for pre-everything hook
BORG_PASSPHRASE=*** BORG_RSH=*** BORG_EXIT_CODES=*** borg --version --debug --show-rc
/etc/borgmatic.d/config.yml: Borg 1.4.0
/etc/borgmatic.d/config.yml: Calling healthchecks hook function initialize_monitor
/etc/borgmatic.d/config.yml: Calling healthchecks hook function ping_monitor
/etc/borgmatic.d/config.yml: Pinging Healthchecks start
/etc/borgmatic.d/config.yml: Using Healthchecks ping URL https://vvvvvvvvvvvvvvv/ping/zzzzzzzzzzzzzzzzzzzzz/backup-ci-yyyyyyyy/start
borgbase: Running actions for repository
/etc/borgmatic.d/config.yml: Running command for pre-actions hook
echo "`date` - Starting Action"
Mon Aug  5 15:56:37 CEST 2024 - Starting Action
/etc/borgmatic.d/config.yml: Running command for pre-backup hook
echo "`date` - Starting Backup"
Mon Aug  5 15:56:37 CEST 2024 - Starting Backup
borgbase: Creating archive
ssh://[email protected]/./repo: Calling mariadb_databases hook function remove_data_source_dumps
ssh://[email protected]/./repo: Removing MariaDB data source dumps
ssh://[email protected]/./repo: Calling mysql_databases hook function remove_data_source_dumps
ssh://[email protected]/./repo: Removing MySQL data source dumps
ssh://[email protected]/./repo: Calling mongodb_databases hook function remove_data_source_dumps
ssh://[email protected]/./repo: Removing MongoDB data source dumps
ssh://[email protected]/./repo: Calling postgresql_databases hook function remove_data_source_dumps
ssh://[email protected]/./repo: Removing PostgreSQL data source dumps
ssh://[email protected]/./repo: Calling sqlite_databases hook function remove_data_source_dumps
ssh://[email protected]/./repo: Removing SQLite data source dumps
ssh://[email protected]/./repo: Calling mysql_databases hook function dump_data_sources
ssh://[email protected]/./repo: Dumping MySQL databases
ssh://[email protected]/./repo: Dumping MySQL database "nextcloud" to /root/.borgmatic/mysql_databases/mariadb/nextcloud
MYSQL_PWD=*** mysqldump --skip-comments --add-drop-database --host mariadb --protocol tcp --user root --databases nextcloud --result-file /root/.borgmatic/mysql_databases/mariadb/nextcloud
ssh://[email protected]/./repo: Ignoring configured "read_special" value of false, as true is needed for database hooks.
ssh://[email protected]/./repo: Collecting special file paths
BORG_PASSPHRASE=*** BORG_RSH=*** BORG_EXIT_CODES=*** borg create --compression lz4 --one-file-system --read-special --list --filter AMEx- ssh://[email protected]/./repo::yyyyyyyy@{now} . /etc/borgmatic.d/config.yml /root/.borgmatic --dry-run --list
BORG_PASSPHRASE=*** BORG_RSH=*** BORG_EXIT_CODES=*** borg create --compression lz4 --one-file-system --read-special --list --filter AMEx- --debug --show-rc ssh://[email protected]/./repo::yyyyyyyy@{now} . /etc/borgmatic.d/config.yml /root/.borgmatic
using builtin fallback logging configuration
33 self tests completed in 0.25 seconds
SSH command line: ['ssh', '-oStrictHostKeyChecking=accept-new', '[email protected]', 'borg', 'serve', '--debug']
Remote: using builtin fallback logging configuration
Remote: borg selftest disabled via BORG_SELFTEST env variable
Remote: using builtin fallback logging configuration
Remote: Initialized logging system for JSON-based protocol
Remote: Resolving repository path b'/./repo'
Remote: Resolved repository path to '/srv/repos/xxxxxxxxx/repo'
Remote: Verified integrity of /srv/repos/xxxxxxxxx/repo/index.51055
TAM-verified manifest
security: read previous location 'ssh://[email protected]/./repo'
security: read manifest timestamp '2024-08-05T13:52:36.934529'
security: determined newest manifest timestamp as 2024-08-05T13:52:36.934529
security: repository checks ok, allowing access
Creating archive at "ssh://[email protected]/./repo::yyyyyyyy@2024-08-05T15:56:37"
Verified integrity of /root/.cache/borg/c156080772560547047266a0d0168781f1757c455c86346e6a0798d7b8df0e51/chunks
Reading files cache ...
Verified integrity of /root/.cache/borg/c156080772560547047266a0d0168781f1757c455c86346e6a0798d7b8df0e51/files
security: read previous location 'ssh://[email protected]/./repo'
security: read manifest timestamp '2024-08-05T13:52:36.934529'
security: determined newest manifest timestamp as 2024-08-05T13:52:36.934529
security: repository checks ok, allowing access
Processing files ...
A /root/.borgmatic/bootstrap/manifest.json
A /root/.borgmatic/mysql_databases/mariadb/nextcloud
Remote: Cleaned up 0 uncommitted segment files (== everything after segment 51055).
Remote: Verified integrity of /srv/repos/xxxxxxxxx/repo/hints.51055
Remote: Storage quota: 375.92 GB out of 2.87 TB used.
Remote: check_free_space: required bytes 211581658, free bytes 15142892871680
Remote: Storage quota: 375.93 GB out of 2.87 TB used.
security: saving state for c156080772560547047266a0d0168781f1757c455c86346e6a0798d7b8df0e51 to /root/.config/borg/security/c156080772560547047266a0d0168781f1757c455c86346e6a0798d7b8df0e51
security: current location   ssh://[email protected]/./repo
security: key type           5
security: manifest timestamp 2024-08-05T13:57:08.582466
RemoteRepository: 2.58 MB bytes sent, 215.10 kB bytes received, 14 messages sent
terminating with success status, rc 0
/etc/borgmatic.d/config.yml: Calling mariadb_databases hook function remove_data_source_dumps
/etc/borgmatic.d/config.yml: Removing MariaDB data source dumps
/etc/borgmatic.d/config.yml: Calling mysql_databases hook function remove_data_source_dumps
/etc/borgmatic.d/config.yml: Removing MySQL data source dumps
/etc/borgmatic.d/config.yml: Calling mongodb_databases hook function remove_data_source_dumps
/etc/borgmatic.d/config.yml: Removing MongoDB data source dumps
/etc/borgmatic.d/config.yml: Calling postgresql_databases hook function remove_data_source_dumps
/etc/borgmatic.d/config.yml: Removing PostgreSQL data source dumps
/etc/borgmatic.d/config.yml: Calling sqlite_databases hook function remove_data_source_dumps
/etc/borgmatic.d/config.yml: Removing SQLite data source dumps
/etc/borgmatic.d/config.yml: Running command for post-backup hook
echo "`date` - Finished Backup"
Mon Aug  5 15:57:09 CEST 2024 - Finished Backup
/etc/borgmatic.d/config.yml: Running command for post-actions hook
echo "`date` - Finished Action"
Mon Aug  5 15:57:09 CEST 2024 - Finished Action
/etc/borgmatic.d/config.yml: Calling healthchecks hook function ping_monitor
/etc/borgmatic.d/config.yml: Pinging Healthchecks log
/etc/borgmatic.d/config.yml: Using Healthchecks ping URL https://vvvvvvvvvvvvvvv/ping/zzzzzzzzzzzzzzzzzzzzz/backup-ci-yyyyyyyy/log
/etc/borgmatic.d/config.yml: Calling healthchecks hook function ping_monitor
/etc/borgmatic.d/config.yml: Pinging Healthchecks finish
/etc/borgmatic.d/config.yml: Using Healthchecks ping URL https://vvvvvvvvvvvvvvv/ping/zzzzzzzzzzzzzzzzzzzzz/backup-ci-yyyyyyyy
/etc/borgmatic.d/config.yml: Calling healthchecks hook function destroy_monitor
/etc/borgmatic.d/config.yml: No commands to run for post-everything hook

summary:
/etc/borgmatic.d/config.yml: Loading configuration file
/etc/borgmatic.d/config.yml: Successfully ran configuration file
# borgmatic list --archive latest  --verbosity 2

BORG_PASSPHRASE=*** BORG_RSH=*** BORG_EXIT_CODES=*** borg --version --debug --show-rc
/etc/borgmatic.d/config.yml: Borg 1.4.0
borgbase: Running actions for repository
/etc/borgmatic.d/config.yml: Running command for pre-actions hook
echo "`date` - Starting Action"
Mon Aug  5 15:59:02 CEST 2024 - Starting Action
BORG_PASSPHRASE=*** BORG_RSH=*** BORG_EXIT_CODES=*** borg list --last 1 --short ssh://[email protected]/./repo
ssh://[email protected]/./repo: Latest archive is yyyyyyyy@2024-08-05T15:56:37
ssh://[email protected]/./repo: Listing archive yyyyyyyy@2024-08-05T15:56:37
BORG_PASSPHRASE=*** BORG_RSH=*** BORG_EXIT_CODES=*** borg list --debug --show-rc ssh://[email protected]/./repo::yyyyyyyy@2024-08-05T15:56:37
using builtin fallback logging configuration
33 self tests completed in 0.23 seconds
SSH command line: ['ssh', '-oStrictHostKeyChecking=accept-new', '[email protected]', 'borg', 'serve', '--debug']
Remote: using builtin fallback logging configuration
Remote: borg selftest disabled via BORG_SELFTEST env variable
Remote: using builtin fallback logging configuration
Remote: Initialized logging system for JSON-based protocol
Remote: Resolving repository path b'/./repo'
Remote: Resolved repository path to '/srv/repos/xxxxxxxxx/repo'
Remote: Verified integrity of /srv/repos/xxxxxxxxx/repo/index.51059
TAM-verified manifest
security: read previous location 'ssh://[email protected]/./repo'
security: read manifest timestamp '2024-08-05T13:57:08.582466'
security: determined newest manifest timestamp as 2024-08-05T13:57:08.582466
security: repository checks ok, allowing access
Archive authentication DISABLED.
TAM-verified archive
drwxr-xr-x root   root          0 Mon, 2024-08-05 12:29:59 .
drwxr-xr-x xfs    xfs           0 Sun, 2023-03-19 23:32:48 nextcloud-config
drwxr-xr-x xfs    xfs           0 Mon, 2024-08-05 13:59:22 nextcloud-custom_apps
drwxrwx--- xfs    root          0 Fri, 2024-07-26 15:41:11 nextcloud-data
drwxr-xr-x xfs    xfs           0 Sun, 2023-02-12 18:31:34 nextcloud-themes
-rw-r--r--   1000 video      1445 Mon, 2024-08-05 15:52:02 etc/borgmatic.d/config.yml
drwxr-xr-x root   root          0 Mon, 2024-08-05 15:56:37 root/.borgmatic
drwxr-xr-x root   root          0 Wed, 2023-07-05 10:10:01 root/.borgmatic/bootstrap
-rw-r--r-- root   root         80 Mon, 2024-08-05 15:56:37 root/.borgmatic/bootstrap/manifest.json
drwxr-xr-x root   root          0 Mon, 2024-08-05 15:56:37 root/.borgmatic/mysql_databases
drwx------ root   root          0 Mon, 2024-08-05 15:56:37 root/.borgmatic/mysql_databases/mariadb
-rw------- root   root   736421728 Mon, 2024-08-05 15:56:37 root/.borgmatic/mysql_databases/mariadb/nextcloud
RemoteRepository: 318 B bytes sent, 227.77 kB bytes received, 7 messages sent
terminating with success status, rc 0
/etc/borgmatic.d/config.yml: Running command for post-actions hook
echo "`date` - Finished Action"
Mon Aug  5 15:59:05 CEST 2024 - Finished Action

summary:
/etc/borgmatic.d/config.yml: Loading configuration file
/etc/borgmatic.d/config.yml: Successfully ran configuration file

Notice that these folders are present but empty:

drwxr-xr-x xfs    xfs           0 Sun, 2023-03-19 23:32:48 nextcloud-config
drwxr-xr-x xfs    xfs           0 Mon, 2024-08-05 13:59:22 nextcloud-custom_apps
drwxrwx--- xfs    root          0 Fri, 2024-07-26 15:41:11 nextcloud-data
drwxr-xr-x xfs    xfs           0 Sun, 2023-02-12 18:31:34 nextcloud-themes

Variant "files"

Run again, after having commented out the mysql_databases: block in config.yml.

# borgmatic --version

1.8.13
# borgmatic create --list --verbosity 2

/etc/borgmatic.d/config.yml: No commands to run for pre-everything hook
BORG_PASSPHRASE=*** BORG_RSH=*** BORG_EXIT_CODES=*** borg --version --debug --show-rc
/etc/borgmatic.d/config.yml: Borg 1.4.0
/etc/borgmatic.d/config.yml: Calling healthchecks hook function initialize_monitor
/etc/borgmatic.d/config.yml: Calling healthchecks hook function ping_monitor
/etc/borgmatic.d/config.yml: Pinging Healthchecks start
/etc/borgmatic.d/config.yml: Using Healthchecks ping URL https://vvvvvvvvvvvvvvv/ping/zzzzzzzzzzzzzzzzzzzzz/backup-ci-yyyyyyyy/start
borgbase: Running actions for repository
/etc/borgmatic.d/config.yml: Running command for pre-actions hook
echo "`date` - Starting Action"
Mon Aug  5 16:07:58 CEST 2024 - Starting Action
/etc/borgmatic.d/config.yml: Running command for pre-backup hook
echo "`date` - Starting Backup"
Mon Aug  5 16:07:58 CEST 2024 - Starting Backup
borgbase: Creating archive
ssh://[email protected]/./repo: Calling mariadb_databases hook function remove_data_source_dumps
ssh://[email protected]/./repo: Removing MariaDB data source dumps
ssh://[email protected]/./repo: Calling mysql_databases hook function remove_data_source_dumps
ssh://[email protected]/./repo: Removing MySQL data source dumps
ssh://[email protected]/./repo: Calling mongodb_databases hook function remove_data_source_dumps
ssh://[email protected]/./repo: Removing MongoDB data source dumps
ssh://[email protected]/./repo: Calling postgresql_databases hook function remove_data_source_dumps
ssh://[email protected]/./repo: Removing PostgreSQL data source dumps
ssh://[email protected]/./repo: Calling sqlite_databases hook function remove_data_source_dumps
ssh://[email protected]/./repo: Removing SQLite data source dumps
BORG_PASSPHRASE=*** BORG_RSH=*** BORG_EXIT_CODES=*** borg create --compression lz4 --list --filter AMEx- --debug --show-rc ssh://[email protected]/./repo::yyyyyyyy@{now} . /etc/borgmatic.d/config.yml /root/.borgmatic
using builtin fallback logging configuration
33 self tests completed in 0.26 seconds
SSH command line: ['ssh', '-oStrictHostKeyChecking=accept-new', '[email protected]', 'borg', 'serve', '--debug']
Remote: using builtin fallback logging configuration
Remote: borg selftest disabled via BORG_SELFTEST env variable
Remote: using builtin fallback logging configuration
Remote: Initialized logging system for JSON-based protocol
Remote: Resolving repository path b'/./repo'
Remote: Resolved repository path to '/srv/repos/xxxxxxxxx/repo'
Remote: Verified integrity of /srv/repos/xxxxxxxxx/repo/index.51059
TAM-verified manifest
security: read previous location 'ssh://[email protected]/./repo'
security: read manifest timestamp '2024-08-05T13:57:08.582466'
security: determined newest manifest timestamp as 2024-08-05T13:57:08.582466
security: repository checks ok, allowing access
Creating archive at "ssh://[email protected]/./repo::yyyyyyyy@2024-08-05T16:07:59"
Verified integrity of /root/.cache/borg/c156080772560547047266a0d0168781f1757c455c86346e6a0798d7b8df0e51/chunks
Reading files cache ...
Verified integrity of /root/.cache/borg/c156080772560547047266a0d0168781f1757c455c86346e6a0798d7b8df0e51/files
security: read previous location 'ssh://[email protected]/./repo'
security: read manifest timestamp '2024-08-05T13:57:08.582466'
security: determined newest manifest timestamp as 2024-08-05T13:57:08.582466
security: repository checks ok, allowing access
Processing files ...
A nextcloud-config/.htaccess

LOTS OF FILES

M /etc/borgmatic.d/config.yml
A /root/.borgmatic/bootstrap/manifest.json
Remote: check_free_space: required bytes 211581858, free bytes 15111968206848
Remote: Storage quota: 378.49 GB out of 2.87 TB used.
security: saving state for c156080772560547047266a0d0168781f1757c455c86346e6a0798d7b8df0e51 to /root/.config/borg/security/c156080772560547047266a0d0168781f1757c455c86346e6a0798d7b8df0e51
security: current location   ssh://[email protected]/./repo
security: key type           5
security: manifest timestamp 2024-08-05T14:40:41.081192
RemoteRepository: 2.57 GB bytes sent, 284.14 kB bytes received, 7599 messages sent
terminating with success status, rc 0
/etc/borgmatic.d/config.yml: Calling mariadb_databases hook function remove_data_source_dumps
/etc/borgmatic.d/config.yml: Removing MariaDB data source dumps
/etc/borgmatic.d/config.yml: Calling mysql_databases hook function remove_data_source_dumps
/etc/borgmatic.d/config.yml: Removing MySQL data source dumps
/etc/borgmatic.d/config.yml: Calling mongodb_databases hook function remove_data_source_dumps
/etc/borgmatic.d/config.yml: Removing MongoDB data source dumps
/etc/borgmatic.d/config.yml: Calling postgresql_databases hook function remove_data_source_dumps
/etc/borgmatic.d/config.yml: Removing PostgreSQL data source dumps
/etc/borgmatic.d/config.yml: Calling sqlite_databases hook function remove_data_source_dumps
/etc/borgmatic.d/config.yml: Removing SQLite data source dumps
/etc/borgmatic.d/config.yml: Running command for post-backup hook
echo "`date` - Finished Backup"
Mon Aug  5 16:40:45 CEST 2024 - Finished Backup
/etc/borgmatic.d/config.yml: Running command for post-actions hook
echo "`date` - Finished Action"
Mon Aug  5 16:40:45 CEST 2024 - Finished Action
/etc/borgmatic.d/config.yml: Calling healthchecks hook function ping_monitor
/etc/borgmatic.d/config.yml: Pinging Healthchecks log
/etc/borgmatic.d/config.yml: Using Healthchecks ping URL https://vvvvvvvvvvvvvvv/ping/zzzzzzzzzzzzzzzzzzzzz/backup-ci-yyyyyyyy/log
/etc/borgmatic.d/config.yml: Healthchecks error: 400 Client Error: Bad Request for url: https://vvvvvvvvvvvvvvv/ping/zzzzzzzzzzzzzzzzzzzzz/backup-ci-yyyyyyyy/log
/etc/borgmatic.d/config.yml: Calling healthchecks hook function ping_monitor
/etc/borgmatic.d/config.yml: Pinging Healthchecks finish
/etc/borgmatic.d/config.yml: Using Healthchecks ping URL https://vvvvvvvvvvvvvvv/ping/zzzzzzzzzzzzzzzzzzzzz/backup-ci-yyyyyyyy
/etc/borgmatic.d/config.yml: Healthchecks error: 400 Client Error: Bad Request for url: https://vvvvvvvvvvvvvvv/ping/zzzzzzzzzzzzzzzzzzzzz/backup-ci-yyyyyyyy
/etc/borgmatic.d/config.yml: Calling healthchecks hook function destroy_monitor
/etc/borgmatic.d/config.yml: No commands to run for post-everything hook

summary:
/etc/borgmatic.d/config.yml: Loading configuration file
/etc/borgmatic.d/config.yml: Successfully ran configuration file
# borgmatic list --archive latest --verbosity 2

BORG_PASSPHRASE=*** BORG_RSH=*** BORG_EXIT_CODES=*** borg --version --debug --show-rc
/etc/borgmatic.d/config.yml: Borg 1.4.0
borgbase: Running actions for repository
/etc/borgmatic.d/config.yml: Running command for pre-actions hook
echo "`date` - Starting Action"
Mon Aug  5 16:58:17 CEST 2024 - Starting Action
BORG_PASSPHRASE=*** BORG_RSH=*** BORG_EXIT_CODES=*** borg list --last 1 --short ssh://[email protected]/./repo
ssh://[email protected]/./repo: Latest archive is yyyyyyyy@2024-08-05T16:09:44
ssh://[email protected]/./repo: Listing archive yyyyyyyy@2024-08-05T16:09:44
BORG_PASSPHRASE=*** BORG_RSH=*** BORG_EXIT_CODES=*** borg list --debug --show-rc ssh://[email protected]/./repo::yyyyyyyy@2024-08-05T16:09:44
using builtin fallback logging configuration
33 self tests completed in 0.22 seconds
SSH command line: ['ssh', '-oStrictHostKeyChecking=accept-new', '[email protected]', 'borg', 'serve', '--debug']
Remote: using builtin fallback logging configuration
Remote: borg selftest disabled via BORG_SELFTEST env variable
Remote: using builtin fallback logging configuration
Remote: Initialized logging system for JSON-based protocol
Remote: Resolving repository path b'/./repo'
Remote: Resolved repository path to '/srv/repos/xxxxxxxxx/repo'
Remote: Verified integrity of /srv/repos/xxxxxxxxx/repo/index.51071
TAM-verified manifest
security: read previous location 'ssh://[email protected]/./repo'
security: read manifest timestamp '2024-08-05T14:40:41.081192'
security: determined newest manifest timestamp as 2024-08-05T14:40:41.081192
security: repository checks ok, allowing access
Archive authentication DISABLED.
TAM-verified archive
drwxr-xr-x root   root          0 Mon, 2024-08-05 12:29:59 .
drwxr-xr-x xfs    xfs           0 Sun, 2023-03-19 23:32:48 nextcloud-config
-rw-r--r-- xfs    xfs         495 Sun, 2023-02-12 18:31:34 nextcloud-config/.htaccess

LOTS OF FILES

-rw-r--r--   1000 video      1457 Mon, 2024-08-05 16:07:05 etc/borgmatic.d/config.yml
drwxr-xr-x root   root          0 Mon, 2024-08-05 15:57:08 root/.borgmatic
drwxr-xr-x root   root          0 Wed, 2023-07-05 10:10:01 root/.borgmatic/bootstrap
-rw-r--r-- root   root         80 Mon, 2024-08-05 16:10:21 root/.borgmatic/bootstrap/manifest.json
RemoteRepository: 23.96 kB bytes sent, 20.56 MB bytes received, 460 messages sent
terminating with success status, rc 0
/etc/borgmatic.d/config.yml: Running command for post-actions hook
echo "`date` - Finished Action"
Mon Aug  5 17:00:30 CEST 2024 - Finished Action

summary:
/etc/borgmatic.d/config.yml: Loading configuration file
/etc/borgmatic.d/config.yml: Successfully ran configuration file

Notice that files are being backed up, but not database:

drwxr-xr-x xfs    xfs           0 Sun, 2023-03-19 23:32:48 nextcloud-config
-rw-r--r-- xfs    xfs         495 Sun, 2023-02-12 18:31:34 nextcloud-config/.htaccess
-rw-r--r--   1000 video      1457 Mon, 2024-08-05 16:07:05 etc/borgmatic.d/config.yml
drwxr-xr-x root   root          0 Mon, 2024-08-05 15:57:08 root/.borgmatic
drwxr-xr-x root   root          0 Wed, 2023-07-05 10:10:01 root/.borgmatic/bootstrap
-rw-r--r-- root   root         80 Mon, 2024-08-05 16:10:21 root/.borgmatic/bootstrap/manifest.json

docker-borgmatic version

  • 1.8.5: works
  • 1.8.6: manifest for b3vis/borgmatic:1.8.6 not found: manifest unknown: manifest unknown (not on Docker Hub anymore)
  • 1.8.7: works
  • 1.8.8: works
  • 1.8.9: works
  • 1.8.10: does not work
  • 1.8.11: does not work
  • 1.8.12: does not work
  • 1.8.13: does not work

docker-borgmatic installation method

    image: b3vis/borgmatic:version

identical to

    image: ghcr.io/borgmatic-collective/borgmatic:version

Borg version

borgbackup==1.2.7 until b3vis/borgmatic:1.8.9
borgbackup==1.2.8 from b3vis/borgmatic:1.8.10
@witten
Copy link
Collaborator

witten commented Oct 1, 2024

I still don't know what caused this apparent regression, but I may have a fix anyway. The next version of borgmatic will no longer auto-enable the "one_file_system" option when databases are configured. I think this may prevent the behavior you're seeing and ensure that all the expected files get included in the backup. The downside is you'll need to wait for that borgmatic release and the corresponding container release.

@mrclschstr
Copy link

mrclschstr commented Nov 7, 2024 •
edited
Loading

Related #317

EDIT: If you dig through the comments, I already found the root cause in PR #216. This is a "special issue" of the borgmatic container version and has nothing to do with borgmatic version >=1.8.10 itself.

@witten
Copy link
Collaborator

witten commented Nov 7, 2024

Note that this unfortunate interaction may go away with borgmatic 1.9.0 because (from the release notes):

  • BREAKING: When databases are configured, don't auto-enable the "one_file_system" option,
    as existing auto-excludes of special files should be sufficient to prevent Borg from hanging on
    them. But if this change causes problems for you, you can always enable "one_file_system"
    explicitly.

See https://projects.torsion.org/borgmatic-collective/borgmatic/issues/918 for more info.

@modem7
Copy link
Member

modem7 commented Nov 25, 2024

@raph-topo - Can we close this issue seeing as @witten has a solution?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@witten @modem7 @mrclschstr @raph-topo