Understanding borg-diff output

[mbwgh]

I wanted to try out borg-diff in order to see if maybe there are some directories I keep backing up unnecessarily (like caches, build artifacts, etc.) and which I should blacklist if their sizes are large.

Invoking borg diff my-repo::penultimate-archive last-archive yielded some output which contains output including the following:

      0 B       0 B home/mbw/Mail/sent/cur/1692947982.R287735136380536416.legion:2,S
      0 B       0 B home/mbw/Mail/sent/cur/1692971008.R6427850541844380043.legion:2,S
[ctime: Fri, 2023-09-15 10:13:19 -> Sun, 2023-10-08 14:10:06] [mtime: Fri, 2023-09-15 10:13:19 -> Sun, 2023-10-08 14:10:06] home/mbw/Mail/sent/tmp
[ctime: Sat, 2023-09-30 09:30:53 -> Wed, 2023-10-11 09:56:18] [mtime: Sat, 2023-09-30 09:30:53 -> Wed, 2023-10-11 09:56:18] home/mbw/Mail/.notmuch/xapian
   +158 B    -160 B [ctime: Sat, 2023-09-30 09:30:53 -> Wed, 2023-10-11 09:56:18] [mtime: Sat, 2023-09-30 09:30:53 -> Wed, 2023-10-11 09:56:18] home/mbw/Mail/.notmuch/xapian/iamglass
      0 B       0 B [ctime: Sat, 2023-09-30 10:53:53 -> Wed, 2023-10-11 10:11:22] [mtime: Sat, 2023-09-30 10:53:53 -> Wed, 2023-10-11 10:11:22] home/mbw/Mail/.notmuch/xapian/flintlock
+461.8 MB -461.8 MB [ctime: Sat, 2023-09-30 09:30:53 -> Wed, 2023-10-11 09:56:18] [mtime: Sat, 2023-09-30 09:30:53 -> Wed, 2023-10-11 09:56:18] home/mbw/Mail/.notmuch/xapian/postlist.glass
 +24.6 kB  -24.6 kB [ctime: Fri, 2023-09-01 08:54:27 -> Sun, 2023-10-01 15:17:05] [mtime: Fri, 2023-09-01 08:54:27 -> Sun, 2023-10-01 15:17:05] home/mbw/Mail/.notmuch/xapian/docdata.glass
+285.7 MB -285.7 MB [ctime: Sat, 2023-09-30 09:30:53 -> Wed, 2023-10-11 09:56:18] [mtime: Sat, 2023-09-30 09:30:53 -> Wed, 2023-10-11 09:56:18] home/mbw/Mail/.notmuch/xapian/termlist.glass
  +1.6 GB   -1.6 GB [ctime: Sat, 2023-09-30 09:24:52 -> Wed, 2023-10-11 09:56:18] [mtime: Sat, 2023-09-30 09:24:52 -> Wed, 2023-10-11 09:56:18] home/mbw/Mail/.notmuch/xapian/position.glass
[ctime: Sat, 2023-09-30 10:59:53 -> Wed, 2023-10-11 10:11:21] [mtime: Sat, 2023-09-30 10:59:53 -> Wed, 2023-10-11 10:11:21] home/mbw/.getmail
[ctime: Tue, 2022-11-08 18:21:09 -> Thu, 2023-10-05 14:39:06] [mtime: Tue, 2022-11-08 18:21:09 -> Thu, 2023-10-05 14:39:06] home/mbw/.getmail/config
  +3.4 MB   -3.3 MB [ctime: Sat, 2023-09-30 09:24:50 -> Wed, 2023-10-11 09:56:12] [mtime: Sat, 2023-09-30 09:24:50 -> Wed, 2023-10-11 09:56:12] home/mbw/.getmail/log

What do these +/- numbers mean? Does this mean for example that the position.glass file takes up 1.6 GB additional space in the new backup (I hope not?).

Is this output explained somewhere or should I call borg-diff with different arguments to get better answer my question?

[ThomasWaldmann]

The +/- refers to the overall size of the content chunks of the current file which were added or removed.

[mbwgh]

Sorry I don't understand this.
Does this say anything about how much is contributed to the deduplicated size of the archive?

For instance, does this mean that the position.glass file increases the deduplicated backup size by 1.6GB? (It shows up as 1.5GB via ls and would need to have changed significantly). Or did it probably stay the same? Since most files seem to add/remove equal-sized content chunks.

[ThomasWaldmann]

For simplicity, imagine using a fixed chunk size of 1MB and a file consisting of 2 chunks, 1MB each.
First chunk is all 0x11, second chunk is all 0x22.

You back up that file into archive A.

Then you change that file: you would modify the second MB of the file to be 1MB of 0x33.

You back up that file into archive B.

borg diff A B would tell you that there is -1MB (because second chunk, 1MB of 0x22 removed) and +1MB (because there is a new different second chunk of 1MB 0x33).

[mbwgh]

(Belated) Thanks for your patience and the thorough answer :)

That sounds indeed like these .glass files bloat the incremental backup size unduly.
Since they DB indices only, I now backup the ~4MB DB dump created with notmuch dump instead.