restic compression and deduplication

I use restic to backup data on servers. It is a great tool and manages the entire lifecycle of a backup. In restic a backup job produces a snapshot. Files in snapshots are deduplicated and compressed.Back rotation can be setup on a daily, weekly, monthly or yearly basis. If restic compresses and dedups files, does it makes sense to zip backup files beforehand?

I had to answer this question, because I ran into a problem. The main job for restic is to backup Odoo. The Odoo backup produces a zip file containing the database dump and the filestore. As storage backend I am using a restic server. Every server creates a restic repo on the restic server. A few weeks ago I got an alert, one backup repo grew out of size.

When I list the snaphots on the problematic server it looks like this:

...
7c8f089c  2026-05-25 02:11:58  ov-c7f397   odoo,odoo-main  /mnt/data/backup/odoo-main.zip  7.652 GiB
6c53e3c7  2026-05-26 02:11:58  ov-c7f397   odoo,odoo-main  /mnt/data/backup/odoo-main.zip  7.664 GiB

For some reason the snapshots were not deduplicated. So every snapshots required the full size on backup server. With backup rotation this accumulated into something like this:

7 GiB + 7 GiB + … + 7 GiB = 90 GiB

Why did deduplication not work in this case? The deduplication mechansim splits the file blob into 512 KiB to 8 MiB chunks. For each chunk a fingerprint is created. A chunk is only stored once. You can list chunks/blobs with restic list blobs.

If the deduplication does not work, then the chunks have to be different for every backup. To state the obvious, the zip does not have the same structure for diffs.

I decided to create tar files instead. This is what the snapshots look like on the server.

...
5b76821f  2026-05-26 10:56:59  ov-c7f397   odoo,odoo-main  /mnt/data/backup/odoo-main.tar  22.627 GiB
a3ad3e79  2026-05-27 02:06:11  ov-c7f397   odoo,odoo-main  /mnt/data/backup/odoo-main.tar  22.733 GiB

Each backup went from 7 GiB to 22 GiB. However, on the restic server it looks like this:

user@restic-server:/mnt/sdb# du -hs ./ov-c7f397/ | sort -h
16G    ./ov-c7f397/

The compression and deduplication did some work here.