I recently started using uncompressed, encrypted 7z archives for backups. I chose 7z because it has easy to use desktop apps for Windows and Linux, in case I need to recover the data quickly.
I store checksums next to the archive file to detect file corruption. Currently the archive is written to an external HDD and then read again for checksumming. This is necessary because 7z seeks while creating a 7z archive so I can't its output through a pipe. Reading the file again is pretty slow though.
I discovered that 7z has the "t" action to "Test[s] archive files". Running "7z -h" describes it as "Test integrity of archive". Unfortunately that's all help and documentation say about it. I had a look at the 7z format specification, which seems to include places for CRC checksums of some kind. I also had a glance at the source code, but I didn't understand where to look for. I couldn't find the part responsible for the test action.
So my question is:What kind of checksums does 7z store in .7z archives, and are they sufficient to detect file corruption?
I don't count corruption to the archive metadata, as that is obvious when 7z won't open the archive.
2 Answers
I dug a bit deeper, and 7z stores CRC-32 checksums for files in the archive.
The 7z SDK contains a file format description (a copy of which can be found online here). However it is pretty short and doesn't tell what the different parts really do.
I found Py7zr which also documents the 7z file format, in a much more understandable way. The file data is stored in so-called Packed Streams. A CRC-32 checksum is saved for every Packed Stream, which may be checked on extraction.
Additionally there exist CRC-32 checksums for the uncompressed data of every file in the Coders Information header section. When testing an archive with the 7z GUI, it shows the files it checks, so it probably decompresses the files and checks their CRC.
As CRC-32 has a probability of about 2^32 for having an accidental collision when files get corrupted, it should be pretty usable up to a few 100k files in the archive. There also is a Stackoverflow Question for the suitability of CRC-32 to detect file errors.
As I only have a few thousand files by now, the checksums integrated in the 7z file format should be sufficient to detect file corruption.
With 7-Zip you can display the checksum/hash using several methods, but unfortunately you have to store the values yourself and compare them manually.
The 7Zip h (Hash) commanddoes the following:
5calculates CRC32 for all files in current folder and all subfolders
