HN Hall of Fame Weekly email

Xz format considered inadequate for long-term archiving (2016)

www.nongnu.org Essays & writing Essays & articles Software engineering Class of 2024-11 Hall of Fame

Resurfaced independently across 4 calendar years, with breakout response in 3 of them.

submissions
6
submitters
6
observed span
2018–2025
peak thread · 157 comments
248 pts
latest 20+ return · 2025-08-26
22 pts

Submission timeline

2007–2026

One slot for every year since HN launched. Height is that year's peak points; orange marks a 100+ point or 50+ comment breakout. Select a bar to open its strongest thread.

First comments on top threads

HN comment order

We're talking about long-term archiving here. That means centuries. My brother the archaelogical archivist of ancient (~2000 years BCE) mesopotamian artifacts has a lot to say about archival formats. His raw material is mostly fired clay tablets. Those archives keep working, partially, even if broken. That's good, because many of them are in fact broken when found. But their ordering and other metadata about where they were found is written in archaeologists' notebooks, and many of those notebooks are now…

This article again? In my opinion, this article is biased. The subtext here is that the author is claiming that his "lzip" format is superior. But xz was not chosen "blindly" as the article claims. To me, most of the claims are arguable. To say 3 levels of headers is "unsafe complexity"... I don't agree. Indirection is fundamental to design. To say padding is "useless"... I don't understand why padding and byte-alignment that is given so much vitriol. Look at…

I think none of these issues really matter. Sure, it isn't perfect. But all of those issues are liveable-with. Most of them involve handling corrupt data - but any serious archive will hash the whole file as part of the cataloging process. The commonly-used technology is by far the best choice for long term archiving, because something that has a billion users will go obsolete/unreadable a long time after a fancy compressor written by a phd student in your lab…

There's a lot here about potential ways that data corruption could make the archive difficult or impossible to recover (?), but no real quantification of risk. How long is "long term"? How often is such corruption expected to occur? In practical terms, that is, why should one care about these issues, or consider them to outweigh other benefits?

zahlman·4-point thread·

The first top-level comment from each of the four biggest threads, in HN’s own order. Excerpts are shortened; open a comment for full context.

Breakout years
3

100+ points or 50+ comments

Total points
627

reference only — not used in Hall rules or ranking

Total comments
339

reference only — not used in Hall rules or ranking

Every submission

DateTitle as submittedByPointsComments
2018-04-20Xz format inadequate for long-term archiving (2017)First breakoutpandalicious235135
2022-07-24Xz format considered inadequate for long-term archiving (2016)Best threadgoranmoomin248157
2024-03-29Xz format inadequate for long-term archiving (2016)MaximilianEmel179
2024-03-30Xz format inadequate for long-term archiving (2016)oriettaxx10125
2024-11-06Xz format inadequate for long-term archivingHall inductionfanf241
2025-08-26Xz format inadequate for general useLatest 20+ point returnBogdanp2212