Xz format considered inadequate for long-term archiving (2016)
Report a link problem
Submission timeline
2007–2026One slot for every year since HN launched. Height is that year's peak points; orange marks a 100+ point or 50+ comment breakout. Select a bar to open its strongest thread.
First comments on top threads
HN comment orderWe're talking about long-term archiving here. That means centuries. My brother the archaelogical archivist of ancient (~2000 years BCE) mesopotamian artifacts has a lot to say about archival formats. His raw material is mostly fired clay tablets. Those archives keep working, partially, even if broken. That's good, because many of them are in fact broken when found. But their ordering and other metadata about where they were found is written in archaeologists' notebooks, and many of those notebooks are now…
This article again? In my opinion, this article is biased. The subtext here is that the author is claiming that his "lzip" format is superior. But xz was not chosen "blindly" as the article claims. To me, most of the claims are arguable. To say 3 levels of headers is "unsafe complexity"... I don't agree. Indirection is fundamental to design. To say padding is "useless"... I don't understand why padding and byte-alignment that is given so much vitriol. Look at…
I think none of these issues really matter. Sure, it isn't perfect. But all of those issues are liveable-with. Most of them involve handling corrupt data - but any serious archive will hash the whole file as part of the cataloging process. The commonly-used technology is by far the best choice for long term archiving, because something that has a billion users will go obsolete/unreadable a long time after a fancy compressor written by a phd student in your lab…
There's a lot here about potential ways that data corruption could make the archive difficult or impossible to recover (?), but no real quantification of risk. How long is "long term"? How often is such corruption expected to occur? In practical terms, that is, why should one care about these issues, or consider them to outweigh other benefits?
The first top-level comment from each of the four biggest threads, in HN’s own order. Excerpts are shortened; open a comment for full context.
- Breakout years
- 3
- Total points
- 627
- Total comments
- 339
100+ points or 50+ comments
reference only — not used in Hall rules or ranking
reference only — not used in Hall rules or ranking
Every submission
| Date | Title as submitted | By | Points | Comments |
|---|---|---|---|---|
| 2018-04-20 | Xz format inadequate for long-term archiving (2017)First breakout | pandalicious | 235 | 135 |
| 2022-07-24 | Xz format considered inadequate for long-term archiving (2016)Best thread | goranmoomin | 248 | 157 |
| 2024-03-29 | Xz format inadequate for long-term archiving (2016) | MaximilianEmel | 17 | 9 |
| 2024-03-30 | Xz format inadequate for long-term archiving (2016) | oriettaxx | 101 | 25 |
| 2024-11-06 | Xz format inadequate for long-term archivingHall induction | fanf2 | 4 | 1 |
| 2025-08-26 | Xz format inadequate for general useLatest 20+ point return | Bogdanp | 22 | 12 |