HN Hall of Fame Weekly email

CDC File Transfer

github.com Software & code Software projects Software engineering Candidate
Screenshot of github.com captured 2026-07-20
Page preview · captured 2026-07-20

Resurfaced independently across 2 calendar years, with breakout response in 2 of them.

submissions
3
submitters
3
observed span
2023–2025
peak thread · 219 comments
917 pts
latest 20+ return · 2025-10-01
389 pts

Submission timeline

2007–2026

One slot for every year since HN launched. Height is that year's peak points; orange marks a 100+ point or 50+ comment breakout. Select a bar to open its strongest thread.

First comments on top threads

HN comment order

Content Defined Chunking is one of my favorite algorithms because it has some "magic" similar to HyperLogLogs, Bloom filters, etc... This algorithm is good to explain to people, to get them inspired by computer science. I usually explain the simplest variant with rolling hashes. It is interesting what the result will be (average saving on deduplication) if it is applied globally to a large-scale blob storage, such as Amazon S3 or Google Drive (we need metadata storage about chunks, and…

I’ve also been doing lots of experimenting with Content Defined Chunking since last year (for https://bonanza.build/). One of the things I discovered is that the most commonly used algorithm FastCDC (also used by this project) can be improved significantly by looking ahead. An implementation of that can be found here: https://github.com/buildbarn/go-cdc

Basically, CDC File Transfer is a way for streaming syncing and streaming files from Windows to Linux.

The first top-level comment from each of the four biggest threads, in HN’s own order. Excerpts are shortened; open a comment for full context.

Breakout years
2

100+ points or 50+ comments

Total points
1310

reference only — not used in Hall rules or ranking

Total comments
322

reference only — not used in Hall rules or ranking

Every submission

DateTitle as submittedByPointsComments
2023-01-06Google open sourced CDC File Transfer from the ashes of StadiaPoudlardo41
2023-01-08CDC File TransferFirst breakout · Best threadcglong917219
2025-10-01CDC File TransferLatest 20+ point returnGalaxySnail389102