By sampling 4,200 random URLs spanning a 14 year period, Maciej Cegłowski, the creator of bookmarking website Pinboard.in, decided to gather statistics on the extent of link rot and how it progressed across time. Interested in finding out if there is some sort of ‘half life of links’, he found instead that it is a fairly linear, fast deterioration:
Links appear to die at a steady rate (they don’t have a half life), and you can expect to lose about a quarter of them every seven years.
And even that is an optimistic result, says Maciej, as not all dead links were able to be discovered programmatically. There are also several unanswered questions:
- How many of these dead URLs are findable on archive.org?
- What is the attrition rate for shortened links?
- Is there a simple programmatic way to detect parked domains?
- Given just a URL, can we make any intelligent guesses about its vulnerability to link rot?
Interestingly, link rot is what inspired the creation of Pinboard.in (it features page archiving funcitonality). This is similar to why I started Lone Gunman: I was losing track of interesting links and articles, and wanted a way to easily find them again as well as help me build connections between disparate articles and topics.