View source for Link rot

{{Self reference | For link rot on Wikipedia, see [[Wikipedia:Link rot]].}}
{{redirect|Broken link|the ''Star Trek: Deep Space Nine'' episode|Broken Link (Star Trek: Deep Space Nine)}}
[[Image:404 not found.png|alt=Page Not Found|thumb|A rotten link usually leads to an error message]]
{{short description|Phenomenon of URLs tending to cease functioning}}
'''Link rot''' (also called '''link death''', '''link breaking''', or '''reference rot''') is the phenomenon of [[hyperlink]]s tending over time to cease to point to their originally targeted [[computer file|file]], [[web page]], or [[Server (computing)|server]] due to that resource being relocated to a new address or becoming permanently unavailable. A link that no longer points to its target, often called a ''broken'', ''dead'', or ''orphaned'' link, is a specific form of [[dangling pointer]].

The rate of link rot is a subject of study and research due to its significance to the [[internet]]'s ability to preserve information. Estimates of that rate vary dramatically between studies. Information professionals have warned that link rot could make important archival data disappear, potentially impacting the legal system and scholarship.

==Prevalence==
A number of studies have examined the prevalence of link rot within the [[World Wide Web]], in academic literature that uses [[URL]]s to cite web content, and within [[digital library|digital libraries]].

A 2002 study suggested that link rot within digital libraries is considerably slower than on the web, finding that about 3% of the objects were no longer accessible after one year<ref name=Nelson2002>{{cite journal | first1 = Michael L. | last1 = Nelson | first2 = B. Danette | last2 = Allen | year = 2002 | title = Object Persistence and Availability in Digital Libraries | doi = 10.1045/january2002-nelson | journal = D-Lib Magazine | volume = 8 | issue = 1 | url = https://digitalcommons.odu.edu/cgi/viewcontent.cgi?article=1008&context=computerscience_fac_pubs | doi-access = free | access-date = 2019-09-24 | archive-date = 2020-07-19 | archive-url = https://web.archive.org/web/20200719044311/https://digitalcommons.odu.edu/cgi/viewcontent.cgi?article=1008&context=computerscience_fac_pubs | url-status = live }}</ref> (equating to a [[half-life]] of nearly 23 years).

A 2003 study found that on the Web, about one link out of every 200 broke each week,<ref name=Fetterly2003>{{cite conference | first1 = Dennis | last1 = Fetterly | first2 = Mark | last2 = Manasse | first3 = Marc | last3 = Najork | first4 = Janet | last4 = Wiener | year = 2003 | title = A large-scale study of the evolution of web pages | url = http://www2003.org/cdrom/papers/refereed/p097/P97%20sources/p97-fetterly.html | book-title = Proceedings of the 12th international conference on World Wide Web | access-date = 14 September 2010 | conference =  | archive-date = 9 July 2011 | archive-url = https://web.archive.org/web/20110709175020/http://www2003.org/cdrom/papers/refereed/p097/P97%20sources/p97-fetterly.html | url-status = live }}</ref> suggesting a [[half-life]] of 138 weeks. This rate was largely confirmed by a 2016–2017 study of links in [[Yahoo! Directory]] (which had stopped updating in 2014 after 21 years of development) that found the half-life of the directory's links to be two years.<ref>{{cite web |last=van der Graaf |first=Hans |title=The half-life of a link is two year |url=http://blog.zomdir.com/2017/10/the-half-life-of-link-is-two-year.html |website=ZOMDir's blog |access-date=2019-01-31 |url-status=live |archive-url=https://web.archive.org/web/20171017041901/http://blog.zomdir.com/2017/10/the-half-life-of-link-is-two-year.html |archive-date=2017-10-17}}</ref>

A 2004 study showed that subsets of Web links (such as those targeting specific file types or those hosted by academic institutions) could have dramatically different half-lives.<ref name=Koehler2004>{{cite journal | first = Wallace | last = Koehler | year = 2004 | title = A longitudinal study of web pages continued: a consideration of document persistence | url = http://www.informationr.net/ir/9-2/paper174.html | journal = Information Research | volume = 9 | issue = 2 | access-date = 2019-01-31 | archive-url = https://web.archive.org/web/20170911062629/http://www.informationr.net/ir/9-2/paper174.html | archive-date = 2017-09-11 | url-status = live}}</ref> The URLs selected for publication appear to have greater longevity than the average URL. A 2015 study by Weblock analyzed more than 180,000 links from references in the full-text corpora of three major open access publishers and found a half-life of about 14 years,<ref>{{cite web | title = All-Time Weblock Report | date = August 2015 | url = https://weblock.io/report?id=all | access-date = 12 January 2016 | url-status = dead | archive-url = https://web.archive.org/web/20160304081204/https://weblock.io/report?id=all | archive-date = 4 March 2016}}</ref> generally confirming a 2005 study that found that half of the [[Uniform Resource Locator|URLs]] cited in ''[[D-Lib Magazine]]'' articles were active 10 years after publication.<ref name=McCown2005>{{cite conference | first1 = Frank | last1 = McCown | first2 = Sheffan | last2 = Chan | first3 = Michael L. | last3 = Nelson | first4 = Johan | last4 = Bollen | year = 2005 | title = The Availability and Persistence of Web References in D-Lib Magazine | url = http://www.iwaw.net/05/papers/iwaw05-mccown1.pdf | url-status = dead | book-title = Proceedings of the 5th International Web Archiving Workshop and Digital Preservation (IWAW'05) | access-date = 2005-10-12 | archive-url = https://web.archive.org/web/20120717000118/http://www.iwaw.net/05/papers/iwaw05-mccown1.pdf | archive-date = 2012-07-17 }}</ref> Other studies have found higher rates of link rot in academic literature but typically suggest a half-life of four years or greater.<ref name=Spinellis2003>{{cite journal | author-link = Diomidis Spinellis | first = Diomidis | last = Spinellis | year = 2003 | title = The Decay and Failures of Web References | url = http://www.spinellis.gr/pubs/jrnl/2003-CACM-URLcite/html/urlcite.html | journal = Communications of the ACM | volume = 46 | issue = 1 | pages = 71–77 | doi = 10.1145/602421.602422 | citeseerx = 10.1.1.12.9599 | s2cid = 17750450 | access-date = 2007-09-29 | archive-date = 2020-07-23 | archive-url = https://web.archive.org/web/20200723030709/https://www.spinellis.gr/pubs/jrnl/2003-CACM-URLcite/html/urlcite.html | url-status = live }}</ref><ref name=Lawrence2001>{{Cite Q | Q21012586 }}</ref> A 2013 study in ''[[BMC Bioinformatics]]'' analyzed nearly 15,000 links in abstracts from Thomson Reuters's [[Web of Science]] citation index and found that the median lifespan of web pages was 9.3 years, and just 62% were archived.<ref>{{Cite journal | last1 = Hennessey | first1 = Jason | last2 = Xijin Ge | first2 = Steven | title = A Cross Disciplinary Study of Link Decay and the Effectiveness of Mitigation Techniques | journal = BMC Bioinformatics | volume = 14 | pages = S5 | date = 2013 | issue = Suppl 14 | doi = 10.1186/1471-2105-14-S14-S5 | pmid = 24266891 | pmc = 3851533 | doi-access = free }}</ref> A 2021 study of external links in ''[[New York Times]]'' articles published between 1996 and 2019 found a half-life of about 15 years (with significant variance among content topics) but noted that 13% of functional links no longer lead to the original content—a phenomenon called ''content drift''.<ref>{{Cite web|title=What the ephemerality of the Web means for your hyperlinks|url=https://www.cjr.org/analysis/linkrot-content-drift-new-york-times.php|access-date=2021-08-02|website=Columbia Journalism Review|language=en|archive-date=2021-08-02|archive-url=https://web.archive.org/web/20210802134941/https://www.cjr.org/analysis/linkrot-content-drift-new-york-times.php|url-status=live}}</ref>

A 2013 study found that 49% of links in U.S. Supreme court opinions are dead.<ref>{{Cite web |last=Garber |first=Megan |date=2013-09-23 |title=49% of the Links Cited in Supreme Court Decisions Are Broken |url=https://www.theatlantic.com/technology/archive/2013/09/49-of-the-links-cited-in-supreme-court-decisions-are-broken/279901/ |access-date=2024-01-10 |website=The Atlantic |language=en}}</ref>

A 2023 study looking at United States [[COVID-19]] dashboards found that 23% of the state dashboards available in February of 2021 were no longer available at the previous URLs in April of 2023.<ref name="Adams1">{{cite journal |last1=Adams |first1=Aaron M. |last2=Chen |first2=Xiang |last3=Li |first3=Weidong |last4=Chuanrong |first4=Zhang |title=Normalizing the pandemic: exploring the cartographic issues in state government COVID-19 dashboards |journal=Journal of Maps |date=27 July 2023 |volume=19 |issue=5 |pages=1–9 |doi=10.1080/17445647.2023.2235385|doi-access=free }}</ref>

==Causes==
Link rot can result from several occurrences. A target web page may be removed. The server that hosts the target page could fail, be removed from service, or relocate to a new [[domain name]]. As far back as 1999, it was noted that with the amount of material that can be stored on a hard drive, "a single disk failure could be like the burning of the library at Alexandria."<ref name="McGranaghan1999">{{cite journal |last1=McGranaghan |first1=Matthew |date=1999 |title=The Web, Cartography and Trust |url=https://cartographicperspectives.org/index.php/journal/article/view/cp32-mcgranaghan |journal=Cartographic Perspectives |volume= |issue=32 |pages=3–5 |doi=10.14714/CP32.624 |doi-access=free}}</ref>  A domain name's registration may lapse or be transferred to another party. Some causes will result in the link failing to find any target and returning an error such as [[HTTP 404]]. Other causes will cause a link to target content other than what was intended by the link's author. 

Other reasons for broken links include:
* the restructuring of websites that causes changes in URLs (e.g. {{code|domain.net/pine_tree}} might be moved to {{code|domain.net/tree/pine}})
* relocation of formerly free content to behind a [[paywall]]<ref name="Adams1" />
* a change in server architecture that results in code such as [[PHP]] functioning differently
* dynamic page content such as search results that changes by design
*deletion of the target page and/or its content
* the presence of user-specific information (such as a login name) within the link
* deliberate blocking by [[content filter]]s or [[Firewall (networking)|firewalls]]
* the expiration of a [[domain name registration]]

==Prevention and detection==
Strategies for preventing link rot can focus on placing content where its likelihood of persisting is higher, authoring links that are less likely to be broken, taking steps to preserve existing links, or repairing links whose targets have been relocated or removed. {{Citation needed|date=February 2022}}

The creation of URLs that will not change with time is the fundamental method of preventing link rot. Preventive planning has been championed by [[Tim Berners-Lee]] and other web pioneers.<ref name=Berners-Lee1998>{{cite web |author-link = Tim Berners-Lee |first = Tim |last = Berners-Lee |title = Cool URIs Don't Change |year = 1998 |url=https://www.w3.org/Provider/Style/URI |access-date=2019-01-31 | url-status=live |archive-url=https://web.archive.org/web/20000302064802/http://www.w3.org/Provider/Style/URI |archive-date=2000-03-02}}</ref>

Strategies pertaining to the authorship of links include:
* linking to primary rather than secondary sources and prioritizing stable sites<ref name="Koehler2004" />
* avoiding links that point to resources on researchers' personal pages<ref name=McCown2005/>
* using [[clean URL]]s or otherwise employing [[URL normalization]] or [[URL canonicalization]]<ref name=Kille2014>{{cite web | last = Kille | first = Leighton Walter | title = The Growing Problem of Internet "Link Rot" and Best Practices for Media and Online Publishers | publisher = Journalist's Resource, Harvard Kennedy School | date = 8 November 2014 | url = http://journalistsresource.org/studies/society/internet/website-linking-best-practices-media-online-publishers | access-date = 16 January 2015 | url-status = live | archive-url = https://web.archive.org/web/20150112034707/http://journalistsresource.org/studies/society/internet/website-linking-best-practices-media-online-publishers | archive-date = 12 January 2015}}</ref>
* using [[permalinks]] and [[persistent identifier]]s such as ARKs, [[Digital object identifier|DOIs]], Handle System references, [[PURL]]s, {{Citation needed|date=February 2022}} or [[content-addressable storage|content addressing]]<ref>Sicilia, Miguel-Angel, et al. "[https://www.sciencedirect.com/science/article/pii/S1877050919300924/pdf?md5=e9f04c24a1d7114f75df4adbe2a373db&pid=1-s2.0-S1877050919300924-main.pdf Decentralized Persistent Identifiers: a basic model for immutable handlers] {{Webarchive|url=https://web.archive.org/web/20230510232202/https://www.sciencedirect.com/science/article/pii/S1877050919300924/pdf?md5=e9f04c24a1d7114f75df4adbe2a373db&pid=1-s2.0-S1877050919300924-main.pdf |date=2023-05-10 }}." Procedia computer science 146 (2019): 123-130.</ref>
* avoiding linking to documents other than web pages<ref name=Kille2014/>
* avoiding [[deep linking]] {{Citation needed|date=February 2022}}
* linking to [[web archives]] such as the [[Internet Archive]],<ref>{{cite web | url=https://archive.org/ | title=Internet Archive: Digital Library of Free Books, Movies, Music & Wayback Machine | date=2001-03-10 | access-date=7 October 2013 | url-status=live | archive-url=https://web.archive.org/web/19970126045828/http://www.archive.org/ | archive-date=26 January 1997}}</ref> [[WebCite]],<ref name=Eysenbach2005>{{cite journal | first1 = Gunther | last1 = Eysenbach | first2 = Mathieu | last2 = Trudel | year = 2005 | title = Going, going, still there: Using the WebCite service to permanently archive cited web pages | doi = 10.2196/jmir.7.5.e60 | journal = Journal of Medical Internet Research | volume = 7 | issue = 5 | pages = e60 | pmid = 16403724 | pmc = 1550686 | doi-access = free }}</ref> [[archive.today]], [[Perma.cc]],<ref name=permacc>{{cite journal |last1 = Zittrain |first1 = Jonathan |last2 = Albert |first2 = Kendra |last3 = Lessig |first3 = Lawrence |url = https://cdn.harvardlawreview.org/wp-content/uploads/2014/03/forvol127_zittrain.pdf |title = Perma: Scoping and Addressing the Problem of Link and Reference Rot in Legal Citations |journal = Legal Information Management |volume = 14 |issue = 2 |pages = 88–99 |date = 12 June 2014 |doi = 10.1017/S1472669614000255 |s2cid = 232390360 |access-date = 10 June 2020 |archive-date = 1 November 2020 |archive-url = https://web.archive.org/web/20201101012831/https://cdn.harvardlawreview.org/wp-content/uploads/2014/03/forvol127_zittrain.pdf |url-status = live }}</ref> Amber,<ref>{{Cite web|title = Harvard University's Berkman Center Releases Amber, a "Mutual Aid" Tool for Bloggers & Website Owners to Help Keep the Web Available {{!}} Berkman Center|url = https://cyber.law.harvard.edu/node/99276|website = cyber.law.harvard.edu|access-date = 2016-01-28|url-status = live|archive-url = https://web.archive.org/web/20160202042259/https://cyber.law.harvard.edu/node/99276|archive-date = 2016-02-02}}</ref> or Arweave<ref>{{Cite web |title=Arweave - A community-driven ecosystem |url=https://arweave.org/ |access-date=2023-03-15 |website=arweave.org |archive-date=2023-03-15 |archive-url=https://web.archive.org/web/20230315024155/https://arweave.org/ |url-status=live }}</ref>
Strategies pertaining to the protection of existing links include:
* using [[URL redirection|redirection]] mechanisms such as [[HTTP 301]] to automatically refer browsers and crawlers to relocated content. {{Citation needed|date=February 2022}}
* using [[Web content management system|content management systems]] which can automatically update links when content within the same site is relocated or automatically replace links with canonical URLs<ref name="Justaddwater 2007">{{cite web | last = Rønn-Jensen | first = Jesper | title = Software Eliminates User Errors And Linkrot | publisher = Justaddwater.dk | date = 2007-10-05 | url = http://justaddwater.dk/2007/10/05/blog-software-eliminates-user-errors-and-linkrot/ | access-date = 5 October 2007 | url-status = live | archive-url = https://web.archive.org/web/20071011033526/http://justaddwater.dk/2007/10/05/blog-software-eliminates-user-errors-and-linkrot/ | archive-date = 11 October 2007}}</ref>
* integrating search resources into [[HTTP 404]] pages<ref name="GoogleToolbar">{{cite web | last = Mueller | first = John | title = FYI on Google Toolbar's Latest Features | publisher = Google Webmaster Central Blog | date = 2007-12-14 | url = http://googlewebmastercentral.blogspot.com/2007/12/fyi-on-google-toolbars-latest-features.html | access-date = 9 July 2008 | url-status = live | archive-url = https://web.archive.org/web/20080913132848/http://googlewebmastercentral.blogspot.com/2007/12/fyi-on-google-toolbars-latest-features.html | archive-date = 13 September 2008}}</ref>

The detection of broken links may be done manually or automatically. Automated methods include [[Plug-in (computing)|plug-ins]] for [[content management system]]s as well as standalone broken-link checkers such as like [[Xenu's Link Sleuth]]. Automatic checking may not detect links that return a [[soft 404]] or links that return a [[200 OK]] response but point to content that has changed.<ref name=Bar-Yossef2004>{{cite conference |first1= Ziv |last1 = Bar-Yossef |first2 = Andrei Z. |last2 = Broder |first3 = Ravi |last3 = Kumar |first4 = Andrew |last4 = Tomkins |year = 2004 |title = Sic transit gloria telae: towards an understanding of the Web's decay |book-title = Proceedings of the 13th international conference on World Wide Web – WWW '04 |pages = 328–337 |doi = 10.1145/988672.988716 |isbn = 978-1581138443 |citeseerx = 10.1.1.1.9406}}</ref>

==See also==
* [[Archive Team]], web archiving team
* [[Dead Internet theory]]
* [[Deletionism and inclusionism in Wikipedia]]
* [[Digital preservation]]
* [[Infodemic]]
* [[Software rot]]
* [http://info.cern.ch Info - Cern] 

==Further reading==

* {{cite journal | first1 = John | last1 = Markwell | first2 = David W. | last2 = Brooks | year = 2002 | title = Broken Links: The Ephemeral Nature of Educational WWW Hyperlinks | doi = 10.1023/A:1014627511641 | journal = Journal of Science Education and Technology | volume = 11 | issue = 2 | pages = 105–108| s2cid = 60802264 }}
* {{cite conference | first1 = Daniel | last1 = Gomes | first2 = Mário J. | last2 = Silva | year = 2006 | title = Modelling Information Persistence on the Web | url = http://xldb.di.fc.ul.pt/daniel/docs/papers/gomes06urlPersistence.pdf | book-title = Proceedings of the 6th International Conference on Web Engineering | conference = ICWE'06 | access-date = 14 September 2010 | archive-url = https://web.archive.org/web/20110716190802/http://xldb.di.fc.ul.pt/daniel/docs/papers/gomes06urlPersistence.pdf | archive-date = 2011-07-16 | url-status = dead}}
* {{cite journal | first1 = Robert P. | last1 = Dellavalle | first2 = Eric J. | last2 = Hester | first3 = Lauren F. | last3 = Heilig | first4 = Amanda L. | last4 = Drake | first5 = Jeff W. | last5 = Kuntzman | first6 = Marla | last6 = Graber | first7 = Lisa M. | last7 = Schilling | year = 2003 |url= https://www.researchgate.net/publication/9030760 | title = Going, Going, Gone: Lost Internet References | doi = 10.1126/science.1088234 | journal = Science | volume = 302 | issue = 5646 | pages = 787–788 | pmid = 14593153| s2cid = 154604929 }}
* {{cite journal | first = Wallace | last = Koehler | year = 1999 | title = An Analysis of Web Page and Web Site Constancy and Permanence | journal = Journal of the American Society for Information Science | volume = 50 | issue = 2 | pages = 162–180 | doi = 10.1002/(SICI)1097-4571(1999)50:2<162::AID-ASI7>3.0.CO;2-B}}
* {{cite journal | first = Carmine | last = Sellitto | year = 2005 |url= http://vuir.vu.edu.au/336/1/The_impact_of_impermanent_Web.pdf | title = The impact of impermanent Web-located citations: A study of 123 scholarly conference publications | journal = Journal of the American Society for Information Science and Technology | volume = 56 | issue = 7 | pages = 695–703 | doi = 10.1002/asi.20159| citeseerx = 10.1.1.473.2732}}

==References==
{{Reflist|35em}}

==External links==
{{Prone to spam|date=November 2015}}
{{Wikibooks
|1= Authoring Webpages
|2= Preventing link rot
}}
{{Wiktionary|link rot|linkrot}}

*[https://web.archive.org/web/20061109071631/http://www.wrox.com/WileyCDA/Section/id-301495.html Future-Proofing Your URIs]
*{{cite web|url=http://www.useit.com/alertbox/980614.html|title=Fighting Linkrot|authorlink=Jakob Nielsen (usability consultant)|last=Nielsen|first=Jakob|date=14 June 1998|archive-url=https://web.archive.org/web/20121223011620/http://www.useit.com/alertbox/980614.html|archive-date=23 December 2012}}

[[Category:Data quality]]
[[Category:Product expiration]]
[[Category:URL]]