Legal deposit and web archiving - British Library
The British Library and other legal deposit libraries are entitled to copy UK-published material from the internet for archiving under legal deposit.
The British Library and other legal deposit libraries are entitled to copy UK-published material from the internet for archiving under legal deposit.
Following disruption in 2023, the UK Web Archive platform has been restored.
You can search the archive from anywhere, but most of the archived websites can only be viewed within a Legal Deposit Library reading room. Before visiting, please check with the relevant Legal Deposit Library to confirm that this service is available again.
We are continuing to make improvements to the UK Web Archive site following restoration. At the moment:
You can follow the work of the UK Web Archive team on their blog. You can access data about the UK Web Archive on our institutional repository.
Each year, our UK Web Archive collects a ‘snapshot’ of all the UK websites that we can identify. This includes at least four million websites, with several billion files.
We can identify a UK website if it has:
Our archiving of UK websites is governed by legal deposit law:
We don't collect any material that is only shared between restricted private groups, such as material posted to private networks on social media.
We don't collect material on the web where the content is solely either a sound recording or a recording of moving image. This includes material where there is some accompanying text, but the text would not make sense without the sound or moving image.
We do collect material from the web that includes sound and moving image alongside text or other elements, where the text or other elements have meaning independent to the sound and moving image.
The legal deposit libraries use web crawling software wherever possible, especially when collecting for the UK Web Archive. Web crawling is an automated process used to collect content and metadata that is available without access restriction on the open web. Crawled websites and material are preserved in the legal deposit libraries' web archive. We may also use manual or other methods of downloading content and metadata when necessary.
A seed list of domain addresses is programmed into the web crawling software as Uniform Resource Locators (URLs). The software uses these to initiate the process, requesting a copy of the root or home page. Then it automatically follows links to the next levels down within the same domain, issuing a separate request for each URL identified. Target websites respond automatically, delivering a copy of the page or file to which the URL relates.
The web crawling software is also programmed with politeness rules and parameters designed to ensure that there is no harmful impact upon the performance of the target website. For example, they include a limit on how many levels are crawled or how much content is requested from an individual website. Also, when multiple requests for different pages and files are issued to the same website, the software is programmed to leave an interval between each request, to safeguard against using up too much bandwidth and overloading the website.
The web crawling software uses standard automated protocols to identify itself and to inform the publisher’s webmaster (via information called a 'user-agent string' submitted to the web server’s log of server requests) on each occasion that a page is crawled. The website owner can choose whether or not to use this information, but is not required to take any action such as changing the website’s 'robots.txt' permission file.
Where the web crawling software encounters a login facility, it cannot access any material behind the login facility without the appropriate password or access credentials.
Our Framework for UK Legal Deposit (PDF, 289.7KB) provides information on how we are developing our capacity to collect digital publications.
Texto de bl.uk tal como lo publicó, leído de su página y mostrado aquí con el formato de El Autómata. El original, en bl.uk · Volver a la noticia de El Autómata