Web crawling, indexing, caching and scraping
Produced in partnership with Toby Headdon of Bristows and Hannah Crowther of Bristows
Web crawling, indexing, caching and scraping

The following TMT practice note produced in partnership with Toby Headdon of Bristows and Hannah Crowther of Bristows provides comprehensive and up to date legal information covering:

  • Web crawling, indexing, caching and scraping
  • Online operations following IP completion day
  • An explanation of the terminology
  • Crawling
  • Indexing
  • Caching
  • Scraping
  • Legal issues—copyright infringement
  • Indexing
  • Caching
  • More...

Web crawling, indexing, caching and scraping

This Practice Note explains the manner in which intellectual property and certain other rights, as well as data protection obligations, apply in the context of web crawling, indexing, caching and scraping on the web, with specific regard to the perspective of a website operator. It covers:

  1. Online operations following IP completion day

  2. An explanation of the terminology

  3. Legal issues—copyright infringement

  4. Legal issues—database right infringement

  5. Legal issues—website terms of use

  6. Legal issues—data protection

  7. Compliance with the Robot Exclusion Protocol and/or instructions in metatags

  8. Computer Misuse Act 1990

Online operations following IP completion day

Although, for the most part, EU legislation relevant to crawling, indexing, caching and scraping is retained in the UK as retained EU law, the end of the transition period is significant for UK businesses with online operations in the EU. In addition to the UK becoming a third country and no longer a Member State, to the extent that the adopted UK position diverges from EU law or there is further deregulation in the future, the compliance burden for UK organisations operating online will increase.

The ‘country of origin’ principle, established by Directive 2000/31/EC, the EU E-Commerce Directive, and providing, in summary, that an online service provider established in a Member State and complying with its own national laws can provide services across the EU, will not apply to the UK

Popular documents