edit: adjusted title slightly

    • kautau@lemmy.world
      link
      fedilink
      English
      arrow-up
      8
      ·
      1 month ago

      The Wayback machine is a crawler, which is big part of what they do but not everything. The Wayback machine crawls its own pages, but you can also submit URLs to be crawled.

      The other part of what they do is hosting a significant number of digital archives of media that is no longer sold / in print / distributed. Much of that content is user uploaded. Like “oh hey I found this old clip art cd from the early 90s. I don’t really have a use for it, but if this doesn’t get uploaded somewhere it’s probably going to be lost to time. I’ll submit it to the internet archives.”

    • pmc
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 month ago

      They do some crawling themselves, but Archive Team (a third party group) does a lot of web archiving as well.