Wayback Machine Downloader Not Working? Causes & Fixes

When Wayback Machine Downloader is not working, the cause is usually one of five things. The Internet Archive may be rate-limiting or temporarily blocking your IP, which shows up as connection refused. The request for the capture list may have timed out. The files may have come back empty and been deleted by the script. The URL you passed may only match the homepage. Or the pages were never captured. Each problem has its own fix, and this guide matches each symptom to its cause. If you would rather not use Ruby or a terminal, the last sections show how to restore the site in your browser.

What Wayback Machine Downloader is and why it so often stops working

Wayback Machine Downloader is an open-source Ruby script, installed with gem install wayback_machine_downloader. It works in two stages:

1. It asks the Internet Archive's CDX API for a list of every capture of your domain.
2. It downloads each file from web.archive.org, one request per file, using the raw capture without the Wayback toolbar.

The design is simple, and that is why it breaks. A site with a few thousand files means a few thousand requests to a free, heavily used, non-profit service. That service protects itself with rate limits.

The original script has also seen little maintenance in recent years. Over the same period, archive.org has tightened how it treats rapid automated traffic. Much of the advice in older GitHub issues was written before those changes, which is why it often no longer helps.

Symptom check: which failure do you have?

Why does Wayback Machine Downloader get "connection refused"?

The Wayback Machine Downloader connection refused error almost never means the script is broken. It means web.archive.org has stopped accepting connections from your IP address for a while. This is often called a soft ban: a temporary block, not a permanent one.

How to recognise a soft ban

• The first part of the download worked, then errors started piling up.
• Opening web.archive.org in your browser on the same network also fails or hangs, while other sites load fine.
• The same command works from a different network, such as a phone hotspot.

If all three are true, you are rate-limited. Running the script again right away makes it worse, because every new attempt counts as more traffic.

How long does it last, and what should you do?

The Internet Archive does not publish its limits. Users report everything from a few minutes to several hours, so plan for at least an hour. While you wait:

• Stop every running instance of the script, including ones in other terminal windows.
• Lower concurrency. If you used -c or --concurrency with a value above 1, go back to -c 1. Parallel downloads are the most common trigger.
• Download in chunks. Use --only to fetch one section at a time, for example --only "/blog/", with breaks between runs.
• Limit the time range. --from 20150101 --to 20181231 reduces how many captures the script considers.

Don't try to dodge the ban by rotating through VPNs or proxies. It puts extra load on a free public archive, and it tends to lead to longer blocks.

Why are the downloaded files empty, or why do they disappear?

The script creates a file, writes whatever the server returns into it, and deletes the file if the result is empty. So if you watch the output folder during a bad run, files appear and then vanish. Several things can cause this:

• Rate limiting partway through. Once archive.org starts refusing requests, every following file comes back empty. The fix is the one from the previous section.
• Captures that are redirects or errors. Many captures are 301/302 redirects or 404 pages recorded by the crawler. By default the script skips non-200 captures. With -a/--all it keeps them, which can leave you with useless files.
• Truncated captures. Some captures, especially large images and video, were only partly saved. Nothing on your side can recover what the archive never stored.

A quick check: open one of the missing URLs in the Wayback Machine in your browser. If the browser shows the page, the capture exists and the problem was the download. If it doesn't, the script never had anything to save.

Why does the script fail while fetching the list of captures?

Before it downloads anything, the script fetches the capture index from the CDX API. For a large domain with many years of history, that index can contain hundreds of thousands of rows, and the request can time out or return incomplete data. You see this as an error right after "Getting snapshot pages", or as a JSON parsing error.

Fixes, in order:

1. Wait and retry. The CDX API is sometimes slow on its own, and a refused CDX request can also be a sign of a soft ban.
2. Shrink the index with --from and --to. A three-year window is far lighter than fifteen years.
3. Inspect without downloading. wayback_machine_downloader https://example.com --list prints the capture list as JSON and downloads nothing. If that fails, the problem is the index, not the file downloads.

Why does Wayback Machine Downloader only download the homepage?

This is the most confusing failure, because the script reports success. Work through these causes:

• Hostname mismatch. The archive may have captured the site mostly as www.example.com while you passed example.com, or the other way round. HTTP and HTTPS captures can also differ. Try each variant.
• Wildcard scope. If you passed a deep URL, or used --exact-url, only that one URL is fetched. Pass the bare domain without --exact-url. If you query the CDX API directly, add /* to the base URL so it matches everything under it.
• Time range too narrow. In a given period, the crawler may have saved only the homepage. Widen or move the --from/--to window to years where the Wayback calendar shows more activity.
• Thin capture coverage. Run --list and count the entries. If the list holds only a handful of URLs, the archive never crawled deeper. This is common for small sites and for pages that sat behind forms or were only linked from JavaScript menus.
• JavaScript-built pages. If the site loaded its content with JavaScript, the archive may have captured an empty page shell. Why JavaScript sites produce blank copies explains this in detail.

Forks and the JavaScript version: what they change and what they can't fix

Because the original script is barely maintained, the community has produced several forks of the Ruby gem, plus ports to other languages, including JavaScript/Node and Python versions that don't need Ruby. The better-maintained forks typically add:

• built-in pauses between requests and automatic retries with back-off,
• the ability to resume a download that was interrupted,
• fixes for edge cases in the original's file and path handling.

These changes help: a fork that slows itself down is much less likely to trigger a soft ban than the original with -c 8. But no fork can remove archive.org's rate limits, and none can download pages the archive never captured. A JavaScript port spares you the Ruby install, but you still need Node.js and a terminal. Before you rely on any fork, look at when it was last updated and whether its open issues report the same failures you are seeing.

Restoring the site without a command line

If you need the site back and have no wish to debug Ruby gems, use a browser-based web archive website downloader such as the Wayback Machine downloader on this site. It runs on Website Downloader's servers, so nothing is installed on your computer and your IP is not the one sending thousands of requests to archive.org. It works the same on Windows, macOS, Linux, Chromebooks and tablets.

1. Enter the domain of the site that is gone from the web.
2. Pick the capture you want to restore from.
3. Follow the live progress while the pages download. You can close the tab; your computer doesn't need to stay on.
4. Download the result as one ZIP file.

Costs: a restore of up to 100 pages is free. A Wayback Restore is a one-time payment of 14.99 EUR for one site, covers up to 5000 subpages, and keeps the archive for 30 days. Monthly plans don't include restores. See pricing for details. For a full walkthrough, including how to choose a good capture, read Restore Website from Wayback Machine: Step-by-Step Guide.

The same limits as with the script apply: pages that were never captured can't be restored, and content that the archive only stored as a JavaScript shell will stay empty.

Script output vs. a browsable offline ZIP: what you actually get back

Even when the script runs without errors, you get a folder of raw files, not a site you can click through. It saves each file in its original path under websites/example.com/, using by default the latest capture of each file within your range. It does not rewrite links for offline use. Links that were written as full URLs still point to the original domain, so clicking them offline goes nowhere, and stylesheets referenced that way don't load. To browse the result you usually have to run a local web server or fix the links yourself.

A restore with Website Downloader is built to be browsed. The Wayback playback toolbar and the links back to archive.org are removed, and the archive opens offline in any browser.

Search boxes, contact forms, comments and shop checkouts won't work in either copy, because they need a live server. If images or styles are missing after you unpack an archive, see Downloaded Website Missing Images and CSS?

FAQ

Has the Internet Archive blocked my IP permanently?

Almost certainly not. The blocks people run into while using the script are temporary soft bans triggered by too many requests in a short time. Stop all downloads, wait at least an hour, and come back with concurrency set to 1 and a smaller scope.

Is there a Wayback Machine Downloader alternative that doesn't need Ruby?

Yes. There are community ports in JavaScript and Python, which still need a terminal and their own runtime. If you want no command line at all, a browser-based restore like the Wayback Machine downloader handles the crawl on a server and hands you a ZIP.

Can I download an entire website from the Wayback Machine for free?

The open-source script is free, but how much you get depends on rate limits and your patience. With Website Downloader, a restore of up to 100 pages is free. Larger sites need the one-time Wayback Restore, which covers up to 5000 subpages.

Why are some pages missing even after a successful restore?

The Wayback Machine never captured every page of every site. Pages that were only reachable through search, forms, logins or JavaScript menus are often absent. Check a missing URL directly in the Wayback Machine; if it isn't there, no tool can recover it.

Does the restored copy work offline?

A Website Downloader restore opens offline in any browser, with the Wayback toolbar and archive.org links removed. Pages, images, styles and scripts work, but features that needed the original server, such as search, forms and carts, do not. More answers are in the FAQ.