From Apache OpenOffice Wiki
Jump to: navigation, search
NOTE: Metalink is unrelated to Oracle Corporation's support website.

Metalink is a cross-platform and cross-application open standard/framework/file format for programs that download (download managers, BitTorrent clients, Web browsers, FTP clients, & P2P programs). For increased convenience, it stores the multiple download locations for a file (FTP/HTTP/P2P) in a single metafile with the extension .metalink. This provides reliability in case one method fails. For fast Internet connections, it also speeds up the process by allowing chunks/segments of each file to be downloaded from multiple resources at the same time (segmented downloading).

Other features include automatic checksum verification of finished downloads, automatic repair of downloads with errors, listing multiple files to be added to a download queue, and automatic OS/language selection.

Metalink is extensible, and supports listing multiple partial and full file hashes/checksums along with PGP signatures. Most clients only support verifying MD5, SHA-1, & SHA-256 however. Besides FTP and HTTP mirror locations and rsync, it also supports listing the P2P methods BitTorrent, ed2k, and magnet link.

Here's how it works: One link, one click downloads for all people. For instance, a Metalink for the installation files can list every OS and language combination and their mirror locations and checksums. Someone in Brazil that speaks French and uses Linux can click on the link & get the French Linux version from the local Brazilian mirrors. Someone in Japan that speaks English on Mac OS X can click that same link & get the English OS X version from the local Japanese mirrors. Once the download finishes, the checksum is automatically verified, without user interaction. (This is available now, but not all clients function this way by default).

Client programs

  • aria2 (GPL) is a Unix/Windows command line "high speed download utility" that supports BitTorrent and Metalink (HTTP/FTP/BitTorrent).
  • DownThemAll (GPL) is a Firefox extension download manager that has basic Metalink support.
  • FlashGot (GPL), a Firefox extension for integrating with many Download Managers, supports Metalink.
  • Free Download Manager (GPL) is a Windows download manager with Metalink support.
  • GetRight, a Windows Download Manager and BitTorrent client, supports Metalink (HTTP/FTP/BitTorrent).
  • KGet (GPL), the download manager that is part of KDE4, supports Metalink.
  • Metadl (LGPL) is a NSIS download plugin based on cURL that supports Metalink and downloads from one URL, supports download resume, and verifies checksums. Metadl is used by Wubi, the official Windows-based Ubuntu installer.
  • Metalink Checker (GPL) is a cross platform command line downloader written in Python that supports multi-source downloads, as well as checking mirrors listed in .metalinks.
  • Net Transport, a Windows Download Manager, supports Metalink.
  • Orbit Downloader, a Windows Download Manager, supports Metalink (HTTP/FTP).
  • Phex (GPL), a Mac/Unix/Windows Gnutella client, supports downloading with and exporting a library to Metalink for sharing with others.
  • Retriever is a cross platform Java download manager that supports Metalink and BitTorrent.
  • SmartFTP, a Windows GUI FTP client, supports Metalink for adding files to an FTP transfer queue and checksum verification (does not supported segmented downloads).
  • Speed Download, a Mac Download Manager, supports Metalink (HTTP/FTP).
  • TheWorld Browser is a Web Browser on Windows that has built-in Metalink (HTTP/FTP) support.
  • VeryCD easyMule is a version of eMule with Metalink support.
  • wxDownload Fast (GPL) is a Mac/Unix/Windows download manager that supports Metalink (HTTP/FTP).
  • Also, Appupdater (GPL) for Windows "is similar to apt-get or yum on Linux. It automates the process of installing and maintaining up to date versions of programs." It uses Metalinks for a specific set of programs and does not allow downloading with other .metalinks obtained elsewhere.

See a Comparison of download managers for details on which of them support Metalink and have other features.

In use

Currently, uses Metalinks on an alternate P2P download page with Magnet links.

Wubi, the Windows-based Ubuntu installer, uses Metadl (LGPL) to download Ubuntu ISO images and takes advantage of Metalink's features of higher availability and increased reliability. If there are errors in the download, they are repaired, instead of restarting the large download. cURL, a command line download tool, offers Metalinks of their downloads. UniProt Consortium also provides a Metalink for their large protein database.

The following distributions use Metalink for ISO distribution: openSUSE, Sabayon, Arch Linux, DesktopBSD, PC-BSD, BLAG Linux and GNU, StartCom Enterprise Linux, Berry Linux, openSUSE, Ubuntu, PuppyLinux, redWall Firewall.

Appupdater (GPL) for Windows "is similar to apt-get or yum on Linux. It automates the process of installing and maintaining up to date versions of programs."

It is also being tested with Arch Linux's Pacman which is used for package management.

Metalink generation

Metalink is XML so it can be hand crafted or XML tools can be used for automation. These tools are specifically for Metalink.

  • Bouncer (GPL) is a "database driven mirror management app that has three main components - a bounce script, a sentry script to check mirrors and files for integrity and a set of web-based administration tools." Bouncer supports Metalink via that has not yet been integrated upstream.
  • cURL uses Perl scripts to offer Metalinks on their download pages.
  • Geo McFly (GPL) is an on the fly generator of metalinks based on the downloader's geographical location, written in Python.
  • Metalink Editor (GPL) is a Cross Platform GUI Metalink editor written in Python which allows you to create and edit Metalinks.
  • Metalink Library (GPL) is a Cross Platform Metalink library for Python and PHP based on Metalink Editor with many automatic features and an easy command-line interface to generate Metalink and Torrent files.
  • Metalink tools (GPL) Unix/Windows command line tools for generating Metalinks with checksums and mirror/p2p lists.
  • Metamirrors Metagenerate provides an easy to use Web UI to create Metalinks.
  • MirrorManager (MIT X11 License) is used by the Fedora Project for dynamically listing mirrors. This patch has not yet been integrated upstream.
  • openSUSE download redirector (the MirrorBrain) supports Metalink and is used by openSUSE and automatically redirects clients (per HTTP redirection) to a mirror server near them.
  • Origo is a new and easy to use software development and hosting platform for open and closed source projects. Among many features, it has easy release management and mirroring with automatic metalink generation.
  • Simba (GPL), the mirroring tool, includes automatic Metalink generation as one of its features.

Example .metalink file

.metalinks are XML text files.

<metalink version="3.0" xmlns="">
    <file name="example.ext">
      <hash type="md5">example-md5-hash</hash>
      <hash type="sha1">example-sha1-hash</hash>
      <signature type="pgp"/>
      <url type="ftp" location="us" preference="90"></url>
      <url type="ftp" location="uk" preference="90"></url>
      <url type="http" location="us" preference="90"></url> 
      <url type="http" location="uk" preference="90"></url>
      <url type="http" location="de" preference="90"></url> 
      <url type="bittorrent" preference="100"></url>
      <url type="magnet"/>
      <url type="ed2k"/>

Future Use

Possible areas of future investigation for

  • More testing on download pages.
  • "Check for Updates"/Autoupdate feature in 2.0.4+. Appupdater checks for updates to many programs, including Appupdater is similar to apt-get or yum, but for Windows.
  • Integration with Bouncer and a Firefox extension like DownThemAll. (Bouncer patches have not been accepted upstream yet).

Press descriptions

"Downloading bliss with Metalink" by Mayank Sharma:

"Getting popular software off the Internet can sometimes be a struggle, even with all the mirrors and BitTorrent Samaritans out there. When the Fedora project released Fedora Core 6 last month, for instance, even several dozen mirrors weren't enough to serve everyone, and torrent speeds weren't good enough because of a scarcity of seeders. But thanks to Metalink I was able to sleep while my FC6 ISOs were downloading.

Metalink is an open standard that claims to make downloading easier, faster, and more reliable by helping users extract the last drop of juice out of their connection. But Metalink isn't your run-of-the-mill download accelerator. It is in fact a framework for use by other download clients, and bundles traditional HTTP and FTP methods of downloading files along with BitTorrent.

The Metalink standard replaces static URLs with a .metalink file, which is a simple XML file. The file in turn contains locations of all the mirrors of the application you want to download. In addition to HTTP and FTP mirrors and rsync, Metalink supports several P2P methods as well, including BitTorrent, ed2k, and magnet links. For example, the metalink contains links to more than 50 HTTP and FTP servers and a torrent.

If I were using a regular hyperlink to download and the server went down midway through the transfer, I would be left with an incomplete transfer that could either be resumed or not depending on the download client. But with Metalink, if one server goes down, the client software simply jumps to another mirror and resumes downloading from that point. In effect, this increases reliability, as all listed servers would need to be down for the file to be unavailable.

Files downloaded with metalinks are automatically verified. Metalink supports both MD5SUM and SHA1SUM checksums as well as PGP signatures, which are embedded in the .metalink file itself. Checksums are unique for every file, like a fingerprint. If there's an error in transfer, or if someone has maliciously replaced a good file with a bad one, the checksums won't match. If that happened with another file transfer method, you'd have to download the file again from another mirror. With Metalink, if the application you are downloading has a torrent, Metalink can use a torrent's partial file or chunk checksums to verify mirror downloads as well. If only a small chunk of a download has errors, Metalink just re-downloads that part instead of the whole file."

OSresources article:

"Metalink is a system which improves the download process by increasing availability and guaranteeing integrity. It can give your users a more reliable download by providing multiple links to the same file, which can be switched to if one server is down or fails during transmission. It can also make downloads faster by using multiple resources at once. Organizations use mirror servers in multiple locations to provide redundancy and place content nearby users in the hopes that it will be faster for local users. Examples would be mirrors on each continent or near a concentration of users. Downloaders in Europe would manually choose a mirror located in a European country, preferably their own, and people in Japan or other countries would choose mirrors close to them. If mirrors are down they can be used by downloaders in other locations as a backup or second option. Metalink lists mirrors with machine readable information on priority and location so their efficient use can be automated by download programs. It can list mirrors around the world, but will automatically default to mirrors closer to you and by priority.

Download programs (also known as download managers) that are Metalink-aware read the information from a Metalink file (one with a .metalink extension), which is just plain XML. The download programs split the file tranfer into independently verifiable segments which are re-assembled. By downloading multiple segments from multiple servers, download speeds are usually three to four times faster on an average broadband connection and much faster on business grade connections. The checksum verification process, usually manual and arcane to most people, is automated with Metalink, so files are guaranteed to be an exact copy of the file you downloaded, free of errors. Metalinks can also contain publisher information, Operating System and architecture, language, file descriptions, mutliple files (to be added to a download queue) and so on. All this extra information allows download programs to do interesting things. It lets you have one download link for multiple Operating Systems and languages, and downloaders will get the correct version for their system."

TorrentFreak explains Metalink like this:

"We’ve all seen those download pages where a couple of http, ftp and bittorrent mirrors are listed. We often just pick one of these, but wouldn’t it be great if you could use them all at the same time? Metalink has the solution. It works like this, the .metalink file tells your download manager where it can get the file, and connects to these different sources at the same time."

downloadsquad describes it this way:

"Metalinks make complex download pages obsolete by replacing long lists of download mirrors and BitTorrent trackers with a single .metalink file. As you might have already guessed, a .metalink file is a file that tells a download manager all the different ways it can download a file. The file itself takes the form of an open XML standard that can list an unlimited number of HTTP and FTP sources as well as BitTorrent trackers and ed2k and magnet links."

External links

Personal tools