Metalink

Metalink is an open standard/framework for programs that download (download managers, BitTorrent clients, Web browsers, FTP clients, & P2P programs). For increased convenience, it stores the many locations of files (FTP/HTTP/P2P) in a single file (a .metalink) so chunks/segments of each file can be downloaded from multiple resources at the same time (known as accelerated/multi-threaded/segmented downloading). If you have a fast Internet connection, this usually results in a more reliable and much faster download than a typical single segment download from one server. Other features include automatic checksum comparison of finished downloads, and automatic OS/language selection. Metalinks are also backwards compatible with regular hyperlinks by using the format http://URL#!metalink3!http://URLtoMetalink.

Metalink is extensible, and supports listing MD5 & SHA-1 checksums along with PGP signatures. Besides FTP and HTTP mirror locations and rsync, it also supports listing these P2P methods: BitTorrent, ed2k, and magnet link.

Here's how it works: One link, one click downloads for all people. For instance, a Metalink for the OpenOffice.org installation files can list every OS and language combination and their mirror locations and checksums. Someone in Brazil that speaks French and uses Linux can click on the link & get the French Linux version from the local Brazilian mirrors. Someone in Japan that speaks English on Mac OS X can click that same link & get the English OS X version from the local Japanese mirrors. Once the download finishes, the checksum is automatically verified, without user interaction. (This is available now, but not all clients function this way by default).

Client programs

• aria2 (GPL) is a Unix command line "high speed download utility" that supports BitTorrent and Metalink (HTTP/FTP/BitTorrent).
• FlashGot (GPL), a Firefox extension for integrating with many Download Managers, supports Metalink.
• GetRight 6, a Windows Download Manager and BitTorrent client, supports Metalink (HTTP/FTP/BitTorrent).
• Speed Download, a Mac Download Manager, supports Metalink (HTTP/FTP)as of version 4.1.0.
• wxDownload Fast (GPL) is a Mac/Unix/Windows download manager that supports Metalink (HTTP/FTP).

See a Comparison of download managers for details on which of them support Metalink and have other features.

In use

Currently, OpenOffice.org uses Metalinks on an alternate P2P download page with Magnet links.

The following distributions use Metalink for ISO distribution: Arch Linux, DesktopBSD, PC-BSD, BLAG Linux and GNU, StartCom Enterprise Linux, Berry Linux, Ubuntu Christian Edition, PuppyLinux, redWall Firewall.

It is also being tested with Arch Linux's Pacman which is used for package management.

Metalink generation

Metalink is XML so it can be hand crafted or XML tools can be used for automation. These tools are specifically for Metalink.

• RoPkg::Metalink / Metalink @ Packages Resources (GPL) for automated Metalink generation.

• Metalink Generator can also be used to create Metalinks by filling out a form.

• Metalink tools (GPL) Unix/Windows command line tools for generating Metalinks with checksums and mirror/p2p lists.

• Simba (GPL), the mirroring tool, includes automatic Metalink generation as one of its features.

Example .metalink file

.metalinks are XML text files.

<metalink version="3.0" xmlns="http://www.metalinker.org/">
  <files>
    <file name="example.ext">
    <verification>
      <hash type="md5">example-md5-hash</hash>
      <hash type="sha1">example-sha1-hash</hash>
      <hash type="pgp"/>
    </verification>
    <resources>
      <url type="ftp" location="us" preference="90">ftp://ftp.example1.com/example.ext</url>
      <url type="ftp" location="uk" preference="90">ftp://ftp.example2.com/example.ext</url>
      <url type="http" location="us" preference="90">http://www.example1.com/example.ext</url> 
      <url type="http" location="uk" preference="90">http://www.example2.com/example.ext</url>
      <url type="http" location="de" preference="90">http://www.example3.com/example.ext</url> 
      <url type="bittorrent" preference="100">http://www.ex.com/example.ext.torrent</url>
      <url type="magnet"/>
      <url type="ed2k"/>
    </resources>
    </file>
  </files>
</metalink>

Future Use

Possible areas of future investigation for OpenOffice.org:

• More testing on download pages.
• "Check for Updates"/Autoupdate feature in OpenOffice.org 2.0.4+.
• Integration with Bouncer and a Firefox extension like DownThemAll.

Blog descriptions

TorrentFreak explains Metalink like this: "We’ve all seen those download pages where a couple of http, ftp and bittorrent mirrors are listed. We often just pick one of these, but wouldn’t it be great if you could use them all at the same time? Metalink has the solution. It works like this, the .metalink file tells your download manager where it can get the file, and connects to these different sources at the same time."

downloadsquad describes it this way: "Metalinks make complex download pages obsolete by replacing long lists of download mirrors and BitTorrent trackers with a single .metalink file. As you might have already guessed, a .metalink file is a file that tells a download manager all the different ways it can download a file. The file itself takes the form of an open XML standard that can list an unlimited number of HTTP and FTP sources as well as BitTorrent trackers and ed2k and magnet links."

External links

• Metalink homepage
• Metalink @ Packages Resources : Automatically generated Metalinks for Open Source software along with source code for the Metalink generator.
• Simba (GPL), the mirroring tool.
• OpenOffice.org P2P page