The MakeUseOf Guide to File Sharing Networks

Have you ever wondered what are the biggest file sharing networks out there? What are the differences between BitTorrent, Gnutella, eDonkey, Usenet etc.?

Want to know which file sharing network gives you best download speed or highest level of privacy?

The guide explains the pros and cons for each. We also look at privacy issues and share some excellent tools helping you to protect your privacy while downloading files on these networks.

§1–Introduction

§2–File Sharing and Copyright

§9–Other File Sharing Networks

§10–Privacy

§11–Conclusion

1. Introduction

We start off another eBook, another great journey I hope you’ll enjoy. This time we’re going to take a look at file sharing, the different software, protocols, problems and pretty much else you can think of. Why write an eBook about file sharing? Well, although things got a lot simpler since the days of Napster and ‘sneakernet’, and a lot of faster, there’s quite a few roadblocks especially for non-technical computer users.

In this eBook, we’re going to take a look at the major file sharing technologies available and debate each one’s features and caveats, along with the best ways to make use of them. But before we get into the breadth of the subject, let’s take a moment to discuss a very important issue that is inevitably going to arise due to the nature of this guide.

Copyright, especially intellectual copyright, the type that covers any kind of text, photo, video, software or work of art is something to be treated with respect. Although some parties, like the infamous torrent tracker called ‘The Pirate Bay’, would like (you) to completely ignore copyright, it’s neither legal nor moral to do so. There have been a few grassroots campaigns that promoted the ‘fact’ that downloading copyrighted work without authorization is actually all right, and cannot be compared to stealing, because of the nature of the medium in which it is stored – electronically. Their argument was quite easy to understand and it got some traction because of this. When downloading music or books online, there’s no cost to the publisher to print the book, manufacture the CD, pay for transport or clerks and salespeople. The only cost is the power and Internet bill that you pay yourself. Therefore why should we pay for content?

Creative Commons Attribution. Photo by Daveybot via Flickr. Purchased music arranged in an album art wall.

I hope that didn’t convince you too, because it’s completely wrong. They are ignoring the part when the musician, writer or programmer spent hours and hours, training and working in order to create that material. That’s a pretty big mistake if you ask me. Put yourself in that man’s shoes. Wouldn’t you want to be able to provide for yourself, to receive compensation for your work? While I certainly can’t condone the actions of RIAA, IFPI and similar anti-piracy organizations, which quite frankly strike as almost childish in their approach, I cannot promote stealing other people’s work. Think about what Uncle Ben said to Peter in Spider-Man: “With great power comes great responsibility”.

Creative Commons Attribution by everyskyline via Flickr.

That being said, there are many legal uses of File Sharing, such as the distribution of open source software, public domain, creative commons and copyleft works. You should also realize that the technology behind many P2P applications and services isn’t inherently bad. Bit Torrent could potentially speed up many legitimate downloads, including your iTunes and Amazon purchases if CDNs (Content Delivery Network) would implement it.

Disclaimer

MakeUseOf or its writers cannot be held liable for any actions that might undertake as a result of following the instructions presented here. Copyright infringement is a felony in the United States of America as well as many other countries. Please read your local legislation on copyright before attempting any of the actions described in this eBook. This information presented here is solely for educational purposes.

3. What’s P2P?

P2P (Peer-to-Peer) is an acronym that stands for a group of technologies, which use the same principle to function. Instead of using a central server system that feeds data to the other client computers, in the P2P architecture, every computer is both a client and a server. Each computer will try to download from any of the computers that have the file or in return will itself share the data it has to other computers in the ‘swarm’. A swarm is defined as the totality of computers that are downloading or uploading the same content.

Creative Commons Attribution by RocketRaccoon via Flickr.

This approach to distributing content is efficient for dispensing large amounts of data to a large number of computers, almost simultaneously. Whereas by increasing the number of users connected to a centralized download server would inherently reduce overall speed and eventually bring it to a halt, in a P2P architecture, the overall speed increases as more computers complete the download and in return share it with others, multiplying the number of available sources exponentially.

Bit Torrent, alone, is believed to drive 27 to 55% of the total Internet traffic depending on the region.

Organizations acting on behalf of the music and movie industry, namely RIAA and MPAA, have managed to shut down many P2P services during the years and have even proceeded with litigation against individuals.

4. Bit Torrent

Bit Torrent, a technology developed by Bram Cohen in 2001, is by far the most widely used P2P technology. It has proved to be superior in many ways; allowing for a semi-decentralized environment, redundancy and significant gains in speed compared with

4.1 How does it work?

To use the Bit Torrent protocol to download and upload files, a user must download and install an application that implements the technology. There are many options available, and we’ll discuss which one is best for each platform, shortly. For now, let’s focus on how it works.

Creative Commons Attribution by nrkbeta via Flickr.

In order to share files with other peers, a Bit Torrent application must first gather information about the respective files and stored it into a file. Often referred to as ‘metadata’, the ‘.torrent’ file that is created will contain information such as the size, name, type of the file(s) to be shared. Depending on how many files there are, the metadata file ranges from 1KB to 200KB for multi-gigabyte containers.

To initiate a download, you must first use a web site that aggregates these metadata files, such as The Pirate Bay or ISOHunt. (They are reviewed in more detail below under 'Torrent Trackers’). In simple words, they are search engines where you can search for what you want to download and find the relevant torrent file. Then you may download this torrent file to your computer and open it in the torrent client to start the download process.

An initial peer, called the ‘uploader’, will initially seed the torrent tracker with the metadata file and wait for other peers to connect to it and download the file.

After the transfer reaches 100%, your role changes from ‘leecher’, or downloader, to ‘seeder’, or uploader. The ‘ratio’ number represents how much of the download throughput you gave back to the community by dividing the total downloaded size to the total uploaded size. It is considered good practice in the Bit Torrent community to maintain a ratio exceeding 0.7 at all times. The community disregards peers with lower ratios as ‘leechers’ and many private torrent trackers impose limits to their accounts.

4.2 Technical notes

Bit Torrent downloads differ in many ways from traditional ftp or http based requests. For one, the download are not sequential, the client usually requesting the rarest bits first to ensure availability or otherwise downloading at random depending on the availability and speed of each peer.

Many modern Bit Torrent clients include functionality that reduces the reliance on a centralized tracker. Although many, relatively redundant trackers are available on the net and it is highly unlikely that they will all cease to function at one time, PEX (Peer Exchange) and DHT (Distributed Hash Tables) can serve as a backup if the tracker performs slowly or is temporarily out of service. While a tracker-less environment will perform well (or faster) in a small group or local network, global reliance on PEX suffers from slower propagation rates.

4.3 Torrent Client Applications

These are the applications that can read the metadata and proceed with downloading the actual files you requested.

There are many Bit Torrent applications available and choosing the best one might not be obvious even to the technical audience. We judge an application by many characteristics such as size, features, speed, reliability, security and user interface.

Windows users will most likely end up using uTorrent, a freeware closed source torrent client created by Bit Torrent, Inc. It is best known for its large feature set, simple yet rich user interface, stability and not least, for having one of the smallest resource footprint (<300KB executable). It is considered to be the most popular Bit Torrent client and has consistently received good press from tech media.

Linux and Mac users will certainly enjoy using Transmission, an open source client also recognized for its usability and small resource footprint. Jackson wrote an article that compares Transmission and uTorrent. Wez Pyke, one of the MakeUseOf writers has an in-depth article about it.

5. Torrent Trackers

Using popular, well-regarded torrent trackers is always a good thing. Many people are going to be downloading and uploading, creating a healthy swarm – which equals a fast download speed and a wide range of available files.

A standard ‘The Pirate Bay’ style response to copyright lawyers. Copyleft.

My all-time favorites are:

• The Pirate Bay, with a very large pool of peers and files, their uncanny humor addressing threats from copyright holders and a well laid out interface, it’s really hard not to love them. Not sure how much time it’s going to be available though, considering the creators are in jail and the property is sold to some media company.

• ISOHunt has been around for quite a long time, but it hasn’t disappointed its users yet. The tracker deserves a solid second place in my bookmarks.

• Coda.fm, with its awesome UI and integrated reviews, is the best way to get your music from the torrents.

• If you’d rather not be on the edge of your seat each time someone knocks on your door – check out LegalTorrents, for – you’ve guessed it – stuff that isn’t protected by copyright. There’s also LegitTorrents, and Vuze.

• TorrentScan does a great job at searching all the major public trackers for content. If you don’t have a certain preference for a tracker or don’t mind the extra jump to the actual site – it’s worth a bookmark. You can peek at the list on the right sidebar to get some other good recommendations for public trackers.

MakeUseOf author Saikat Basu has written a comprehensive Beginner’s guide to Bit Torrent. I strongly recommend that you read through his guide if you’re new to this space. There is a section with excellent tips for advanced users as well.

We also have a continually updating page for articles related to torrents where you can find tips, reviews of clients and trackers.

5.1 Scorecard for Bit Torrent:

• Speed: 5/5

• File Availability: Excellent • Privacy: 1/5

• Malware Danger: Fair

• Ease of Use: 4/5.

5.2 Why Bit Torrent?

Today, Bit Torrent represents the way to get content from the Internet to many people, including me. I tried numerous file-sharing services and none had the speed, sheer number of files and were as reliable as Bit Torrent. I remember downloading 700 MB disc images (.iso) just to find out that they had passwords or were fake, peers disconnecting at 99% as well as other problems. Many of these problems were solved with the advent of Bit Torrent, and I personally wouldn’t switch back to any of the other services.

But Bit Torrent isn’t panacea. Let’s check out some of the other protocols available.

6. eDonkey

Don’t be fooled by the name, the eDonkey network had the lion share of file sharing users for quite a while. While it is a decentralized network, it still relies on servers to handle the search queries, which contain the indexes of shared files. The original creator had to settle with the RIAA for about $30 million, but the community, millions strong, continued development of the client and server software and kept the network running.

eDonkey is different from Bit Torrent in many ways. To touch just the surface, it doesn’t require tracker websites and uses another protocol for finding and downloading files, which is in most cases, slower than Bit Torrent.

The best eDonkey client for Windows appears to be the open-source Shareaza. It successfully handles Gnutella, Gnutella2, eDonkey, BitTorrent, magnet, ed2k and Piolet links. The interface is well laid out, the application was stable during testing and I successfully used each of the protocols supported to download files.

Mac and Linux users will most likely use aMule, a solid client for eDonkey and Kad networks.

The eDonkey network is large enough to contain a critical mass of files and peers, and can be particularly fast on a metropolitan network. Unfortunately, it’s prone to fake files, viruses and tracking.

Each download client (I.e. Shareaza) has a slightly different interface but the general idea is to find the ‘Network’ or ‘Hub’ button and refresh the list of available servers that you can connect to. Read the description, location and choose what you think is more likely to provide better speed and range of files. It is possible to have more than one connection active at a time, but you shouldn’t have more than 5 open because it will slow the search.

Availability of files and speed depends on time of the day, location and other factors. The eDonkey network is almost always slower and has less files than Bit Torrent and there’s a high likelihood you’re going to get a virus, as there is no rating or commenting system.

6.1 Scorecard for eDonkey:

• Speed: 3/5

• File Availability: Fair

• Privacy: 2/5

• Malware Danger: Severe

• Ease of Use: 3/5

7. Gnutella

2007’s most popular file sharing network is still live and kicking. After the 2001 surge in popularity, mostly due to Napster’s demise, the protocol and clients started getting better and better, and still get an honorable second place to Bit Torrent. While the developers envisioned a completely decentralized system, Gnutella, much like eDonkey, it needs caches, ultra-peers and ‘leaves’ to remain stable.

The most popular download client for Gnutella network is the notorious LimeWire, which is written in Java and therefore cross-platform. I would recommend it to Windows and Linux users. Mac folks should at least have a look at Acquisition, which is written in native code and retains the UI style of Mac applications.

Having used both Gnutella and eDonkey, I would lean over to the first. The download clients are a bit more polished, the network appeared more responsive and lastly, you don’t have to manually pick and choose servers like on eDonkey.

Another advantage to LimeWire is the filtering capability for your searches; eliminate executables, documents, photos or copyrighted content – your choice.

LimeWire is so easy to use you could tell your dog to download the files for you. Open the application and enter your search terms, then click on the download button. There’s even a first time user wizard that will help you if you get lost.

7.1 Scorecard for Gnutella:

• Speed: 4/5

• File Availability: Good

• Privacy: 3/5

• Malware Danger: Severe

• Ease of Use: 5/5

8. Usenet

In many ways the child of the old bulletin board systems, USENET (user network) users share articles, photos, music and binaries via a semi-centralized mesh of servers. Also called newsgroups, USENET articles work very similar to email. Once connected to a newsgroup, let’s say sci.math, in your newsreader, you can post and reply to messages other users have posted. The messages are threaded, just like your Gmail inbox. Most USENET servers do not require any personal information or registration or keep logs.

USENET’s importance has declined over the years as a medium to post, disseminate information and engage in conversations with other people mostly because of forums and blogs. While it is uncontested that the user base is getting smaller and smaller every year, there are newsgroups dedicated to posting applications, videos and music, or binaries.

Unlike other P2P networks, USENET provides its users with good privacy by default. The nature of the protocol, similar to SMTP (Simple Mail Transfer Protocol), imposes a trust chain. The sender’s (or uploader’s) identity can be easily obfuscated by simply altering the message header. The server is the only machine that knows the address of the user and many newsgroups do not keep logs; those who do are not likely to divulge the information.

Another argument in favor of USENET is that the servers operate under the DMCA Safe Harbor regulation as a telecommunications provider; if they provide a way to remove copyrighted information to the owners, they are not liable in court. The architecture of the mesh network and the policies of many server administrators make completely removing the content nearly impossible, as the content propagates to many servers in the mesh and they generally ignore cancel messages.

An example of a mesh network. Creative Commons by Marc_Smith via Flickr.

There are a few limitations though. Most free newsgroups do not keep the data for long periods of time due to the limited storage capacity of the server. For pay newsgroups will have a longer data retention, a year at maximum. Speed is another issue; newsgroups can be significantly slower than other P2P technologies during peak hours (afternoon, evening) because of their reliance on servers. Uploading and downloading from USENET is cumbersome when compared to other technologies. Files must be broken down to smaller pieces (<50MB), compressed (.RAR archives), encoded (with .PAR recovery files), attached with file information (.NFO) and uploaded. The data needs to be reconstructed after it is downloaded.

Unfortunately there are no free USENET applications designed for sharing binaries. All I can do is recommend the best of the for-pay ones. On Macs, Unison wins hands down, for Linux there’s BinBot and on Windows, NewsLeecher is the best choice.

8.1 Scorecard for Usenet:

• Speed: 2/5

• File Availability: Inconsistent

• Privacy: 4/5

• Malware Danger: Negligible

• Ease of Use: 1/5

There are other services that are abused by people uploading copyrighted files. One of such services is Rapidshare. They’ve been either, shut down and people who used them tracked down: “In April 2009 Rapidshare handed over the personal details of alleged copyright infringers to several major record labels.” Do yourself a favor and don’t use them.

For Rapidshare you don't need any download client, you basically need the to know the location of the file before you can download it. There are several search engines that help you find the location of the files you're looking for.

Smaller networks, spinoffs of larger networks and stuff that’s advertised to be ‘the fastest ever’ on shoddy websites are another big mistake.

10. Privacy

Your ISP, the peers you are downloading from, the tracker and even a neighbor using a network sniffer know what you are doing. Just so we are clear, there is no privacy when using the P2P services above, You’re playing a probabilities game: out of the millions of file-sharers, will I be one of the thousands to get sued?

10.1 How can you protect your privacy?

• ItsHidden Beta, a free VPN service that was created to serve this exact purpose, promises not to keep logs, provides a decent speed and is currently free.

• IPREDATOR, a similar service to ItsHidden, is an initiative started by the Pirate Bay guys. The beta is currently closed, but you might still get an invitation if you sign up.

• Other paid VPN services like Witopia.net, do keep minimal logs, mainly to prevent abuse (hacking, spamming, etc.). Of course, the lawyer-approved version of the TOS clearly states that copyright infringement is not okay.

• OneSwarm, while still a backwards-compatible Bit Torrent client, it implements the concept of friends and peer address obfuscation. Check out Simon’s review of it.

• BitBlinder, is the most promising service for online privacy I’ve seen so far. Unfortunately, it’s in private beta right now, but you should definitely read up on the specifications and subscribe to their blog.

• Private trackers, while they have strictly moderated content and generally less leeching problems, aren’t the solution for privacy conscious people. If you can get an invite for the service, believe me the RIAA can too. They don’t even have to subpoena the tracker, just connect to the same torrent as you.

• IP filtering software, namely ProtoWall and PeerGuardian (discontinued), will try to block connections from known IP addresses from Media Sentry, MPAA and the likes. They are not 100% infallible, but they should help. Linux users should look into MoBlock

11. Conclusion

We’ve looked at the most popular file sharing networks. We’ve looked at the best applications for each. Which one should you use? That’s a tough question, but Bit Torrent seems to be the most versatile of them all.

If you’re interested in sending files occasionally to your friends or family, and especially if they’re large, then you should definitely read Tim Watson’s article on 4 dead simple web sites that solve that problem. Or maybe step on the techie side and set up your own encrypted VPN and use the integrated tools in your OS; Dave Dragger’s article shows you how to use Hamachi to do just that.

“There is no delight in owning anything unshared.” –Seneca, roman Philosopher

MakeUseOf

The MakeUseOf Guide to File Sharing Networks

Table of Contents