How P2P (Peer to Peer) File Sharing Works

featured piratebay   How P2P (Peer to Peer) File Sharing Works There has always been a dark side to the Internet, and from the earliest days piracy was rampant. It began with message boards even before the traditional “internet” as we know it was even born, progressing to warez sites and private FTPs hosted on home computers. Finding pirate software and files used to be a slow and arduous task; it was more common to get the music or software from a friend as a physical copy (a so-called the “sneakernet”). P2P filesharing technology changed all that – but what does “peer to peer” even mean, and where did it all start?

Of course, peer to peer file sharing technology isn’t only used for piracy, but let’s be honest here: that’s the predominant usage, and that’s certainly where its roots began. There’s no need to flame off in the comments saying there are legitimate uses for it too; we know, but we’re not going to dress up the truth. Today we’ll be talking mostly about the filesharing aspect, but this certainly isn’t the only use case.

Client/Server

It’ll help to give some context on what “peer to peer” isn’t, first. The internet is traditionally what’s known as a client-server environment. Web services sit on a powerful server somewhere remote, and your computer, the client, requests information from it.

A single server can host files for hundreds of simultaneous clients, but scaling is difficult for a number of reasons.

client server   How P2P (Peer to Peer) File Sharing Works

Firstly, there’s the physical hardware requirements. This isn’t such an issue when you’re just hosting files, but if computation is required – such as when you’re hosting a dynamic website like MakeUseOf – then the CPU must work to customize those pages to individual users. Massive amounts of memory are needed, and these have a physical limit ultimately meaning more servers must be brought in to cope with demand.

Secondly, each client takes up a small slice of the connection; as a theoretical example, if the server has a 100mb/second connection, then 100 simultaneous users will only get 1mb/second maximum, each. Scale that to a 1000 users and the speed drops ten fold to 100k/second. So the more users you have, the less speed each of them are able to utilize.

Data transfer is another concern. A single 1mb file requested by 1000 people will mean you’ve used a 1gb of data transfer. When you pay per gigabyte, that can really add up; bear in mind that a single HD movie can often be around 4gb in size. Pushing huge files out to thousands of users is an expensive business.

So what is peer to peer?

Peer to peer is a different model, in which everyone becomes a server. The server role is distributed to users; instead of simply taking files, peer to peer makes it a two way street – you could now give back. In fact, giving back (known as “seeding” nowadays) is critical to the success of peer to peer networks (which is exactly why downloading without seeding – or leeching – is looked down upon as a cardinal sin in filesharing circles).

p2p network   How P2P (Peer to Peer) File Sharing Works

Unlike the client-server model in which performance degrades with more users; the peer to peer model actually works more effectively with more users in the network. The more users that make a particular file available from their hard drives, the easier it is for new users to acquire that file.

In some p2p networks, it’s faster once a certain threshold is reached; instead of taking the whole file from one user, you’re taking a smaller piece of the total file from hundreds of other users – combining connections to use your own at maximum efficiency. It should be noted that not all p2p software work this way though: BitTorrent was one of the first to aggregate connections in order to speed up the download by taking just a small part of the file from many different places simlutaneously.

Initially with p2p networks, some form of central server was needed to organize the network – to act act a database that holds information on currently connected users, files available in the system etc. Though the heavy lifting of file transfers was done directly between users, the networks were vulnerable since knocking out that central server meant disabling communications completely. This is no longer the case with recent developments; you can ask peers directly if they’ve seen a particular file. There is no way to knock out these networks; they are effectively indestructible.

History

Now you have an idea of why Peer to Peer networks were such a revolution compared to the Client/Server model, let’s take a quick look at the historical context.

Napster was the first widely available implementation of a peer to peer model in 1999. A central database held information about all the music files held by members; when you searched for a song to download, you would actually be connecting to another online user and downloading from them. In turn, once you had that song in your Naptster library, it would be available as a “source” for others on the network. You could also just add your own files, which would then be indexed and added to the database, ready to propogate across the world. The implementation was limited in that you could only download from one person though – though there was a high availability of songs, speeds were not so great.

napster   How P2P (Peer to Peer) File Sharing Works

Napster was eventually shut down in 2001, but not before similar networks had arisen that offered more than just music: movies, software, and images would now also be made available on Morpheus, Kazaa, and Gnutella networks (Limewire is probably the most famous Gnutella client).

Over the years, various protocols and peer to peer file sharing software came and went, but one open protocol has really taken hold: BitTorrent.

BitTorrent

Designed in 2001, BitTorrent is an open protocol whereby users would create a “descriptor” file (a .torrent file) containing information about the download, but not the actual download. A tracker is needed to store these descriptors, along with who currently holds that file, but it is an open protocol in the sense that anyone can make a client and anyone can host a tracker. Yes, even though it needed a central tracker, multiple trackers could exist, and any single torrent descriptor file could be registered with multiple trackers – meaning the network was incredibly robust. Knocking out one tracker wouldn’t neccessarily make a file unavialable, and another tracker can simply pop up to take it’s place.

Since then, incredible advances have been made that essentially remove the need for a central tracker. DHT – a distrubuted hash table – is one such technology that’s been implemented by BitTorrent, enabling the job of indexing files to be also be distributed amount all the users. Magnet links are another – Tim wrote all about these before, so be sure to read that for an in-depth overview of how magnets differ from traditional .torrent files.

Finding content in the first place is still a work in progress; if you have the hash ID, you can find peers with the files available – but what if you don’t have the hash yet? Clients such as Tribler have attempted to solve this, but it’s mostly client independant and not a part of the core BitTorrent protocol, so there’s certainly still room for improvement in this respect.

tribbler   How P2P (Peer to Peer) File Sharing Works

I hope this has shed some light on the meaning of peer to peer and where it began. I think it’s fair to say p2p software changed the internet and our lives forever; it is estimated that p2p software is responsible for between 40–70% of all internet traffic. The primary usage remains piracy, but there’s no reason media outlets shouldn’t embrace the protocol. The linux movement advocates using torrents to distribute large ISO images of the various OS flavours, thereby avoiding heavy hosting costs.

Did you get a chance to use Napster back in the day? Or was your first introduction to filesharing through the humble torrent? Tell us – where did your first mp3 come from?

Image Credits: ShutterStock – client server , ShutterStock – peer to peer vector

The comments were closed because the article is more than 180 days old.

If you have any questions related to what's mentioned in the article or need help with any computer issue, ask it on MakeUseOf Answers—We and our community will be more than happy to help.

18 Comments -

0 votes

Erlis Dhima

I would say greatly explained!
I studied about P2P at school, but it wasn’t explained so good!
Also love the illustrations you put on it…
If you had posted it earlier, I would have showed it at my professor when we did a presentation… Anyway, got some extra info too! ^_^

0 votes

susendeep dutta

In schools and colleges,P2P is taught as though it’s same as client server network since they don’t put much emphasis on it and they lack any awareness.

0 votes

Erlis Dhima

No, we could get the difference by then too.. But it wasn’t illustrated with such good pictures…

0 votes

susendeep dutta

I firstly got my mp3 in a form of CD as a complement in disc-man purchase from a store retailer.

0 votes

Juan Carlos Espinosa Agudelo

My mom told me she was a user of Napster and became pretty sad when it taken down.

Personally I’ve tried some clients, Bittorrent, uTorrent and Tixati and a couple of my classmates use others, such as BitComet and one of my classmates used Tribler for a short while after I told him about it.

Considering uTorrent and Bittorrent are so alike now, I can’t really say which is better, except in the details I think that uTorrent’s a bit better. Tixati’s community all say it’s better than uTorrent, but considering I don’t know the details of each very well, I can’t tell.

Vuze’s a huge packet of stuff, so it’s not really something I want if ‘all’ I want is to download something quick. I don’t know much about BitComet, except that their Wikipedia page says they had some controversy. BitSpirit’s apparently used by anime-downloaders.

Xunlei’s the most used Bittorrent-Client worldwide, since most of the users are in China. It has gotten some criticism for not being that great at the download/upload ratio. It doesn’t help the P2P community that much, just gives it’s users some downloads.

0 votes

muotechguy

I didnt know xunlei could do torrents; I thought it was typical spamware filled chinese download manager crap ;) Vuze/Azureus was my favourite for a while, but the bloat got too much (the name change was the last straw). I can’t say I’ve ever checked out the “official” bittorrent client though.

thanks for the input!

0 votes

Juan Carlos Espinosa Agudelo

On a different topic, I have one question, which just randomly popped up in my mind while reading your article:

Would it be possible for a server to work in an identical way? Like for example a game server, working on a P2P network, instead of one main server. Is that possible? And would it be useful?

0 votes

muotechguy

That’s an interesting question, but I don’t think the model is suitable for things like a game server. Games rely on low latency; something which p2p is terrible at – they can help shift large chunks of data, and fast, but getting things started is slow and the data is of a different type to that used in gaming. Playing a game simply doesn’t need a huge chunk of data from many different users.

Syncronising all that data between p2p users would be exceedingly difficult; you really do need a central server to handle it all. Even if you did have each peer sending their actions to every other peer, there would quickly be a mismatch where one peer got the news they’d been shot earlier than another, so one is thinking they’re safe, while the rest say they’ve been shot.

Things may change when a new p2p protocol is invented, but personally I dont think logically it would.

However, p2p could still be useful in gaming: like distributing a new map pack.

0 votes

Ashwin Ramesh

Very well explained! Thanks for shedding light on this topic, James :)

0 votes

GrrGrrr

Nice article James, and well explained.

On where did your first mp3 come from? – through data transfer..from someone else

0 votes

Valentin

Same here, using a Terminal port. If that even makes sense

0 votes

muotechguy

Is that the same as a null modem cable? I had one of those for 2 player duke nukem before we could buy network cards… I think that was well before the time of mp3 though.

My mp3s were inherited from a friends brother who was at uni with a fast internet pipe. He had 40 CDs full of 80′s and 90′s pop; I forget how many songs there were in total, but around 1000 or so?

0 votes

venkatp16

Nice explanation.

0 votes

MerVzter Balacuit

additional information about p2p thumbs up!

0 votes

JbstormburstADV

As a regular user of P2P torrenting, I can tell you right now that the best clients are BitTorrent and uTorrent (whiich is essentially BitTorrent but as a smaller client) because since they are associated with the creator of the protocol, they would know the most about it.

In fact, websites for private and semi-private trackers are often the best source for discovering good clients due to the presence of “whitelists,” or their list of approved clients for use with their tracker. So, let me tell you right now that you want to stay strictly the heck awau from BitComet.

As for my first mp3, likely LimeWire, since I know by the times I started DLing music, iTunes was distributing AAC almost universally.

Now, as a non sequitor, EAC. Use it. Love it. Look for burns from it.

0 votes

muotechguy

Care to elaborate on why BitComet is so bad?

0 votes

JbstormburstADV

Those rumors about BitComet messing around with the DL/UL ratio? Truer than true can be. Additionally, BitComet has some features that create problems in certain seeding situations (i.e.: initial seeds, DLing with only 1 seed and no leeches). All in all, it’s a ratio cheater’s bittorrent client.

0 votes

Tashfi Neutron

Wow! Really well written article. Really like this & I guess P2P really changed the internet :)