Tutorial Optimizing a torrent for best upload speed/resource usage. How to set the right piece size value.

Nae0zZ

Super moderator
Staff member
Super moderator
Uploader



Setting the right Piece Size for optimal speed/resource balance


How torrents work

Before we dive in we need to understand how the torrent system works. Torrenting is a P2P (Peer to Peer) method of sharing files with other users. The smartness and uniqueness of the torrent method is that it allows for a file to be split into pieces and the initial seed can transmit different pieces to different users and then have the users transit the pieces they have to each other and so the upload speed becomes the collective upload bandwidth of everyone in the swarm.
You can see how this works in the diagram below.
In the beginning, the Initial seed breaks the file in 3 pieces (A, B, C) and then transmits 1 piece to each client
Once the clients have a complete piece, they then start exchanging pieces with the other users.
This is why torrents start slow but speed up with more users joining. In the end, all peers will have all the pieces.

....



So piece sizes are basically a fraction of a torrent or the complete file.
Of course, the torrents we host here don't just split into 3 pieces, actually, the truth is that sometimes they split into the thousands. Piece size is the setting that can make a torrent seeded on a home connection scale well, or make even the best-seeded torrent bog down.

At its heart, the issue is, how big each piece should be. Each piece has a specific amount of the torrents data, but also data about hash/fail and tracking control. Make the pieces too few and big, and it can be very hard for a peer to get started, too many small pieces will use more of a peer's connection for overhead.

It’s a delicate balance, that is not easily found. Small pieces make it less susceptible to poisoning attacks and errors and will help a torrent deal with sudden increases in peers, by making it easy to get a piece or two to trade. However, keeping track of who has what piece requires bandwidth, and small pieces mean that you and the swarm will be telling connected peers about pieces you have just got more often.

The benefit of using smaller/larger pieces
One thing that you may not know is that each time we transmit a piece we don't only upload pure torrent data but also extra information such as Hash data and traffic/handshake data. That's why if you have a bandwidth monitor you will see different upload/download values from your monitor and from the torrent client. Most clients dont show (or rather have it out of sight) the additional bandwidth used (as well as retries), and that is why size optimization does improve performance.

Small pieces
We will consider as "small pieces" those that are less than 1MB. The benefit to small pieces is that if you lose a piece (hush errors/bad connections) it is much less cost to redownload. This also means that more people could download more pieces and the torrent will have people downloading pieces faster at the beginning since more people have them, and a new peer could get one piece from multiple people faster. The bad part is that the more peers that you are connected to the more bandwidth you are going to use because you have to connect to a lot more people and keep track of a lot more pieces. That is more so the case for the original seeder.

The bad thing about small pieces is that the more pieces you have the more resources they take up. If you have a big torrent with small pieces then think of all the resources it will take up. 10GB torrent with piece sizes of 256kb would have 40960 pieces which would take up 1280MB of ram.


Big pieces
The benefit of bigger pieces is that they use fewer resources. You should use bigger pieces (1MB and above) for bigger torrents. As you saw in the example earlier smaller pieces used on a big torrent will use up a lot of resources. Bigger pieces also mean that the torrent will last longer. If you have 1 seeder on a 10GB torrent that has been divided into 4MB, each piece will take longer to download but a seeder will be able to seed a bigger piece (256kb up compared to 4MB up). Once you have a lot of seeders big piece torrents can have extreme speeds because each peer could supply you with a big chunk of the torrent.

Sadly the big piece torrent is a double edge blade, If multiple are downloading only parts of the torrent (not complete it) then you could lose the speeds and the torrent. If errors happen during the download they will take longer to redownload. The torrent will be slower at the start. If we use too big of a size like 16mb or larger, it will actually perform worse for both your client and the swarm. We need to remember that not all peers have the same internet speeds. If we use too big of size we may get stack feeding a 16MB piece to a small peer for a really long time and that will slow piece propagation throughout the entire swarm. Remember that at the start each client gets a different piece so the slow one will slow down everyone else.

Conclusion/Short story
There is no right and wrong, no DOs or DONTs. only appropriate and adjusted.
There are some considerations for the uploaders that are based on
How many torrents they plan or already seed on their PCs for memory/CPU usage?
How "popular" those torrents now or going to be later, How many users will probably stay to help seed?
Small piece size makes torrent start faster but not peak after several peers become seeders.
Big piece size consumes less ram/CPU/bandwidth (for those that host multiple torrents)
Too many pieces are resource wasting and too few pieces are swarm/peering/speed waste!

So what do we do?
A shart guide (but not absolute) would be

50MB to 300MB: 128KB piece size
300MB to 600MB: 256KB piece size
600MB to 1.0GB: 512KB piece size
1.0GB to 2.0GB: 1MB piece size
2.0GB to 4.0GB: 2MB piece size
4.0GiB to 8.0GiB: 4MB piece size
8.0GiB to 16.0+GiB 8MB piece size


Keep in mind that those are set for Personal computer or a home NAS doing the seeding. If we use a seedbox you can have a bit higher values as most seedboxes offer good upload speeds.

Please keep in mind that the above list is not absolute and as I said many other factors should be considered.
Also as internet speeds increase, it blurs the lines so some experimentation might be in need.


Open to any questions and I hope the guide is of some help.

Edit:
After the suggestion of @skip1 (Thank you), I added those 3 pictures from the most used torrent clients so you know where to find the size option.
In general, for most clients, it should be in the torrent creation options.
Tixati

Qbittorrent

Utorrent

 

Top