Infrastructure Seed our torrents

[The]

1007M swap. I don't know... This is from dmesg:

[   24.553452] Adding 1030140k swap on /var/swap.  Priority:-2 extents:5 across:3536892k SSFS

 

20 minutes ago, Igor said:

This is my torrent server on the same 8Gb SD card.

/dev/mmcblk0p1    7458672    1058648   6297556  15% /

/dev/sdb1      7752457508 6878909040 482830824  94% /ext

 

I saw with lsblk, that sda1 has no mountpoint... that could be the reason. I will check it out.

[The]

I found the error. It was the mount point. Everything's working fine now for almost 7 weeks.

[MacBreaker]

Hello Igor,

since years i seed Armbian torrents but i noticed that i'm running out of space.
My hdd was 248GB till now...

 

Looking for further information and saw in the first post is written 512GB.

Maybe you can add a big ">NOTE!< since Version 20.05.xx you need 512GB free space".

 

Also after you installed transmission by ->Software ->Softy ->Transmission you get asked to seed Armbian torrents.

Here a screen:

 

 

You have to change the number from 80GB to 512GB.

 

Just my thinking about this..

 

Markus

[Werner]

5 minutes ago, MacBreaker said:

Hello Igor,

since years i seed Armbian torrents but i noticed that i'm running out of space.
My hdd was 248GB till now...

 

Looking for further information and saw in the first post is written 512GB.

Maybe you can add a big ">NOTE!< since Version 20.05.xx you need 512GB free space".

 

Also after you installed transmission by ->Software ->Softy ->Transmission you get asked to seed Armbian torrents.

Here a screen:

 

 

You have to change the number from 80GB to 512GB.

 

Just my thinking about this..

 

Markus

Yeah that number is a bit outdated. But you can see how much the project has grown over the years.

[MacBreaker]

Quote

Yeah that number is a bit outdated. But you can see how much the project has grown over the years.

 

Yes, and you all can be proud of this...

[Igor]

512Gb is "a bit" future proof. Current real number is 259G and can cleaned up to fit 256GB drive size ... but its a manual work.

[MacBreaker]

OK, i'm prepaired with my 500GB HDD on a Cubietruck right now...

[Werner]

The current torrent size is 283,x GB atm for those who are interested: https://werner.armbian.de/_offtopic/torrent_statistics.txt

[usb-driver]

Hello there,

I'm trying to adapt the cron script to deluged and I came out with this

#!/bin/bash
#
# armbian torrents auto update
#
# download latest torrent pack
TEMP_DIR=$(mktemp -d || exit 1)
chmod 700 ${TEMP_DIR}
trap "rm -rf \"${TEMP_DIR}\" ; exit 0" 0 1 2 3 15
wget -qO- -O ${TEMP_DIR}/armbian-torrents.zip https://dl.armbian.com/torrent/all-torrents.zip
# test zip for corruption
unzip -t ${TEMP_DIR}/armbian-torrents.zip >/dev/null 2>&1
[[ $? -ne 0 ]] && echo "Error in zip" && exit
# extract zip
unzip -o ${TEMP_DIR}/armbian-torrents.zip -d ${TEMP_DIR}/torrent-tmp >/dev/null 2>&1
# create list of current active torrents
deluge-console 'info' | sed '1d; $d' > ${TEMP_DIR}/torrent-tmp/active.torrents
# loop and add/update torrent files
for f in ${TEMP_DIR}/torrent-tmp/*.torrent; do
        deluge-console "add ${f}" > /dev/null 2>&1
        # remove added from the list
        pattern="${f//.torrent}"; pattern="${pattern##*/}";
        sed -i "/$pattern/d" ${TEMP_DIR}/torrent-tmp/active.torrents
done
# remove old armbian torrents
while read i; do
        [[ $i == *Armbian_* || $i == *gcc-linaro-* || $i == *tar.lz4 ]] && 
        deluge-console "rm $(echo "$i" | awk '{print $1}';) --remove_data"
done < ${TEMP_DIR}/torrent-tmp/active.torrents

I'm not sure with the "active.torrents" part because i don't have transmission and i don't know how the informations from the "-l" are listed.

For now the adding part is working and the "active.torrents" file is written! 

[Werner]

There was an issue with the torrent network which lead into adding torrents to the queue that were no longer existing. This has been fixed. If you have setup your transmission-daemon with the update script mentioned at the first post you will receive the fix automatically or already might have.

[amontero]

Is there any way of seeding only some files? I would love to contribute some bw, but I'm a bit short on disk space.

Thanks.

[Igor]

3 hours ago, amontero said:

Is there any way of seeding only some files? I would love to contribute some bw, but I'm a bit short on disk space.

Thanks.

Files are changing at each update so this makes little sense. Current share size is 262 Gb.

[LD-Geek]

i'm very   pleasure to share  the  torrent file for armbian ,but please don't use the /var/lib/..... location,it's not friendly
use the settings.json location and automatic move to new location will be fine.

[Igor]

37 minutes ago, LD-Geek said:

use the settings.json location and automatic move to new location will be fine.

 

If this is yet another task for us, it can easily take years ... 
https://github.com/armbian/config/blob/master/debian-software#L708-L762

https://docs.armbian.com/Process_Contribute/

[Glock24]

Hello, what's the disk space needed to seed the torrents?

 

On a side note, would creating a repository mirror also help?

[djurny]

3 hours ago, Glock24 said:

Hello, what's the disk space needed to seed the torrents?

 

On a side note, would creating a repository mirror also help?

Hi,

I would also be interested in finding out what it would take to host a local repository mirror. Over the past few weeks I did try to rsync the repo to local storage, but I'm not sure if that will affect anything on the 'other' side. Had to break off the rsync as it appeared to sync extremely slow.

Groetjes,

[Werner]

3 hours ago, Glock24 said:

Hello, what's the disk space needed to seed the torrents?

As mentioned in the first post of this topic the recommended disk space is 512GB.

[Igor]

@djurny @Glock24 Stop by on our IRC channel(s), #armbian or #armbian_server to discuss details. Look for me @lanefu or @TheLinuxBug

 

5 hours ago, djurny said:

Had to break off the rsync as it appeared to sync extremely slow.

Try this one: rsync -av rsync://mirrors.dotsrc.org/armbian-dl/ . and rsync -av rsync://mirrors.dotsrc.org/armbian-apt/ .

 

[Igor]

Support project https://gitlab.com/open-transmit/transupdate

[Werner]

The size requirements have been updated.

 

Since we now share images with various desktop environments as well the size of all torrents have now grown to around 500GB.

To become future-proof again the free space requirement has been raised to 1TB.

 

For those who did not notice yet. It became rather simple to become an Armbian http/s mirror as well if you do not like torrent stuff: https://github.com/armbian/mirror

[Nuts_]

Broken seeds?

----

Some days ago I decided to join to seeding of whole Armbian torrents pack, using some of my old junk hardware.

I added cron script without problems and all wend well but i detected some very strange torrents, it lopped on downloading, geting more and more corupted data (as Transmission says)

progress bar value Increased and decreased 

i spend 1,5TB incoming traffic to upload one of this torrent with 1GB size and it still not completed

i spend some time to play with hardware, software and different internet providers

as i find some broken files can be downloaded sometimes, sometimes - not 

all this files not begining from "Armbian" (alll Arbian* is OK)

for example bullseye-minimal-amd64.e5fbf1963190f2b3a9a677d778a7e3f2.tar.lz4

i having at lest 2 different instances of this file but all of them unpucking with error

---------

So Guys, what going wrong? It seems like its not problem of my hardware, isn it?

at all, Not big help, if i downloading much more then uploading

 

[Igor]

52 minutes ago, Nuts_ said:

So Guys, what going wrong? It seems like its not problem of my hardware, isn it?

 

Its a bug in Torrent protocol when using web seeds which dramatically speeds up download. I have removed them. Restart service in about 1h that your torrent files will be refreshed. Perhaps you will need to manually remove them all before (without removing files). Some files will be downloaded again ... that is normal.

[Nuts_]

19.04.2022 в 12:20, Igor сказал:

Its a bug in Torrent protocol when using web seeds which dramatically speeds up download. 

Well, aftter update, this day downloading going well from peers, now but no so fast

Most downloaded via webseeds was OK and download speed was maximum for my ISP.

I thinking some files on different webseeds can be unsynced 

[Igor]

34 minutes ago, Nuts_ said:

I thinking some files on different webseeds can be unsynced 

I think problems started this way. Each time we generate caches, already generated ones are checked for integrity and if that fails for some reason, cache is deleted and its made again but if one program in this package is updated or if there is any other change in the rootfs, which usually is, torrent and its hash is different with the same filename. File on primary server is changed, but not (yet) all others. When you start downloading, you always get latest torrent files which has (had) weblinks. URL to files, which have the same hash but are not the same. This is not treated properly as corruption and download never finishes ... A better hash re-generation mechanism, that would render out this corner case, and we could enable this back. But since

 

34 minutes ago, Nuts_ said:

but no so fast

 

we are already full of work  and this is not that critical problem I choose to just disable web links ...