Jump to content

[ o.k. ] Runtime [ 835 min ] Time for upgrade?

Igor

By Igor
in Advanced users - Development

 Share

Recommended Posts

tkaiser

tkaiser

Posted
Posted
  On 9/13/2016 at 1:08 PM, zador.blood.stained said:

Sending compression to background is a good idea, but I would prefer to refactor this first. In addition I think you can't nice bash functions, you need to nice either zip/7za process or do some magic with "declare -f"

 

Ok, I also made a little mistake regarding armbian.txt (would've been missing). Anyway, it should work (now) as intended: http://pastebin.com/nVgBPj1V

 

 

  Reveal hidden contents

 

Igor

Igor

Posted
  • Author
Posted

1. Clean build of Cubietruck desktop Jessie default with EXTERNAL="yes" and EXTERNAL_NEW="compile" . OK

2. The same once again:
------------------------------------------------------

[ o.k. ] Using user configuration override [ userpatches/lib.config ]                                                                      
[ o.k. ] Starting Armbian build script [ @host ]
[ o.k. ] Downloading sources 
[ o.k. ] Checking git sources [ u-boot v2016.09 ]
[ .... ] Up to date 
[ o.k. ] Checking git sources [ linux-sunxi sunxi-3.4 ]
[ .... ] Up to date 
[ o.k. ] Checking git sources [ sunxi-display-changer master ]
[ .... ] Up to date 
[ o.k. ] Checking git sources [ sunxi-tools master ]
[ .... ] Up to date 
[ o.k. ] Creating board support package [ cubietruck default ]
[ .... ] Fingerprinting 
[ o.k. ] Building package [ linux-jessie-root-cubietruck ]
[ o.k. ] Starting package building process [ jessie armhf ]
[ o.k. ] Packages are up to date [ libDRI2 jessie armhf ]
[ o.k. ] Packages are up to date [ libvdpau jessie armhf ]
[ o.k. ] Packages are up to date [ libUMP jessie armhf ]
[ o.k. ] Packages are up to date [ libcedrus jessie armhf ]
[ o.k. ] Packages are up to date [ xf86-video-fbturbo jessie armhf ]
[ o.k. ] Packages are up to date [ libass jessie armhf ]
[ o.k. ] Packages are up to date [ FFmpeg jessie armhf ]
[ o.k. ] Packages are up to date [ fswebcam-gc2035 jessie armhf ]
[ o.k. ] Packages are up to date [ guvcview jessie armhf ]
[ o.k. ] Packages are up to date [ hostapd jessie armhf ]
[ o.k. ] Packages are up to date [ hostapd-realtek jessie armhf ]
[ o.k. ] Packages are up to date [ libglshim jessie armhf ]
[ o.k. ] Packages are up to date [ libmali-sunxi-r3p0 jessie armhf ]
[ o.k. ] Packages are up to date [ libvdpau-sunxi jessie armhf ]
[ o.k. ] Packages are up to date [ mpv jessie armhf ]
[ o.k. ] Packages are up to date [ sunxi-tools jessie armhf ]
[ o.k. ] Packages are up to date [ swconfig jessie armhf ]
[ o.k. ] Starting package building process [ jessie arm64 ]
[ .... ] Skipping building libDRI2 for [ jessie arm64 ]
[ .... ] Skipping building libvdpau for [ jessie arm64 ]
[ .... ] Skipping building libUMP for [ jessie arm64 ]
[ .... ] Skipping building libcedrus for [ jessie arm64 ]
[ .... ] Skipping building xf86-video-fbturbo for [ jessie arm64 ]
[ .... ] Skipping building libass for [ jessie arm64 ]
[ .... ] Skipping building FFmpeg for [ jessie arm64 ]
[ .... ] Skipping building fswebcam-gc2035 for [ jessie arm64 ]
[ .... ] Skipping building guvcview for [ jessie arm64 ]
[ o.k. ] Packages are up to date [ hostapd jessie arm64 ]
[ o.k. ] Packages are up to date [ hostapd-realtek jessie arm64 ]
[ .... ] Skipping building libglshim for [ jessie arm64 ]
[ .... ] Skipping building libmali-sunxi-r3p0 for [ jessie arm64 ]
[ .... ] Skipping building libvdpau-sunxi for [ jessie arm64 ]
[ .... ] Skipping building mpv for [ jessie arm64 ]
[ o.k. ] Packages are up to date [ sunxi-tools jessie arm64 ]
[ .... ] Skipping building swconfig for [ jessie arm64 ]
[ o.k. ] Starting package building process [ xenial armhf ]
[ .... ] Skipping building libDRI2 for [ xenial armhf ]
[ o.k. ] Building packages [ libvdpau xenial armhf ]
[ o.k. ] Checking git sources [ extra/libvdpau master ]
[ .... ] Up to date 

,,, this ends up in error.

zador.blood.stained

zador.blood.stained

Posted
Posted

Did you delete all existing packages before building (I mean from $DEST/debs/extra)? If you didn't, then this error is due to build dependencies missing in chroot, this should fix itself if you start with no prebuilt packages.

Igor

Igor

Posted
  • Author
Posted

I deleted all deb files. I can repeat to be  I am 100% sure ... it's working only 80 minutes here :)

zador.blood.stained

zador.blood.stained

Posted
Posted
  On 9/13/2016 at 2:45 PM, Igor said:

Do I need to remove buildpkg and cache subdirs too?

No. Then what is the exact output and error message?

 

 

  Reveal hidden contents

 

Igor

Igor

Posted
  • Author
Posted (edited)

logs:

 

 

  Reveal hidden contents

 

Edited by goldfish_paris
spoiler
Igor

Igor

Posted
  • Author
Posted 
./compile.sh EXTERNAL_NEW=compile CLEAN_LEVEL="" KERNEL_ONLY=yes BOARD=orangepipc BRANCH=default

=

wildcat_paris

wildcat_paris

Posted
Posted

zador.blood.stain has 1000 posts here :)

 

(tuning) probably already written here

 

ordering the mass compilation (not by board) but by SoC to reduce kernel compilation

 

then ccache (pick as you need):

- move ccache folder to ramdisk

- tune to add one level in ccache b-tree (2=>3)

- compress lightly the ccache results (there is an overhead)

- or ccache hardlinks (so there is no files moved from ccache to the kernel tree) not compatible with ramdisk or compression

 

maybe deleting ccache folder when ccache is bloated of hashes (there is a tweak: limit the number of files / ccache directory size / change the ratio of ccache replacement)

OR creating a ccache folder per SoC (and removing the full directory from ccache hash) so less likely to overbloat the ccache tree

(note: ccache per SoC could be an alternative of ordering compilation per SoC)

 

just the goldfish_paris passing by :)

https://github.com/igorpecovnik/lib/pull/466

zador.blood.stained

zador.blood.stained

Posted
Posted
  On 9/13/2016 at 3:26 PM, goldfish_paris said:

ordering the mass compilation (not by board) but by SoC to reduce kernel compilation

Build all kernels first (also keeps ccache in "warm" state), then build all images with kernels already built and packaged. Problem solved.

 

  On 9/13/2016 at 3:26 PM, goldfish_paris said:

- move ccache folder to ramdisk

ccache on SSD and in disk cache after few compilations is fast enough to not bother

 

  On 9/13/2016 at 3:26 PM, goldfish_paris said:

- tune to add one level in ccache b-tree (2=>3)

- compress lightly the ccache results (there is an overhead)

- or ccache hardlinks (so there is no files moved from ccache to the kernel tree) not compatible with ramdisk or compression

results may depend on number of files in cache, filesystem, "warm"/"cold" state and type of storage (HDD/SSD). Should be tuned individually.

 

  On 9/13/2016 at 3:26 PM, goldfish_paris said:

maybe deleting ccache folder when ccache is bloated of hashes (there is a tweak: limit the number of files / ccache directory size / change the ratio of ccache replacement)

OR creating a ccache folder per SoC (and removing the full directory from ccache hash) so less likely to overbloat the ccache tree

(note: ccache per SoC could be an alternative of ordering compilation per SoC)

:blink:

wildcat_paris

wildcat_paris

Posted
Posted
  On 9/13/2016 at 3:55 PM, zador.blood.stained said:
creating a ccache folder per SoC (and removing the full directory from ccache hash) so less likely to overbloat the ccache tree

(note: ccache per SoC could be an alternative of ordering compilation per SoC)

 

:blink:

 

b-tree concept is great but overload a b-tree and the performance will degrade (if not, increase the depth of the btree may help).

ccache per SoC is an alternative to avoid bloating the ccache btree. It is probably durable over several "full" creation of images for all the boards.

 

Well, the config is simple, it is only a matter of changing/exporting CCACHE_DIR

with CCACHE_DIR=$DEST/ccache/per_$BRANCH_$LINUXFAMILY

Igor

Igor

Posted
  • Author
Posted

After deleting

/output/buildpkg

... and recompiling problem looks to be gone. Doing more tests ...

Igor

Igor

Posted
  • Author
Posted

Did we forgot to include this into patch? 

CONFIG_MMC_SUNXI_SLOT_EXTRA=2
U-Boot SPL 2016.09-armbian (Sep 14 2016 - 07:34:12)
DRAM: 1024 MiB
Trying to boot from MMC2
MMC Device 1 not found
spl: could not find mmc device. error: -19
SPL: failed to boot from all boot devices
### ERROR ### Please RESET the board ###

BTW: shall we rename .RAW to .ISO ?

zador.blood.stained

zador.blood.stained

Posted
Posted
  On 9/14/2016 at 11:12 AM, Igor said:

Did we forgot to include this into patch? 

CONFIG_MMC_SUNXI_SLOT_EXTRA=2

Yes, I forgot to patch config files for Plus and Plus 2. Others should work fine.

 

 

  On 9/14/2016 at 11:12 AM, Igor said:

BTW: shall we rename .RAW to .ISO ?

Definitely not to .iso (too confusing), but to .img, since this is the default extension expected by Win32diskimager and Rufus.

Igor

Igor

Posted
  • Author
Posted

Repository is updated to latest version (jumped to 5.20) so it's possible to test upgrades.

1
svkatielee

svkatielee

Posted
Posted
  On 9/15/2016 at 7:48 PM, Igor said:

Repository is updated to latest version (jumped to 5.20) so it's possible to test upgrades.

Do I understand this correct?

The 5.20 reflects the patch level of the kernel, not necessarily anything about the "Armbian distribution".

Such as when download images will have NM instead of wicd.

Or maybe you meant:  the kernel that will be in "Armbian distribution release 5.20" is now available.

 

I just did the upgrade:

The following packages will be upgraded:
  linux-firmware-image-sun8i linux-headers-sun8i linux-image-sun8i linux-jessie-root-orangepiplus2e linux-u-boot-orangepiplus2e-default

and the apt-get dist-upgrade says nothing to do.

 

This is:

 cat /etc/armbian-release
# PLEASE DO NOT EDIT THIS FILE
BOARD=orangepiplus2e
BOARD_NAME="Orange Pi+ 2E"
VERSION=5.20
LINUXFAMILY=sun8i
BRANCH=default
ARCH=arm

Boot and login seem faster, but was already nice on the 2E installed to th eeMMC.

Thank you.

Larry

 

zador.blood.stained

zador.blood.stained

Posted
Posted

5.20 is the version for new Armbian packages and prebuilt images.

You can still upgrade u-boot and kernel with apt-get, but to get new desktop features you'll have to either install packages and apply the tweaks from our build system manually, or start with new prebuilt image.

svkatielee

svkatielee

Posted
Posted

Thanks zador. When I asked the question the kernel was in the repo, but none of the images were updated on the download page so I jumped to the wrong conclusion.

 

Just being impatient. haha

Igor

Igor

Posted
  • Author
Posted

OK. This would help pack kernel faster but if not possible, not that critical since this is not done that many times.

 

So no much other alternatives except adjusting "build-all" to work simultaneously.

lanefu

lanefu

Posted
Posted

how often do you want to perform builds for all boards?

 

Tapatalk thinks its important to tell you im using tapatalk from a phone.

Igor

Igor

Posted
  • Author
Posted

I am running daily: all packages recompilation / testing repository update and build few images for one week now. It's working fine, but is still considered experimental. 

Igor

Igor

Posted
  • Author
Posted

Now we are running this daily rebuild for some time and if everything goes fine, I don't mind that building is not optimized. When it breaks, like today, build time becomes important but anyway I simply want to make use of available hardware at full degree ;)

 

Ideas for better CPU utilization:

 

- kernel compilation is optimized, while packaging is not. I would need to do more tests if running more kernel compilation in parallel gives any significant benefit.

- 57 packages are building more or less on single core

- build all, BSP = 376 * single core :)

 

There are some ideas in the thread - but which way seems less complicated to implement?

Guest
This topic is now closed to further replies.
 Share
Go to topic listing
×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines

  I accept