[testing] running Armbian tools with Docker style VM

[wildcat_paris]

Hello,

 

Interested in replacing virtualbox as a VM host, "docker" tool seems to be able to run on Debian/Ubuntu/other linux distro/MacOS/Windows (as they claim)

 

 

[2015/11/27] WORKING for COMPILING kernel, target WHEEZY or JESSIE

 

 

My test:

 

1/

Installing docker on my amd64 4-cores Ubuntu 14.04 (too much tweaking to run "armbian" properly)

see doc here also I have been using "Create a Docker group" and don't forget to run "Hello World" test

 

2/

To create a new container with Ubuntu 14.04 (if you have added docker group to your non-privileged user, if not use "sudo")

 

$ docker run --privileged -i -t ubuntu:14.04 /bin/bash

 

"privileged" is needed for /dev/loop operations, if not the container cannot see the loop device, btw it is less "secure" but not "really" applicable here, I am on my test system.

 

on the host system, the files are stored in "/var/lib/docker", you can move this folder on another HDD/SSD, then you need to (in my example) "cd /var/lib; ln -s /home4/docker docker"

 

3/

once you have the root shell of the VM

 

# mkdir /root/armbian

# cd /root/armbian

 

then feel free to follow the Armbian tool instructions

 

Once all is compiled & package, you can transfer the files with "sftp", "ftp", etc. Or if you want a challenge, grab the files in /var/lib/docker (good luck!)

 

4/

later to come back to the container, list the containers, get the "Container ID" then start the container

$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                   PORTS               NAMES
e12d3a2b9263        ubuntu:14.04        "/bin/bash"         About an hour ago   Up About an hour                             grave_ardinghelli
d1efa716ad64        hello-world         "/hello"            8 hours ago         Exited (0) 8 hours ago                       elegant_meitner

$ docker start -i e12d3a2b9263

(to be tested more carefully)

 

edit:

you need the "under work" script attached as the ubuntu:14.04 is *not* really a ubuntu server packaged style!

 

edit2:

[ o.k. ] Fixing release custom applications. [ jessie ]
[ o.k. ] Install board specific applications. [ lamobo-r1 ]
[ o.k. ] Fingerprinting. [ Armbian 4.6 Lamobo-r1 Debian jessie next Linux 4.2.6 ]
[ o.k. ] Creating boot scripts [ lamobo-r1 ]
[ o.k. ] Possible after install.
[ o.k. ] Writing boot loader [ /dev/loop1 ]
[ o.k. ] Shrink image last partition to [ minimum ]
[ o.k. ] Create and sign [ Armbian_4.6_Lamobo-r1_Debian_jessie_4.2.6.zip ]
[ o.k. ] Runtime [ 81 min ]

et voilà !

 

more testing needed

Edited November 27, 2015 by wildcat_paris

[wildcat_paris]

Apparently, cannot make it work for "trusty" as "image", I cannot figure out the issue (conflict docker guest / host system ?)

Setting up xml-core (0.13+nmu2) ...
Processing triggers for ureadahead (0.100.0-16) ...
Setting up bluez (4.101-0ubuntu13.1) ...
initctl: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused
initctl: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused
 * Starting bluetooth
Can't open RFCOMM control socket: Address family not supported by protocol
   ...done.
Setting up systemd-services (204-5ubuntu20.15) ...
Setting up libpam-systemd:armhf (204-5ubuntu20.15) ...
initctl: Unable to connect to Upstart: Failed to connect to socket /com/ubuntu/upstart: Connection refused
invoke-rc.d: unknown initscript, /etc/init.d/systemd-logind not found.
dpkg: error processing package libpam-systemd:armhf (--configure):
 subprocess installed post-installation script returned error exit status 100
Processing triggers for libc-bin (2.19-0ubuntu6) ...
Processing triggers for sgml-base (1.26+nmu4ubuntu1) ...
Processing triggers for ureadahead (0.100.0-16) ...
Errors were encountered while processing:
 libpam-systemd:armhf
E: Sub-process /usr/bin/dpkg returned an error code (1)

[Igor]

Just a quick add:

 

Some / one error(s) are normal during build and also ureadahead is useless so exclude it. Is image produced? 

[wildcat_paris]

Igor,

no it stopped the process (usually working very fine with virtualbox + ubuntu 14.04 server).

let's see later, thanks for your support :-)

[wildcat_paris]

Igor,

 

the Armbian tools is fine for Wheezy/Jessie but nope for Trusty

 

the process stops with

invoke-rc.d: unknown initscript, /etc/init.d/systemd-logind not found.
invoke-rc.d: policy-rc.d denied execution of stop.

there is a fix here: https://hub.docker.com/r/calvinpark/arduino-ide-builder-linux32/~/dockerfile/ related to https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1325142

Sorry I'm late to the party, but what about this as a workaround?

# dpkg-divert --local --add /etc/init.d/systemd-logind
# ln -s /bin/true /etc/init.d/systemd-logind

In the chroot, obviously. :-)

also same issue http://osdir.com/ml/ubuntu-bugs/2015-07/msg05371.html

 

because yes in a real VM there is no issue

but docker is not a VM, so we leaked an old bug.

 

:-)

[Igor]

Perhaps the best would be to use sysvinit in Trusty and it would work out of the box plus it's less demanding to support?

[zador.blood.stained]

Trusty is using upstart as init system, AFAIK, trying to remove all systemd related packages may break things (you'll need to check reverse dependencies of systemd related packages on actual trusty installation).

[Igor]

I thought changing init system is not that hard ... if this is not trivial, than I'll leave as is. I really don't want to mess with this if not absolutely necessary.

[lanefu]

I fought the good fight over the weekend trying to get Armbian builder to build full images while running in a docker container.    I've had some success, and managed to build a armban 5.11 image for my Orange Pi One.. I'm running on it with wifi etc.

 

I'm using a CentOS7 Docker 1.9.1 host and using Ubuntu 14.04 Trusty Docker Container for the Armbian builder container. (I just used the Dockerfile from the armbian git repo)

 

Bottom line loopback management inside containers is a bad time.. especially when losetup implementations vary between host and container.   The partprobe step in the container doesn't seem to trigger appropriate feedback to udev to create partitiion devices on the /dev loop back devices.   I got around it by creating them ahead of time.

 

Here's a few tricks to limp through.  I'll try to add more clarity later.

 

1.  Don't. just go build a ubuntu 14.04 VM or use a turn key one on amazon and get your life back.

 

 

From Docker Host

#you need modules
modprobe binfmt_misc
modprobe loop
#scrub loopbacks for good luck
rm -rf /dev/loop[0-9]*

 

 

Launch armbian builder container with --privileged=true

#sudo docker run --privileged=true --name=armbian -it armbianbuilder
#or if you're re-using container

#sudo docker start -i armbian

From container

 

#start apt-cacher-ng
service apt-cacher-ng start
​
#scrub loopbacks again in container for good luck
rm -rf /dev/loop[0-9]*

#create all your loop back devices
mknod /dev/loop0 b 7 0
mknod /dev/loop0p1 b 259 0
mknod /dev/loop0p2 b 259 1
mknod /dev/loop0p3 b 259 2
mknod /dev/loop0p4 b 259 3

mknod /dev/loop1 b 7 1
mknod /dev/loop1p1 b 259 0
mknod /dev/loop1p2 b 259 1
mknod /dev/loop1p3 b 259 2
mknod /dev/loop1p4 b 259 3
mknod /dev/loop2 b 7 2
mknod /dev/loop2p1 b 259 0
mknod /dev/loop2p2 b 259 1
mknod /dev/loop2p3 b 259 2
mknod /dev/loop2p4 b 259 3

#build your dreams
cd ~
./compile.sh

Every time you build an image you should destroy the loopbacks and recreate.  It's your best chances of getting the partitions in the loopback image to behave for mounting.

 

 

 

 

 

Edited June 30, 2016 by lanefu

[neomanic]

I've had some great success with Docker, both to build Armbian for and to deploy our application on a Lime2.

 

The catch is, I'm doing this on a Mac, using the latest beta of Docker. The key benefits of this are:

- a single command to build an entire deployable image,

- can test the actual compiled ARM application directly before deploying to the embedded board.

 

Much thanks to lanefu for his comments on the loopback device nodes, which was the final piece of the puzzle.

 

My current Dockerfile, which adds some extra required packages.

# Docker build environment for Armbian
#  
# Build the image:
#   docker build -t armbian-build:clean lib/
#
# To run a container and perform a build:
#   docker run -it --privileged armbian-build:clean
#   ./compile.sh
#
# Once completed, you will need to commit the build as a new container to save cached build and downloads
# OR uncomment the final line of script instead. 
#   docker commit `docker ps -lq` armbian-build:post-build
#
# Next time run 
#   docker run -it armbian-build:post-build

FROM ubuntu:16.04

RUN apt-get update
RUN apt-get install -qy git build-essential binutils binfmt-support qemu-user-static nano pkg-config ccache ntpdate udev

# Recreate loopback and partition devices
RUN rm -rf /dev/loop[0-1]*
RUN mknod /dev/loop0 b 7 0
RUN mknod /dev/loop0p1 b 259 0
RUN mknod /dev/loop0p2 b 259 1
RUN mknod /dev/loop0p3 b 259 2
RUN mknod /dev/loop0p4 b 259 3

RUN mknod /dev/loop1 b 7 1
RUN mknod /dev/loop1p1 b 259 0
RUN mknod /dev/loop1p2 b 259 1
RUN mknod /dev/loop1p3 b 259 2
RUN mknod /dev/loop1p4 b 259 3

WORKDIR /root
RUN git clone --depth 1 https://github.com/igorpecovnik/lib/
RUN cp lib/compile.sh .
# RUN ./compile.sh BOARD=lime2 BRANCH=next RELEASE=xenial KERNEL_ONLY=no PROGRESS_DISPLAY=plain 
    # uncomment this line to run the first build, to populate the docker image with current sources and packages

[lanefu]

 

I've had some great success with Docker, both to build Armbian for and to deploy our application on a Lime2.

 

The catch is, I'm doing this on a Mac, using the latest beta of Docker. The key benefits of this are:

- a single command to build an entire deployable image,

- can test the actual compiled ARM application directly before deploying to the embedded board.

 

Much thanks to lanefu for his comments on the loopback device nodes, which was the final piece of the puzzle.

 

My current Dockerfile, which adds some extra required packages.

# Docker build environment for Armbian
#  
# Build the image:
#   docker build -t armbian-build:clean lib/
#
# To run a container and perform a build:
#   docker run -it --privileged armbian-build:clean
#   ./compile.sh
#
# Once completed, you will need to commit the build as a new container to save cached build and downloads
# OR uncomment the final line of script instead. 
#   docker commit `docker ps -lq` armbian-build:post-build
#
# Next time run 
#   docker run -it armbian-build:post-build

FROM ubuntu:16.04

RUN apt-get update
RUN apt-get install -qy git build-essential binutils binfmt-support qemu-user-static nano pkg-config ccache ntpdate udev

# Recreate loopback and partition devices
RUN rm -rf /dev/loop[0-1]*
RUN mknod /dev/loop0 b 7 0
RUN mknod /dev/loop0p1 b 259 0
RUN mknod /dev/loop0p2 b 259 1
RUN mknod /dev/loop0p3 b 259 2
RUN mknod /dev/loop0p4 b 259 3

RUN mknod /dev/loop1 b 7 1
RUN mknod /dev/loop1p1 b 259 0
RUN mknod /dev/loop1p2 b 259 1
RUN mknod /dev/loop1p3 b 259 2
RUN mknod /dev/loop1p4 b 259 3

WORKDIR /root
RUN git clone --depth 1 https://github.com/igorpecovnik/lib/
RUN cp lib/compile.sh .
# RUN ./compile.sh BOARD=lime2 BRANCH=next RELEASE=xenial KERNEL_ONLY=no PROGRESS_DISPLAY=plain 
    # uncomment this line to run the first build, to populate the docker image with current sources and packages

 

 

@neomanic I think you just hit on a sweet spot that might improve my repeatability.   I've been cheating and just running builds over and over in the same running container.. I need to treat the container as a process like you are.   Thanks for sharing your tweaks.

 

The Mac docker is running an instance of Alpine line for the docker host right?  I'm glad it had all the kernel modules needed for cross compiling.

[neomanic]

@neomanic I think you just hit on a sweet spot that might improve my repeatability.   I've been cheating and just running builds over and over in the same running container.. I need to treat the container as a process like you are.   Thanks for sharing your tweaks.

 

The Mac docker is running an instance of Alpine line for the docker host right?  I'm glad it had all the kernel modules needed for cross compiling.

 

Yes, that's right. I think we're fortunate in that the same kernel features required for running Docker overlap with those to do cross-compiling, I was very pleasantly surprised.

 

I've found a few mistakes in the above Dockerfile. The mknod commands don't seem to do anything, and apt-cacher-ng isn't running for the compile. So it's probably best to just go as far as the apt-get commands, then do the rest in the actual running container, and then commit the result as a new image. Then each time you can just run a build.

 

I'm now working on folding our application into the build so we can build an image from scratch with just a few commands. But first trying to get this eMMC working on the Lime2 boards... argh!

[lanefu]

I'm tinkering with image building via docker again.. I was curious if you have made any improvements to the process..and was curious about your app building

[op1tjaap]

I was wondering if anybody has the steps to build your own Armbian kernel with the help of Docker. I see some scattered postings and processes. Would be nice to add the right way to do it to our documenation. Anybody got a working recept? 

[op1tjaap]

Ok ......asking the question is answering it....

 

In the igorpecovnik lib is an example docker file is an example Dockerfile: https://github.com/igorpecovnik/lib/blob/master/Dockerfile

 

 

I give some hands on tips for novice docker and kernel hackers.

First of all...why docker? 
Well if you only have an Centos 6.x x64 machine (a VPS) like I have and you would like to make your own custom kernel it is handy.
So start with installation first:
https://docs.docker.com/engine/installation/

 

If up and running first test if you understand the basics. Do this as root or use sudo

 

# docker run hello-world 

If correct working you will see a “Hello for Docker!”

Then make a working directory like docker-armbian

 

# mkdir /root/docker-armbian 
# cd /root/docker-armbian 

In this directory you download the Dockerfile:

https://github.com/igorpecovnik/lib/blob/master/Dockerfile

FROM ubuntu:16.04
RUN apt-get update
RUN apt-get install -y git build-essential binutils
WORKDIR /root
RUN git clone https://github.com/igorpecovnik/lib/
RUN cp lib/compile.sh .

 

Then download the image for the Armbian container with this command: 

# docker build -t 3mdeb/armbian .

Don't forget the last dot ( .). It tells docket to read the Dockerfile!
The correct image will be downloaded 

The start it up:

 

docker run  -t -i 3mdeb/armbian

Then check if you are i the container:

# ls –al

You will see that you are  directory /root and the compile scipt is also copied over there. If you want to edit, please do a

# apt-get install nano 
# apt-get install vim

So you can edit the compile scripts to your need.
Important is that in docker you can compile a custom kernel but not a complete image. This have to do with all kind of problems with file systems. ( if anyone succeded in doing with…please add it to this post!)

Finally just do a

./compile.sh

and sometimes press enter, etc. if you are asked. You will reach the limit of the 10BG disk barrier, but please press enter to continue.

 

Succes and fun!