Jump to content

THE testing thread

lanefu

By lanefu
in Armbian build framework

 Share

Recommended Posts

Hijax

Hijax

Posted
Posted

DSC_0001.thumb.JPG.e84bcb4a66f67a94618e42ec65b7f82a.JPG

 

Picture of first power / serial board. This is the old design and I am awaiting for new one (just got DHL notification of incoming parcel) 

 

I expect that till end of August the prototype will be assembled and electrically tested. Then as promised I will publish design and manufacturing files.

3
Hijax

Hijax

Posted
Posted

Thx @lanefu, as I already wrote I am also interested in that design, to sort out my shelf of SBCs and stop messing with SD cards.

That said may it be that in the future I can connect some of my boards (as I have dozen of not used ones) for continuous integration of future Armbian changes.  In a automatic way.

 

So the question quickly pops up: I will continue with HW design but who will do the scripts that will use that HW? :) I.e. run compile.sh, upload to remote SD, power SUT, gather logs?

lanefu

lanefu

Posted
  • Author
Posted
Thx [mention=1231]lanefu[/mention], as I already wrote I am also interested in that design, to sort out my shelf of SBCs and stop messing with SD cards.
That said may it be that in the future I can connect some of my boards (as I have dozen of not used ones) for continuous integration of future Armbian changes.  In a automatic way.
 
So the question quickly pops up: I will continue with HW design but who will do the scripts that will use that HW? :) I.e. run compile.sh, upload to remote SD, power SUT, gather logs?
Im a devops consultant and fluent in Ansible so I think im Obligated in that regard. I can at least handle the plumbing hopefully others can implement the more granular testing once the framkework is in place.
2
Hijax

Hijax

Posted
Posted

Status update. DHL parcel received. New boards photos: 

 

FE49E331-507E-43B6-B5AA-1D29865C7FD1.thumb.jpeg.daaf7e2b1dde2625c14e9bc9f15d55dd.jpeg

 

and SD adapter inserted into Rock64

5A070289-A7CC-4E14-A6A2-9E8F0DE55D9C.thumb.jpeg.fb30d33a3c5bc4732ff28f5caf76ff1d.jpeg

 

Now I am waiting for remaining components then soldering/assembly will start. 

2
Hijax

Hijax

Posted
Posted

Hi @lanefu, things go slowly for two reasons: problem with delivery of sdcard connectors and also lack of free time for soldering ^_^

Now, after summer time period, things hopefully will go in more planned & coordinated way. If you like, I can publish design documentation right away. Or just stay tuned, I do not drop this project, so i will update in one or two weeks status.

  • Igor pinned this topic
Tido

Tido

Posted
Posted
On 9/12/2019 at 8:08 PM, Hijax said:

Small progress update -

Hi, I don't know how life is treating you, I hope good.

 

Did you ever do some testing, further steps?

I mean, is the HW useable or are there any obvious errors on the PCB ?

 

  • lanefu unfeatured this topic
Igor

Igor

Posted
Posted

RFC on auto tests. Ideas how to shape this script are welcome!

 

Its getting better and better so I hope we will soon be able to make regular use of it and equipment from the topic :) 

2
Igor

Igor

Posted
Posted

I have 25 boards in the automated testing facility ATM :) and this is how the report (with bugs) looks like:

 

report.png

3
Tido

Tido

Posted
Posted
9 hours ago, Igor said:

automated testing facility

With the board from @Hijax or what does it look like?

 

Different Symbol, for: not working  /  not applicable/available on the board would be nice.

 

// sent from mobile phone //

Igor

Igor

Posted
Posted
6 hours ago, Tido said:

or what does it look like?


That board will only extend and improve testing procedures by collecting data from serial and by powering the board on/off + writing rootfs from scratch, now I need to image by hand. But still you need an application, software to drive all this. And that's what I am doing. Without, board is paperweight.

 

6 hours ago, Tido said:

Different Symbol, for: not working  /  not applicable/available on the board would be nice.


I will leave this for someone else to fix.

Tido

Tido

Posted
Posted
On 3/5/2020 at 9:24 PM, Igor said:

RFC on auto tests. Ideas how to shape this script are welcome!

Well, you asked I only answered.

 

I was just testing the script - two things weren't easy to begin with

  1. why do I need to:  set test subnet or IP address(s)  (is this the counterpart Network-Device to test?)  -  I have DHCP so I tried without.. $HOST not defined. Exiting.  :(
  2. Bluetooth MAC-Address: Android, I looked first in the device pairing section - nothing.  Then a lucky strike, in the settings/ about my phone I found it.

I think it would be valuable to add this, in the config file and README.md.   I am still a noob on Github.

 

Igor

Igor

Posted
Posted
20 minutes ago, Tido said:

why do I need to:  set test subnet or IP address(s)  (is this the counterpart Network-Device to test?)  -  I have DHCP so I tried without.. $HOST not defined. Exiting. 

 

You are running the script on your desktop / server. It scans armbian boards on the network. If you set SUBNET, it will scan for devices on that subnet. I have dedicated subnet for testing armbian devices. I only hook power and network cable and they are ready for this script. I don't need to do nothing else.

 

If you only run a few devices in your only subnet, use HOSTS and type there IP addresses of devices you want to run tests on.

 

22 minutes ago, Tido said:

Bluetooth MAC-Address

 

Put your phone BT to discoverable mode and run: 

hcitool scan

on a Linux computer that has BT dongle.
 

24 minutes ago, Tido said:

valuable to add this, in the config file and README.md


Welcome to try adding those hints.

2
Tido

Tido

Posted
Posted
19 hours ago, Igor said:

You are running the script on your desktop / server.

smart move, I like that.

 

To improve the How To:

After flashing the SDcard. Put the SDcard into the SBC, starting the SBC, waiting until the device has booted. 

Welcome to Armbian buster with Linux 5.4.20-rockchip64

System load:   0.00 0.00 0.00  	Up time:       2:39 hours		
Memory usage:  3 % of 3867MB 	IP:            192.168.11.7
CPU temp:      53°C           	
Usage of /:    5% of 30G    	

[ General system configuration (beta): armbian-config ]

Last login: Fri Mar 20 18:36:41 2020 from 192.168.11.12

 

What is the next step?

 

Igor

Igor

Posted
Posted
8 minutes ago, Tido said:

What is the next step?


Define HOSTS="192.168.11.7" at your server and run the script. Wireless tests should be optional. Make at least 3 runs, that it switch to beta and back to stable ...

Tido

Tido

Posted
Posted
3 minutes ago, Igor said:

run the script.

You are already ahead of me.

 

Shall I do a first login, change the password, create a user   or not?

Igor

Igor

Posted
Posted
4 minutes ago, Tido said:

Shall I do a first login, change the password, create a user   or not?


nothing.

Tido

Tido

Posted
Posted
2 minutes ago, Igor said:

nothing

I thought so.  In the event I have already done it, could I fix this by adding the password and username I have already given?

 

like: 

USER_ROOT="root"
PASS_ROOT="password I chose before"
USER_NORMAL="username I chose"
PASS_NORMAL="again Passowrd I chose"
NAME_NORMAL="name I chose"
ROOM_NORMAL="Red"
WORKPHONE_NORMAL=""
HOMEPHONE_NORMAL=""
OTHER_NORMAL=""

or could I delete a certain file,  execute a command to return to the state 'nothing' ?

 

Igor

Igor

Posted
Posted
37 minutes ago, Tido said:

could I fix this by adding the password and username I have already given?


root username and password is essential, the rest not.

Tido

Tido

Posted
Posted

@Igor, while collecting data to enhance the ReadMe - I found this:

 

A re-run will delete the previous results - move files to a archive folder?

  Creating manually an archive-folder inside 'logs' will erase it toghether with all the files.

 

Another idea, add a time stamp column and amend the new file/results to the old results.
 

for the ones who want to try:  Prepare your system
To run the bash-scripts you need 4 small softwares, check if they are already installed on your PC:
dpkg -l sshpass git iperf3 jq
If the result shows 'ii' at the begin of the line, you have it installed.   If dpkg-query:  returns with: not found,  you can quickly install these small programs:

sudo apt-get install sshpass git iperf3 jq

 

Igor

Igor

Posted
Posted
1 hour ago, Tido said:

A re-run will delete the previous results - move files to a archive folder?

  Creating manually an archive-folder inside 'logs' will erase it toghether with all the files.


I already fixed this to only delete logs, but archive reports in another folder. This can be improved later, but its not that critical ATM.

 

1 hour ago, Tido said:

Another idea, add a time stamp column and amend the new file/results to the old results.


Yeah, will add that.

If you have time. Or anyone. Icons would be nice to have from one source and such with a proper open licence. Current are just grabbed from the internet - I spent less then 5 seconds / icon. Also some are missing (cycle, board, memory, drive, stress, dvfs, ... and those might be used in the future).

Tido

Tido

Posted
Posted

I would change 'drive' to storage. And adding something like   read/write   Maybe in the same moment Mhz to MHz   Ghz GHz

 

ICONS: should it be SVG so it would scale on displays with high resolution?

 

 

2 minutes ago, Igor said:

If you have time.

Spent the morning writing this (pasted it in markdown), if this is fine with you I can try a pull request

 

Spoiler 

**Introduction**  
TL;DR  It is installed on the desktop. Adapt the configuration-file to your requirements. Start the test procedure. Let it finish. Check the output for errors. Enjoy the nice report.html :)


**Privacy concerns**  
This is opt-in, you decide whether you run it or not. Icons come from https://www.pngkey.com


**Getting started**  
You are running the script on your desktop or server, which saves the hassle of copying the files onto the devices. The bash-script will ping the IP-Address or scan the network. If you set a SUBNET, it will scan for devices on that subnet.   
It will create logins based on the configuration-file and run different tests to see if the board is working properly.  In the event you already have root-user, just put your password there instead of the generic 12345678. More information about the configuration you find below.  
All you need to do is configure the configuration-file, hook up the power and network cable and you are ready for running the test. That's it.

 **Prerequisites**  
Download the latest armbian image. We recommended to use .torrent because this does the file integrity check for you automatically.  
Write the data to the SDcard. I am still running Etcher 1.2.1 on my Ubuntu Mate 18.04, works like a charm (80 Mb) https://docs.armbian.com/User-Guide_Getting-Started/#how-to-prepare-a-sd-card

Put the SDcard into your device, power it up and let it sit for 2 minutes.

What is your device(s) IP-Address or do you have a dedicated subnet for your devices.
To find the device, login to your router or use this tool http://angryip.org/, to find your boards IP-Address(es).


**Prepare your system**  
To run the bash-scripts you need 4 small programs, check if they are already installed on your PC:  
`dpkg -l sshpass git iperf3 jq`  
If the result shows 'ii' at the begin of the line, you have it installed. 
If dpkg-query:  returns with: not found, you can quickly install these small programs:  
`sudo apt-get install sshpass git iperf3 jq`

1. Go to a folder where you want to store it. The following command will create a folder called 'autotests'. Clone the sources from Github and open the folder autotests:
```
git clone https://github.com/armbian/autotests
cd autotests
./go.sh
```
if the download is not finished after 1 min, abort with: ctrl + c  and start over.
 
2. Edit `lib/configuration.sh`

	- set WLAN_SSID and password (2,4 or 5,0GHz)
	- set IP-Address (HOSTS) or Subnet
	- set BLUEDEV MAC-Address of a Blueooth device (Android phone for example)

How to find the Bluetooth MAC-Address - on your mobile or PC. Put your phones BT to discoverable mode and run:  
`hcitool scan` 
on a Linux computer that has BT-Dongle. On an Android device you find the MAC-Address in the phones settings/about the phone/Status.  

	- change number of passes (optional)
	- change stress time in seconds (optional)
3. Run:
```
./go.sh
```
4. Once finished, go to `/autotests/logs`  double click: `report.html`

 

 

Igor

Igor

Posted
Posted
34 minutes ago, Tido said:

Spent the morning writing this (pasted it in markdown), if this is fine with you I can try a pull request

 

Few remarks. I fixed other things, will continue later after the lunch.

 

> It is installed on the desktop.

On any Linux machine that sits in your network.

> **Privacy concerns** 

There is none at this point. Data doesn't go anywhere.

>  Download the latest armbian image.

is enough + link to getting started.

>Put the SDcard into your device, power it up and let it sit for 2 minutes.

Put the SDcard into your device, power it up and let it sit for few minutes. Script will display which board(s) will run tests on.

>**Prepare your system** 

Remove everything what has to be installed. That is/will be done automatically.

> 2. Edit `lib/configuration.sh`

There below you describe how to find IP or choose subnet

Tido

Tido

Posted
Posted
1 hour ago, Igor said:

>**Prepare your system** 

Remove everything what has to be installed. That is/will be done automatically.

This would make it comfortable,  on the other hand, that makes it running unnecessarly  as  sudo, each time.  I would rather do the install once and thats it. Unless you come up with an idea I am not creative enough with your bash-magic :)

 

Tido

Tido

Posted
Posted

Hi @Igor, before I sent my PR I was searching the files for  mhz & ghz but couldn't find it. So, my PR only contains the readme.md

 

On 3/21/2020 at 12:40 PM, Tido said:

I would change 'drive' to storage. And adding something like   read/write   Maybe in the same moment Mhz to MHz   Ghz GHz

 

 Share
Go to topic listing
×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines

  I accept