Improve autotests script


Werner
 Share

3 3

Recommended Posts

Origin:

https://github.com/armbian/autotests

 

Description:

Having a reliable automated testing for boards is absolutely mandatory. While the tests working most of the time the reports are not as reliable as they should. Also the code could need some refactoring and cleaning.

 

Destination:

https://github.com/armbian/autotests

 

Skills required:

Bash

Link to post
Share on other sites

Armbian is a community driven open source project. Do you like to contribute your code?

I will just add a bit more. 

 

We want to rewrite / make this from scratch with Ansible. Basically we seek Ansible expert or someone who wants to become one.  We have this know-how but we are simply too overloaded to move on. This is yet another job for common good! To make Armbian support better. To make Debian support better. To make Arch better, OpenWrt, ... To make every Linux distro out there running better on your board.

 

The person(s) should focus only on testing. Creating and maintaining scripts for automated testing and later, when our support hardware is out of beta testing, implement that as well. It's a continuous project and will not end tomorrow. Also you don't need to stay on it forever. Do what you can. Help us get going ...

 

We would like to automate:

 

- initial board setup (after you flash the image, automated 1st login, setup network, different local repository, ...)
- simple tasks such as upgrade, change to beta repository, downgrade to specific kernel, etc.
- run various of tasks which can be added without limits
- run various of benchmarks which can be added without limits
- run tasks in parallel
- make reporting in HTML, XML (common and per board)


Most of those ideas are covered in some basic form in our first try: https://github.com/armbian/autotests

 

More ideas:

https://github.com/SoInteractive/ansible-benchmark

 

please-help-cat.jpg

Link to post
Share on other sites

Hi Igor, i started to write such a tool for the same ûrpose as yours, it was a log time ago, for exactly the same target as yours (many things to talk;) )   it under Apache License and github, usung python and simple unit test. BTW i know verwell Ansible, and i'll be very happy to help you

 

The tool is available under https://github.com/wbonnet/sbit

 

It looks empty a first i need to push several things if you are interested, i will provide doc and some board auto test. The simpliest is maybe to start talkinng about it here or on IRC. deb packages are also availables online from the DFT project (githhub) also

i'll look forward to you about this subject in the next days, i my have several other thing to share and contribute

cheers William

Link to post
Share on other sites

5 hours ago, William Bonnet said:

Hi Igor, i started to write such a tool for the same ûrpose as yours, it was a log time ago, for exactly the same target as yours (many things to talk;) )   it under Apache License and github, usung python and simple unit test. BTW i know verwell Ansible, and i'll be very happy to help you

 

The tool is available under https://github.com/wbonnet/sbit

 

It looks empty a first i need to push several things if you are interested, i will provide doc and some board auto test. The simpliest is maybe to start talkinng about it here or on IRC. deb packages are also availables online from the DFT project (githhub) also

i'll look forward to you about this subject in the next days, i my have several other thing to share and contribute

cheers William


Wonderful news! I briefly checked your project and it seems to match our ideas on (unit) testing. I am Ansible newbie, know basics, while @lanefu (US) is more experienced in this and we should all come together on chat - we are hanging out here https://docs.armbian.com/Community_IRC/. Our most active hours starts from lunch break, when US people gets up and evenings, late night, CET.

 

Thanks!

Link to post
Share on other sites

Hi, newbie here.

 

I would like to help with those tests, and maybe with Ansible. But I'm new in that, so I will try to learn and help. I learning linux, so I will try to learn bash to help you guys, and because it is usefull skill for linux.

I've got the Rock pi S, Rasp. pi 3b+, rock pi X to try some tests.

Link to post
Share on other sites

Just joined as contributor,

 

I was reviewing https://github.com/armbian/autotests and some of the task can be automated with Ansible very quickly, is there any work on the go or should I start a new repository?

 

Any special format for the html reports?

 

I will be setting up an env to test the actual code [autotests] will report back with doubts and questions...

Link to post
Share on other sites

15 hours ago, Pavel Milanes CO7WT said:

I was reviewing https://github.com/armbian/autotests and some of the task can be automated with Ansible very quickly


Yes, I do believe so. It is probably best to just start from scratch. I made a new repository, named QA, since entire testings goes under quality assurance. https://github.com/armbian/qa I have also added you more privileges.

 

15 hours ago, Pavel Milanes CO7WT said:

Any special format for the html reports?


Keep it generic, perhaps left in XML / JSON format we can import elsewhere or into some generic XML 2 HTML function.

 

15 hours ago, Pavel Milanes CO7WT said:

I will be setting up an env to test the actual code [autotests] will report back with doubts and questions...

 

The goal is that we can test simultaneously on many devices, run as much tests we like and get some results. Also check if it is possible to make a bridge with @William Bonnetwork. To have some a bit more generic testings with an option to have per hardware family. Not needed per board. But perhaps. we can prepare its possible.

Link to post
Share on other sites

Hi, I'm looking at this too, but I'm new to Ansible and starting to learn about it.

I've forked autotests and started to play with playbooks and roles. My modest additions can be viewed here: https://github.com/jeanrhum/autotests

 

As I understand, QA repository is the new way to go and all must be built from scratch. I'll continue my expriments on my side, waiting for guidelines on QA to see if I can help in some way.

Link to post
Share on other sites

2 hours ago, jeanrhum said:

Hi, I'm looking at this too, but I'm new to Ansible and starting to learn about it.


Here I am also newbie, so can't comment much. I understand basic principles and that's about it.

 

3 hours ago, jeanrhum said:

I'll continue my expriments on my side, waiting for guidelines on QA to see if I can help in some way.

 

 

Nice! Experimenting is important and valuable.

 

I have opened an epic with basic requirements and a few tasks. Let's make / expand a plan before jumping on actual coding: https://armbian.atlassian.net/browse/AR-948 (who doesn't have access to Jira, but would need it to add/change things, PM)

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share

3 3