Indeed. One year ago we started with first organized testing. We cover all boards except clones. We used project management application, set milestones from "feature freeze, beta testing, bug fixing, writing release information, launch". A new cycle starts with "feature development" and last for 3-4 months and we have 1-2month for freeze-fix-launch phase. We made this with 9 people involved and the first time it went fairly o.k., second time badly ... and third time we didn't try. There are many reasons why even the way and technology is set up perfectly. A tester is responsible for a single task (board) and he gets email: "We are testing, please check what works and what not, write comments". Like this:
Exactly. We didn't get here in the first place since our first testing cycle went fine, while second failed. And since I am not sure what is the best way to conduct the testing its hard to pass on.
This is good enough if testing process is done properly. At least as we did it the first time. Then it could be also divided into userspace - which should be hardware independent - testing and low-level stuff. IMO, we need a project manager for this testing which can't be me. Organizing testing means some communication but I am in general already up to two weeks behind with my email and other private comm. I can't lead this.