Would it be appropriate if Arcam allowed us in this forum to access the Beta versions of the software. We are a group of people here who clearly have some knowledge and a great desire to get a software that works. Had we had access to FW 1.41 before it was released, we would have been able to quickly announce the problems we have found and hopefully Arcam would have been able to fix these before a release.
Why not? It’s a good idea!
I thought this myself a while ago, but by releasing as they do effectively every user is a beta tester because there doesn’t appear to be a high QA threshold on each release, so why bother? What would perhaps be better, is if they released hot fixes for specific issues and issued only to whoever reported the problem to try before putting it in a formal release. Some rollback instructions wouldn’t go amiss either, so it’s easy to get back to something more stable if you encounter a bigger problem than the fix is trying to address.
I received this answer from Arcam’s support when I asked the question of a possible new release at the end of November this year.
“When releasing software, there is often an imbalance between people wanting software” now "and the time it takes to develop and test software that will actually fix the issues. We can release software at any time, but if we rush to send it out without testing it because of calls for software “now” then there’s a good chance it will not actually fix anything, and then there will be complaints that it wasn’t tested.
There is currently a beta that we are testing to see if it resolves the issues of which you speak. This will be released when the development team are satisfied that it does resolve at least the majority of issues being reported. This may be this week, or the week after, but ideally when it is ready and fit for release. Rushing updates out rarely improves the situation. "
So I assumed they had some kind of beta test group that tested the software properly before releasing it. But it seems like you are saying that it is not tested in a good way before unfortunately. Hence my suggestion that we could help as we are still a group of frequent users who quickly find the bugs in the software that is released.
Although wikipedia defines beta testing as external user acceptance testing I think your email excerpt might refer to other testers besides the development team (e.g. selected partners) which do not have to be the users of this forum.
In my option it also depends on the reasons why you want to get involved in beta testing. If it is the “desire to get a software that works” then this might not always be true because beta builds might expose other unexpected behaviour. If people expect “software that works” and get additional unexpected behaviour then there is a risk of getting frustrated. I think this should be prevented at all times.
An everlasting engineering credo is: “you cannot test quality into a product”. Quality is the result of all engineering aspects from requirements interception down to the release of the product. Testing is just one aspect of it.
So the question is: are you prepared to get (a lot of) additional unexpected behaviour besides the issues you are facing now?
Testing provides evidence of the level of quality hence often also referred to QA, but as you say QA processes go beyond testing of software. The testing will have pass and fail targets to meet as “exit criteria” then to allow the general release. My gut feeling is that these targets are too low, but as Arcam say there is a fine balance because if they wait too long to release an update waiting for it to be 100% users still get annoyed that Arcam isn’t listening to them. So some activity I guess is best even if things aren’t perfect.
Beta testing is user testing, but usually to a restricted set of users otherwise there is no difference to a general release. I think the idea the OP had is rally to do this to put the product out there into real life use as an extra test phase but obviously whoever was in that group would have to accept early uptake of enhancements are likely offset against potential risk for more bugs than you’d expect in a full general release. I still think Arcam are putting releases out without calling them beta and yet that is kind of the reality what they are IMHO. But I can live with this personally. Clearly there are plenty of people out there who won’t accept anything other than 100% perfection and whilst not all may agree it is understandable given this is not a cheap product and Acmam’s flagship integrated amp at the mo.