The role of automation in software testing

Over the years, QA software testing has evolved from being a largely manual process and today testers are able to automate on many more features than ever before. At CA we have been using automated software testing as part of our QA processes since 2015 and it’s become an essential part of project delivery.

However, automation, by the very nature of its name, suggests a dream that is misleading and so setting expectations for how it will work is key to its success. It’s certainly true that it can take away the more tedious parts of testing and reduces overall time to test – which has clear project cost benefits – but it is no silver bullet.

Automation testing sounds great in theory. You let your test run and everything is done. Right? Well no – wrong. Unfortunately, that’s not the case, even though part of that statement is true.

There’s a lot more consideration needed to make automation testing run and be a useful tool. As with anything automated, it’s a case of ‘rubbish in rubbish out’ so it needs time and effort to create the automation suite – and thereafter to maintain it. The real rewards come when the automation is used to assist the manual test. The result is a testing process that is more reliable because bugs are caught much earlier in the process. As a result, it allows better regression and smoke test coverage and estimation.

Things can go wrong. Remember we talked about the concept of rubbish in rubbish out? When this is not observed in set up of the automation suite there’s the potential to get into all kinds of trouble.

If there’s not much confidence in the automation suite you may end up in a situation where you’re not totally sure whether bugs are due to the tests not working properly or whether they are actual bugs in the software. This is when the automation dream can become a bit of a nightmare. When this happens, you have an automation suite that’s not serving its purpose.

One of the benefits of automated testing is reducing the time and costs associated with some repetitive parts of testing. For instance, checking all 200 names are displayed correctly on a page or similar. The testers’ dream would be to press a button and a machine would do the checking in seconds. Sadly, not everything can be automated. Many features still require a manual check. For example, making sure animations work as intended, images display as they are designed, fonts and more.

One of the main areas that it really benefits testing is when regression testing is required; this way the QA will have the ability to run the automation test – and it dramatically reduces time to test.

There are many frameworks available to automate. Frameworks that allow you to automate web apps, as well as mobile, iOS and Android devices and more. Choosing the best framework is vital to ensure that what you’re trying to accomplish is possible.

At CA when we’re automating on a Xamarin project we use the UI Test framework. It integrates well with Xamarin projects, as it’s created using the same coding language. This enables developers within the team to assist, which saves time.

One major benefit of UI test is that it allows you to seamlessly use visual studio app centre, which automatically builds and deploys to real devices, to run your test, collect crash logs to identify causes of crashes. This way, developers have a better understanding of the cause of the bug.

Automated QA testing has its place but it’s not the panacea suggested by its name. There’s plenty of effort needed behind the scenes to make it work as a useful tool in the QA’s armoury. Using it judiciously and in conjunction with manual testing has the best outcomes and cost benefits.

At Chelsea Apps QA is an integral part of the digital products we build. Quality is not optional – it’s inbuilt to everything we do, and our QA professionals understand the balance of manual vs automation to achieve optimal project outcomes for our clients.

If you’d like to understand more about QA testing and how we use it please get in touch – we’d love to hear your experiences and views.