Building Scalable Robot Framework Test Automation: Benefits and Challenges

Internal testers can bring a unique perspective to the transition to test automation. Situated between the development and operational teams, the role has the opportunity to witness business requirements translated into code. The following is an inside look at one pathway toward implementing Robot Framework for test automation, both the benefits and why an organization might opt for a more comprehensive solution for easier setup and less maintenance.

Happy Path Leads to Unhappy Users

Many companies, especially young startups, lack systemic testing processes. As you can imagine, the pressure of delivering a new feature in a startup company can be immense. All independent software components—not to mention custom integrations—are expected to work harmoniously for a functional product.

Continuously running unit tests can be put in place for each of the components. Then, many companies rely on happy path test cases as the final functionality testing. The rest is often left for end customers to explore, which backfires nearly all of the time. Some may call this the shift-right approach. No matter what you call it, the repercussions are detrimental: 

• Brand image takes a hit. The problem with happy path testing is that a significant amount of users stray from that joyous route. 
• Bugs are pervasive. The swarm forces developers to spend far too much time on avoidable patches rather than developing new features.
• You are burning through resources. The business model costs are unsustainable. 

Introducing Exploratory Manual Testing

One way to improve testing is to assign the exploratory testing of new features and the constantly expanding regression set to a dedicated manual tester rather than have developers do the testing. There’s some truth to the cliche that developers and testers have totally different brains. Why? Because one finds joy in creating something new and the other in breaking it. Accordingly, developers often make terrible testers and vice versa.

Breaking News: Manual Regression Testing Can’t Work at Scale

With manual regression testing, you can find and fix more bugs before production. Developers can turn their attention away from patches and toward new features. Of course, this leads to a far superior user experience. Unfortunately, it can only take a few months for the manual tester to become the most significant bottleneck between development and deployment.

It can take weeks to run through an entire testing cycle. With bug fixes, the cycle gets put on repeat. For example, imagine a lone tester running through 250 end-to-end test cases to be performed on four different browsers and 25 end-to-end test cases to be performed on the physical product’s user interface (UI.) 

With such an unsustainable workload, testers begin to experience fatigue. This leads to them making assumptions and becoming blind to regression bugs. Worse, they often can’t finish their testing, period.

The Shift Towards Automation

The obvious next step is automation. Many companies go through an initial phase of test automation skepticism. For example, testers may fear becoming redundant. However, a manual testing process is simply not sustainable at scale.

So, you have to sit down to ask the important questions. Where do we start? Who is going to do it? Which framework should we use, and how can we measure its success?

Criteria for Selecting Test Automation Tools

Choosing a framework is an interesting challenge, especially since those using the framework are usually not developers. Here are a few characteristics to prioritize while searching for a testing framework:

• Readability. Test case readability is crucial. The more self-explanatory the test case, the more people can review it. This way, the business owner can review test cases if necessary (think low-code).
• Support for appending functionalities. Look for a framework that can do this to respond to fluctuating needs. 
• Fit with your budget and needs. Open-source tools are often free, but commercial testing tools offer functionalities that go beyond their open-source siblings, especially for low-coders and cloud-native architecture.

Testing solutions that incorporate the Robot Framework test automation ecosystem provide both flexibility and readability.

Let’s look at the benefits and challenges of implementing Robot Framework alone and see when you might want to consider a more comprehensive option.

Step One: Implementing Robot Framework Test Automation

One downside to Robot Framework is setup. Many companies find they’re in for a rocky start. Unlike cloud-based testing software, it can take a couple of weeks to get Robot Framework up and running on a local machine. For a non-developer, heavy installation work is required to sort out the browser version, driver version, and so on.

However, once you have Robot Framework up and running, you need to define a test automation strategy. For many testers, two key concerns are

1. How do we accomplish library development for non-supported functionalities?
2. How can we conduct manual testing in parallel?

Step Two: Strategizing Test Automation 

Some easy test cases to start automating are web application tests that run against different browsers because. These often have the most regression tests, many of which involve similar, repetitive steps. For example, you can automate a test involving application availability to different user groups. When deciding between automating an end-to-end process or an application login process, the latter is a much easier choice.

A test automation solution built on Robot Framework has the ability to append its functionality with additional libraries. Establishing a proper development process is essential to implementing a library that addresses all your functionality needs. The tester can own the library, but the code itself could be peer-reviewed by a seasoned developer.

The Journey to 80% Test Automation

One thing to keep in mind is that automating everything immediately isn’t feasible. In the beginning, you can continue to test manually whatever isn’t automated. Once the present regression set is automated, test new functionalities manually for exploratory purposes. Then append them to the automated regression asset. 

For example, if you run into availability issues with our web application, schedule availability test cases during production. Your team can receive a heads up to their email, Slack, and common monitor when they fail. With this responsiveness, you can stay nearly a step ahead of your users.

6 Months into Test Automation

After six months, you can successfully automate 80% of your regression test cases. As a result, your DevOps maturity will grow exponentially. You can continuously build, integrate, and test with your automated test assets. Then deploy in a production-like environment. 

Manual tests can be reserved for major releases and distribution. With automation, testing takes several hours, rather than several weeks. This allows everyone to focus on new features, exploratory testing methods, and maintaining assets.

The Downside of an Open-source Scalable Solution

When the trade-off is monumental release cycle acceleration, no price seems too high. But a wise man once said that progress makes your problems evolve, not disappear. The biggest setback with Robot Framework test automation is maintenance. Maintenance duties can weigh a testing team down and make it impossible to manage scalability. This issue isn’t exclusive to Robot Framework. Upkeep is one of the most common problems among test automation experts.

How to Secure the Scalability of Test Automation

To reap all the benefits of automation without incurring the wrath of continuous maintenance, invest heavily in expertise from the jump. There are services made to accompany open-source software like Robot Framework, and for good reason. Comprehensive test automation solutions like Copado Robotic Testing (CRT) use cloud architecture and AI to solve many of the maintenance challenges you are bound to face. With a cloud-based solution, you can test from anywhere, and in any browser. AI massively cuts down on maintenance workload by self-healing broken tests. CRT’s machine learning can also predict the quality of future releases. Copado Robotic Testing has all the flexibility of Robot Framework in an easy-to-use, cloud-based package.