Having received full information about services we provide at Railsware, people often wonder in the end: You do not have QA? How do you guarantee the quality of the code and product?

Yes, we do not have QA as a separate role. Intentionally. For many years already. Any code written by a Railswarian is 100% covered with unit and integration tests. Moreover, the functionality of every new feature ready for staging is double-checked during the TestFest.

Wait, the TestFest?!

One more frequent question. Probably justified. The name “TestFest” is not commonly known but is Railswarian lingo. In wider circles it may be known as Software Quality Assurance. But even this name does not make the process much clearer if you are not a tester or do not actively participate in product development. Considering this, we decided to write an article, which hopefully will provide a clear answer to the question above.

So, what is a TestFest?

TestFest is a session of manual software testing. Its main purpose is to guarantee defect-free features before delivering them to production. Frequency depends on the development stage. At the active development phase, TestFest ends each iteration, which in most common practices happens every 1-2 weeks. Later, when iterations are bigger or when refactoring becomes the main focus, TestFest may be needed less frequently.

By the way, we are hiring. Check out our job openings.

Planning

TestFest may be a time-consuming procedure: depending on the size of the project, team and features to be tested it can last from a few hours and up to a day. TestFest needs to be scheduled beforehand to ensure no interruptions or overlapping with other activities. It shouldn’t take place right before release. In case of bugs you may need some time to fix the most critical ones which cannot wait till the next iteration.

Scrum Master, Product Manager or Lead Engineer (depending on the team structure) is responsible for creating a spreadsheet with the list of user stories, chores, bugs that need to be tested. Before Scrum Master starts working on the list of items scheduled for iteration and thus testing, s/he asks the team to update statuses of tasks in tracking system. This step is important not to overlook anything that is in fact ready for release, but got lost in the “in progress” column.

Starting with preparing a list of user stories or flows to test, Scrum Master also agrees with the team on the Testing Environment:

• devices to be used for testing – desktop, laptop, tablet, smartphones with different display resolutions
• operational systems – Macintosh, Windows, Linux, iOS, Android, etc.
• browsers – Safari, Chrome, FireFox, Internet Explorer, Edge, etc.

Having finished with this part, Scrum Master makes sure each TestFest Participant receives a different combination of device-OS-browser.

In case the team does not have all the devices or operational systems in its possession, some emulating services may help here. For instance we often use BrowserStack.com – a tool created specially for testing, which imitates almost any device or OS.

Session

The process requires participation of the whole development team – engineers, product manager, sometimes UI designer, – and if it’s a client’s project then voluntarily someone from the client’s team can join. Right before starting with software testing it’s important to check that the latest version is delivered to the Stage/Test environment.

So the TestFest starts. As a rule it is split into 2 parts. First part is manual testing of the set of stories, that are going to be released to production. Depending on the product and team approaches, TestFest process may differ a bit. Each participant may be assigned to test the whole new functionality or separate features. In the latter case engineers do not test their own features, but each other’s. For example, if Alex built feature A and Mark built feature B, then Alex tests feature B and Mark – feature A.

The second part of the process is manual regression testing which serves to check, whether there are no conflicts between the new and the old functionality. It may take a while to go through existing functionality. Especially when a new big feature is introduced to already fully-functioning application and may cause unexpected behavior in any flow.

During the session any bug detected is recorded to the SpreadSheet Log or Issue Tracker.

Outcomes

As a result of TestFest the team has a list of bugs and issues. At Railsware we prefer to work with outcomes the same day: in case some fixes cannot be delayed till the next iteration, they should be taken to work right away. The rest of issues are discussed and prioritized and added to the general scope to be fixed and covered with unit and integration tests.

Here how the log looks like when session ends:

Although rarely, it does sometimes happen that the bug is both serious and urgent. That is when an extra day between a TestFest and release may save you. We recommend scheduling another TestFest in this case. Especially, when some bug prevents you from testing the rest of functionality.

See what else Railsware has to offer – visit Careers.

Conclusion

Even without dedicated QA, quality is never a compromise for us. Yes, we are big fans of automation and TDD, but we have never rejected manual testing. The ability to predict the behavior of an end user and prevent issues s/he my face are obvious advantages of manual software testing and validation. That is why we started practicing TestFests as a part of our development process long ago.

Of course, its main purpose is to make sure there are no conflicts and the old application functionality is not broken after introduction of any changes. But one more benefit of this approach is that it helps to keep the team involved and aware of the whole application functionality instead of being focused on its separate features.