There are a lot of things to consider when taking your startup’s QA to the next level by introducing automation. At what stage do you implement these changes? What software do you choose? What frameworks do you stick to? These are not easy questions to answer and will depend on your product, your plans for growth, and the industry itself, but we hope walking you through our rollercoaster experience at MUNCH:ON will give you a helping hand.
What we were doing
As a growing company, our product is constantly changing and growing with us. In the past, this meant the SQA team needed to thoroughly and constantly QA the products, including web applications, Android, and iOS apps, which became an increasingly time-consuming process. Manually doing this would take our team around 7-9 hours to complete. This began to affect our ability to deliver and ship new features to our customers because we were obsessed with ensuring our products were always exceptional. This was when we knew it was time to trigger automation.
We had three different systems to automate:
- Web Applications
- Mobile apps, which includes both Android and iOS
- Backend APIs for both Mobile apps and Web apps.
Testing all the above applications manually took us more than 8 hours per team member to check the flow after changes were made in the sprint.
What we are doing now
Given the above challenges, we continued to research the best practices and tools for our product and our company. We dedicated a lot of time to experiment with different tools and frameworks because testing and experimenting are what drives MUNCH:ON’s technology beyond the status quo, with the tech team’s motto being “failure is the best teacher.” Initially, our team worked on Selenium, Katalon Studio, WebDriver.io, Appium, Protractor, Cypress as well as custom hybrid frameworks. Through this experimentation, and because our automation team was experienced and well-versed in multiple tools, we came up with a cross-platform hybrid framework for mobile automation using Appium and Cypress for our Web E2E and backend services to suit our product’s automation needs.
1. Tools for Web (E2E) and Backend Services Testing:
There were a number of reasons why we were interested in Cypress. Cypress was a new tool in the market and was making a big impression in the automation industry. It was made for modern browsers and for modern web app frameworks like React.js, node.js, Angular, Next.js. It addresses the key pain points developers and QA engineers face when testing modern applications. It is a complete solution to automating our user frontend and backend API layers. This is one of the biggest reasons why Cypress is so powerful, it can handle the full stack automation development cycle of any product.
2. Tools for Mobile (E2E) including iOS and Android Testing:
We dedicated a lot of time to experimenting with different tools and frameworks for mobile E2E testing such as Testdroid, ExperiTest, TestComplete as well as custom hybrid frameworks. Through this experimentation, we came up with a hybrid framework to suit our product’s automation needs. After spending a couple of weeks coding custom Appium framework, using Java combined with the building power of Maven, TestNG, the team was confident that we could use this for the majority of the cross-platform mobile-side automation. Cross-platform mobile automation means that we can test two platforms i.e Android and iOS with a single test case.
Test cases under all the pipelines of web, mobile, backend services are running in the bitbucket-CI/CD environment for continuous testing, whenever the developers make changes and push the code to the main branch, all the test cases for both markets will automatically start running. These results then further integrate with TestRail to track our test cases and test suites. Our pipelines are also linked with Slack in order to update the team about the status of the running suites.
Automating the end-to-end test cases for Web and Backend Services
We have started working with Cypress and automated our test cases for the web application. These test cases covered almost every critical case.
Example: Login Test case for Web (an example of what Cypress’s code looks like)
Example: Simple API Test Case:
The above image shows the code written in Cypress to get an API response, and when the response is retrieved the assertions are made on the response of the API. These assertions are used to check whether the response from the API is the same as we had expected or not.
Automating the end-to-end test cases for Mobile Android/iOS
As mentioned above, we created a platform using hybrid tools (Appium), which can be used as a cross-platform for testing. Automation scripts cover both iOS and Android platforms with the same codebase verifying every desired flow. The approach used in the mobile-side is the PageObject Model (where every page has its own specific code and works independently from other pages). This approach makes the automation framework comprehensive, independent, and friendly.
Example Test Case:
- Automated 75% of user webapp, mobile apps, and backend APIs.
- 100% identification of backend and front-end bugs via continuous execution of pipelines.
- It now takes around 35 minutes for the entire automation test cycle to run, which has greatly reduced our sanity testing time ( vs. 7 hours with manual testing).
We’re always looking to refine our process and improve our testing speed so we’re continuing to develop our automation features. Stay tuned for our next detailed blog article about how we achieved a consumer-driven contract testing for our microservices!
Written by: Mehran Shafqat, Aqsa Manzoor, Hassan Iqbal, and Furqan Shah.