Webomates: Delivering Software Testing Results within 24 Hours

Follow Webomates on :

Aseem Bakshi, CEO

With an app for everything, today’s rapid digitalization demands software development teams release new products and technologies to the market quickly. In this race against time amid intense market competition, development teams lack the capacity and manpower to scrutinize their products for errors and bugs. Born out of this need, Webomates, a provider of cutting-edge SaaS-based testing services, began delivering comprehensive regression testing solutions to expedite an organization’s software testing process. Webomates helps customers innovate products with a reduced time-to-market.

In an interview with CIO Applications, Aseem Bakshi, CEO of Webomates, discusses the company’s inception, growth, milestones, and research and development in the field of software testing.

What sparked the ideation behind Webomates?

Webomates was created out of our own need. The founders of this company are people who have a strong development and testing background and have been in R&D companies across the U.S. A fundamental problem that we often witnessed was whenever a new version of software had to be released, justifiably, the rest of the company would not display much confidence in the quality and performance of the application and hold the R&D teams accountable for any mishaps post the release. So, the genesis of Webomates was our answer to the question, “What would be the perfect solution to help a team release a product in under 24 hours?”

What challenges do you see in the landscape and how does your company address them?

There are four direct value propositions that we provide through our Webomates CQ methodology.

1. Guaranteed Execution Time: After building the software and preparing to ship it, the development team requires a quick feedback loop to significantly increase their confidence in the latest build. This leads to the first statement in our Service Level Agreement (SLA) that says we will finish our testing in less than 24 hours. Webomates actively commits to an SLA that any number of test cases on any number of browsers, mobile platforms, or environments will be executed within a day’s time.

2. Guaranteed & Auditable Execution of all Test Cases. Test case execution reports suffer from two major types of flaws that cause incorrect results of 40 percent to 100 percent of test cases failing in an execution. These are:

a. Tests that are classified as failed even though the test case should succeed. For example, automation systems have timeout errors that can occur that indicate test failure. However, the timeout is caused by an automation system issue, not an actual test case failure.

b. Modified test case failure: The software build has modifications for feature changes or bug fixes that result in modification of the test case. For example, say the font of some text changes. This can result in a test failure. However, this was an intentional change.

So, the team and I devised the Webomates Execution Guarantee, which states that if a test case is modified, we still ensure that every single test case will be executed.

Webomates actively commits to undertake a service for as many test cases on any number of browsers, mobile platforms, and native Windows

This is huge because when a tester is handed a new software build, there are always changes in the build; by definition, this is why the new build exists. Depending on the bug fixes or the addition of features, the resulting modified software is subject to many changes in the test cases. Regardless of this, Webomates remains true to its guarantee that every single test case is a valid positive or failed test case thus significantly decreasing the “noise”— the false positives or false negatives.

3. Triaged defects, not just Pass/Fail reports. Receiving a Pass/Fail report that says, “out of 1,000 test cases, 700 passed and 300 failed,” does not thoroughly answer many questions. What the clients want to know is, “do we have any bugs? If so, what is the priority?” Frequently, one defect can cause multiple test case failures. This is why we provide:

a. A set of detailed pass/fail reports

b. A prioritized list of defects that allow our customers to have a clear visibility into the problems and bugs within their application. These defects even have videos showing the defect occurring!

4. Usage-based testing: Lastly, unlike most existing testing solutions, which are driven by either licensing models or a Time and Material (T&M) model, we charge clients by the test case. If the client wants to execute only 100 test cases a month, there is just a linear increase in the cost if they increase the frequency. This AWS-like usage-based business model ensures no confusion around how the process is structured.

Could you elaborate on Webomates CQ?

Our patented testing service, Webomates CQ, is completely horizontal and requires minimal domain or functional knowledge from the teams. By utilizing the black box method for regression tests, we ensure the tester needs to only look at the system from an external user’s perspective and validate the responses. This ultimately results in achieving accurate reports of the tests with which the software product can be improved concerning its UI and APIs. We do not see the internal workings of the code and only check if the UIs and APIs are responding in an appropriate fashion. What we do is request the URLs/ mobile version from the customer and go into the system and validate the output and the input values. In fact, 80 percent of the test cases are written even before conversing with the customer. The solution fundamentally improves the productivity of the software teams that utilize the system. Furthermore, we also offer overnight, 12-hour testing, which is a guaranteed execution based solely on a test case and smoke tests for a subset of test cases. In this overnight regression, all test cases pertaining to a single module are executed. This provides an agile development team, working on the same module in the day, the confidence that all the baseline test cases for the module are working. Smoke tests, on the other hand, are subsets of test cases that are completely automated and can be run at any point in time by the customer themselves. Even better, these can be executed in 15 minutes, regardless of the number of test cases. These tests are structured to be end-to-end as well as module focused.

How has your solution helped a customer in the past?

Priceline, a leading online travel agency, was looking to increase their productivity and reduce software development costs while maintaining their standard of quality across multiple browsers, operating systems, and mobile platforms. With Webomates CQ regression testing, Priceline was able to identify defects within 24 hours of creating the build. Our solution provided the developers at Priceline with the ability to continuously monitor their software for bugs which enabled them to decide on how to deploy the build into production faster and cost-effectively. This resulted in over 15,000 test executions on four platforms in three months, where Webomates CQ identified 168 defects. 38 of these defects were business impacting Priority 1 defects that Priceline stated saved them thousands of dollars.

What does 2019 hold for the company?

We are planning to expand to Europe by the start of the second quarter. Subsequently, we are leveraging a concept called shift left in development. We are adding a CI/CD offering. We intend to continue delivering speedy 24-hour testing and execution services for our clients and utilize AI across all our solutions and systems. In addition, we are starting to offer our own patented tools that we have created for R&D teams. This toolset called QA Assist significantly increases their productivity. In other words, we train teams on our tools so that they can achieve the same productivity boosts that we have seen by using our own tools.