Software testing as a function until recently had not developed the level of maturity seen in the developed markets of US, EU, etc. The focus was mainly on functional testing with very limited attention given to other types-automation, performance and other non-functional testing. This is all changing and we now see a lot more interest in testing and greater extent of automation due to increasing adoption of IT, growing importance of quality and awareness & focus on different types of testing. Dataquest spoke to Rajesh Sundararajan, Practice Head, Testing Services, Marlabs Software. Excerpts
Breif us about cloud based software testing.
As we all know, ‘cloud computing’ refers to the utility based model of consuming shared computing resources as a service over the Internet. Let’s see what it means in the context of software testing.
There are 2 primary categories of computing resources that can be accessed over the cloud:
Application Test Environments: The test environment infrastructure can be provisioned on the cloud with the appropriate configuration for the required timelines rather than have a dedicated infrastructure in-house. The application under test is then hosted on this cloud infrastructure and tested. This is an example of IaaS (infrastructure as a service) or PaaS (platform as a service).
The Testing Tool Rig: which includes the Testing tool (Automation, Performance etc.) software and/or infrastructure required to host the testing tools – can be provisioned and accessed from the cloud. This can be SaaS (Software as a service), PaaS or IaaS depending on the Test setup. This testing tool can test applications hosted on either cloud or dedicated in-house test environments.
What are the benefits of testing and running enterprise software in the cloud?
The benefits if planned well are significant.
Cost is a primary factor:
I.Large upfront costs of capex like servers & other hardware, testing tools are replaced by opex which are spread out over a period of time.
II.With cloud, there is value for money as we pay only for the resources that we consume. The testing life-cycle is often intermittent and is a good candidate for utilizing the cloud. At the end of testing, the cloud based resources can be stopped and restarted when testing resumes to avoid accumulating higher costs.
III.Some activities like performance testing require large production-like configurations for short cycles. Creating this environment on the cloud is an attractive proposition for IT organizations which otherwise often avoid performance testing due to prohibitive investments.
Dedicated physical infrastructure requires a lot of time to procure & setup. That is where the cloud provides an option of – quick provision of the required environment, a set configuration which can be recorded and applied at short notice. It is also very easy to scale up the environment and provision additional resources whenever required.
There are other interesting possibilities. The global distribution of the cloud enables us to run ‘distributed’ performance tests and generate load from multiple regions across the globe to simulate live usage patterns
What are the risks or downsides?
The cloud comes with its fair share of risks and users need to comprehensively evaluate the relative benefits it offers.
First of all, Organizations face a risk in giving control of their assets to an external entity
a)The data and resources are not in your physical control. The responsibility for the data and its safety is with someone else.
b)Business continuity and disaster recovery are completely in the hands of the cloud provider.
c)There is a dependency on the cloud provider for SLAs and other support activities.
But there is another side to it- many organizations especially those lacking in technical expertise and dealing with less critical data may actually be more comfortable with leaving security and infrastructure challenges to the expertise of Amazon EC2, Microsoft azure, Google, etc.
Compared to the live production, the test environment is a less risky option when testing the waters with a movement to the cloud. So, organizations are more open to moving test environments to the cloud
A very well know issue is that of security. The cloud brings in additional dimensions which impact security:
a)Multi-tenancy: the cloud is a shared resource and there are risks, known/unknown of data leakage to other users sharing CPU, Memory, and Servers etc.
b)Virtualization-While the vulnerabilities to the underlying physical resources remain, the additional vulnerabilities of the virtualized environment pose an added layer of risk.
c)Data stored on the internet especially in large volumes on the cloud are prone to cyber-attacks, having said that most cloud providers do have stringent security measures in place.
How does a company pick a strong test tool?
There are tools for a range of testing activities. For each type of testing, there are many tool vendors who have come out with products. Many of the vendors provide tools for end-to-end test activities. Choosing a tool or tool platform is very much context driven depending on:
Application technology: Web applications are supported by most tool vendors. Applications based on niche technologies have limited tool options for testing.
Cost of the tools- This is relative and depends on the size of business, application criticality, company policy, etc.
Availability of skill set , ease of use, learning curve of the tool.
Extent and quality of support provided by the tool vendor.
Alignment with existing tools and technologies. An organization using UFT from HP may prefer to extend their partnership and select load runner performance testing tool from HP itself.
Two strong trends in recent years are:
The growing maturity and adoption of open source tools for a wide range of testing activities. Though these tools require skilled resources to use, their cost advantages make them a very good option for companies to choose.
Many of the tools are now being delivered in SaaS model over the cloud. This is increasingly becoming a criterion for companies to choose a testing tool for to gain the benefits of cloud.
What is the role of APM (Application Performance Monitoring) in cloud testing?
APM tools find usage in- test and live environments, in-house as well as cloud deployments. Major vendors like – Compuware and CA to name a couple have come out with products to address these use cases.
During performance testing, the APM tools provide a complete and in-depth view of system and application performance including:
Resource utilization at the level of server, application, user, transaction to help in better analysis, performance troubleshooting and capacity planning.
Analyze performance down to the method level, break-down performance bottlenecks and pin-point latencies at each layer of the architecture from the UI to the back-end database and service components.
This advanced analysis is especially helpful in the cloud context because of the dynamic, complex and multiple components involved in cloud architecture. This analysis is also very helpful in defining auto-scaling strategies for cloud hosting.
How do you see India as a market for software testing?
There are 2 parts to this. One is the software testing services market. The Indian IT services industry is expected to see about 14% growth to reach a market size of about $16 bn by 2016, according to a report brought out by BCG (Boston Consulting Group) & CII. This is mainly due to increasing adoption of IT by companies, more outsourcing- reflected by many of the recent big ticket IT deals e.g. indiaPost, Bharti Airtel and growth of new technologies like mobile, cloud, big data. So, the software testing market can be expected to grow on similar lines and the growth will be more in testing related to these new technologies.
The other component is the market for software testing products (tools). Apart from the above reasons, the fact that India has become a preferred outsourcing destination for software testing along with a large pool of technical resources has led to sustained demand for software testing tools. The visible presence of the large Product companies- HP, IBM as well as many of the niche players- Neotys, PerfectoMobile etc. at various forums is a testimony to the importance of India as a market for testing products.
Do you think adoption of this is still less in India as compared to other countries?
IT & software adoption has been lagging in India compared to some of the developed markets. For this reason, until a few years back the India market was not the focus for most of the software service vendors and the Indian market still forms a small component of India’s more than $100 bn software industry. Software testing as a function until recently had not developed the level of maturity seen in the developed markets of US, EU, etc. The focus was mainly on functional testing with very limited attention given to other types- automation, performance and other non-functional testing. This is all changing and we now see a lot more interest in testing and greater extent of automation due to increasing adoption of IT, growing importance of quality and awareness & focus on different types of testing.