Month: July 2014

Introduction

Our client is a leading solution provider to educational institutions in US. Their application technology is used at universities for students to enrol for different courses.

The Application has modules such as Dashboard, Survey, VIP, content and import. Students can have online chats with other contacts about subject topics and stay connected till the completion of course.

Testing Challenges

They were looking for an Independent QA Vendor to assist them to automate regression suite and bring down the regression cycle time to catch up with the sprint release.

Some of the key challenges they faced were test coverage, test efficiency, testing time and script maintenance.

Business Challenges

A QA partner to maintain accuracy, reliability and effectiveness of the application during enrolments, perform Load test the application to ensure maximum number of concurrent users and find other performance bottlenecks to improve the response time of the portal.

Our unique test strategy was instrumental in streamlining our client’s testing process in order to improve application’s functionality and performance.

Our Approach

We have devised an approach to meet both the ‘Testing’ and ‘Business’ challenges faced by our client.

• KT to understand the application functional features and architecture.
• Unique test design to ensure test coverage.
• Perform gap analysis and suggest a suitable Test Automation Framework
• Identifying test cases for Automation
• Created Check-lists to ensure the application functionality
• Adapt agile methodology and take part in Scrum Meetings
• Parallel QA team,  strategic overlap of time – for  real time collaboration
• Collaborative approach to performance test plan and implementation
• Test the application by scaling concurrent users for critical business transactions

Resource Growth over a period of time

Our Solution

The testing services we offered are listed below.

• Apply Yourselves: Functional Regression
• EMT- Connect ( Enrolment Management Technology- Connect) :  Performance Testing / Functional Regression / Accessibility Testing – ADA(American Disabilities Act- 508 Compliance)
• Constituent Relationship Management: Functional Regression and API testing / Security Testing
• Marketing Solution: Functional Regression

The main objective is to perform Automation & API Testing for their enrolment management application.

We have designed and implemented hybrid framework at business process level reliability, accessibility and dynamic execution of test suites on need basis.We created and upgraded a repository of re-useable test artefacts.

We also created a unified process and model for web application security testing and risk modelling. We used a licensed Rational Appscan tool through a VPN connected server to perform Vulnerability Scanning.

We have also recommended OS and database level parameters for subsequent production deployments for EMT and tuned session parameters for scalability.

Metrics:

Totally 1685 test cases were automated across all 4 application and 820 defects were reported during execution of the project.

We took complete ownership of regression script maintenance and used Rational Appscan and WebScarab for Web Application Vulnerability Scanning.

Our Value Adds

Business Level:

1. Business Value: Vulnerability Scanning and Accessibility Testing (Section 508 Compliance) were suggested by us in order to improve the PR and User Base respectively.
2. Total Cost of Testing: Client was using QTP. As the number of users increased gradually, the licensing cost was also increasing exponentially. We did a thorough tool assessment study to find an appropriate cost effective tool and suggested Selenium (open source tool) without compromising on quality. This eventually reduced their overall testing cost drastically.

Delivery Level:

We got a minimum time frame for gathering application knowledge our team gained more than 80% application knowledge in 2 weeks’ time and that made us to gain our client’s confidence.

Over a period of time, we were able to get control of smoke testing that happens on the production for every sprint release and on-going maintenance of automation scripts. Release cycle efforts were reduced from 4 weeks to 10 days.

Dedicated Test lab to provide end to end testing services

Are you sure about the Quality of the Product produced..?

To confess that your output is Right, You should be very confident on your Measuring System which produces the Result/Output.

So Gage Your Measuring System First. !!!

What can be defined as Measurement System??

Measurement System – is the complete process of obtaining measurements.  This can be

• Test Method / Procedures.
• Instruments/Tools used.
• Resources/People Involved.

Two Major concepts involve identifying the capability of Measurement System.

• Repeatability
• Reproducibility

*** Do you remember in “+2” Physics Practical Exam – were all of them used “Vernier Caliper” to measure the given object. Each time we measure the same object we’ve got different/similar measurement readings and the same for other students.

Similarly, the technical example given below will explain the above said method.

Let’s take this Scenario– You’re now reviewing the Standards of Java Test script written by your peer. Your task is to identify the defects. So you will be producing the defect list as an output.

Now answer this….

Are you producing the Same Result when you review the same Test Script Every time (Today/Next week)?? 

Track your outputs by doing 3 to 5 iterations of review to decide your ability.

• As a team we should assure that Measurer (Reviewer) is capable in repeating the accurate result every time he/she reviews. This is called Repeatability.

And Now…

Were All the Reviewers in the group are getting the Same Result when reviewing the Same Java Script?

Try out this analysis with the group of 3 to 5 Reviewers for more than 3 iterations of review. Track all the outputs produced by all the reviewers.

• We should ensure that more than or equal to 90% of reviewers produce the same and accurate results as other person do. This is called Reproducibility.
• If the team is scoring less than 90% accuracy then you need to re-do the analysis with the same set of people after completing the awareness/training session on Java Standards.

The above 2 techniques proves that your measuring system is good enough to produce a Quality Product.

The Process of Analyzing the Measurement System is called Gage R&R (Repeatability & Reproducibility)in Software Industry.

So What Next…!!! Gage your System Today to Improve Your Product Quality.!!!

Accolades to the team for the excellent work that was delivered with dedication, commitment & Team work. Thank you for all the hard work and for the initiatives implemented adding values to the product. 

The team’s ability to accommodate, even when it wasn’t easy or convenient (with no requirement document, not much detailed KT on the domain / product) assures us of the Quality of people with us, who are willing to make it big even with limitations/ restrictions.

A note from the Client: “Once again, I take this opportunity to appreciate you guys for your commitment, co-operation and support to make this exercise successful.”

How well does your application behave to numerous or Large number of user requests? Will it give up due to increasing load? Can it handle enhancements? Have you ever realized that your application lacks usability?

This is where Indium’s Software Testing Services comes into play. Our Quality Management Suite is a comprehensive solution that provides rich insight into application’s performance, functionality and its overall architecture. Let’s look at a case study that demonstrates our delivery capabilities…

Business Challenge:

Our client, a leader in providing comprehensive maintenance management software (MMS) solutions to the healthcare industry, revamped their current MMS application to reflect current design and technological updates, so as to improve usability and performance of the entire system.

As the application will be deployed across several hospitals, the system should not fail at any point.

As a result, the product required end to end functionality and performance testing.
We had to ensure that there are NO code breaks and NO new regression bugs in application on new builds.

Also there was a need to identify an appropriate functional automation tool that supports WCF and XBAP technologies.

Approach:

The objective of this project is to perform manual and automation regression for the maintenance management system.

We have adopted realistic and innovative approach for test automation and performance testing.

After understanding the application’s functionality, the business level scenarios were developed and the gaps in client’s existing test process were studied thoroughly. Performance test were also carried out for every product release.

Solution & Value Adds:

We supported 2 major releases each with 14 and 20 builds respectively. A total of 2118 test cases were executed per build.

126 automated scripts were developed and 148 existing scripts were maintained throughout the engagement.

We adopted an innovative method of utilizing the functional automation scripts for performance testing in distributed threads considering none of the performance testing tools thoroughly supported their application due to XBAP architecture.

Our strategy blend resulted in enhanced customer experience and provided suggestions to streamline their existing QA process. We also adopted an exploratory testing approach to uncover defects. Indium was proactive in identifying and reporting issues.

Performance testing team helped the client identify critical system bottlenecks prior to production and tuned the database parameters for optimization.

Indium started its services with manual testing and became their trusted software testing consulting services to offer other services such as Test Automation, Performance, Compatibility Testing, Usability and Mobile App testing (iOS).