Watershed World Quality Report illuminates growing importance and challenges for testing teams

Each year for the past five years, I’ve had the opportunity to engage with and review an insightful and rather amazingly comprehensive set of research. This research focuses on the adoption, practice and maturity of software quality across the world.  This year, the wait was worth it as I just got my hands on the fifth annual World Quality Report orchestrated and published via collaboration between Cap Gemini, Sogeti and HP. This rather weighty piece of analysis (61 pages in total) represents an exhaustive study of over 1,500 professionals across the globe conducted via 40 minute telephone interviews in seven different languages. and is bursting with useable and enlightening insights.   

You can download the full report here.

 

 

WQR.png

 

From this information, several key trends surfaced that validate much of our on-going interactions with all of you—the QA, testing and ALM teams in organizations worldwide.  Some of the most illuminating insights adjust our thinking in terms of the vital role quality plays for the business in the delivery of software, and that means success for organizations across the globe.  Take a short journey with me as I highlight, summarize and editorialize some of the attention-getting insights from this world quality view.  This is part I of a three-part blog on insights from the World Quality Report, or “The state of the Quality nation”.

 

We hold these truths to be self-evident-- IT Macro Economic factors that impact quality

 

No matter where you work in an organization today, you can’t get away from the impact of the global economy.  Slow economic recovery continues to increase pressure on organizations everywhere and they are re-evaluating the basic components of IT operations, including the QA/testing function.

The report highlights three interrelated IT business drivers are influencing today’s corporate IT agenda regardless of region or industry:

 

  1. A renewed focus on customer experience—at HP we call this “user-centric IT”
  2. Increasing globalization and cross border trade – It’s truly a world without borders and this means more rapid changes to business processes and the applications that support them.
  3. An expectation that IT departments take on more business accountability –this drives the need for more collaboration between IT and the business and well-designed metrics that allow IT and QA to show their value in business terms. 

Given these drivers—there is no surprise that Quality is becoming even more critical to the business. Quality directly impacts customer experience, business process agility and reflects directly on the brand.

 

Out-Of-Quality-Business-Sign-S-8440.gif

 

So how is QA responding to the conflict of economic pressure vs. increased strategic business impact? 

 

From this research—QA and testing are maturing at a faster rate than before and are adopting best practice and standardization to ensure efficiency and effectiveness. The research clearly shows that QA is developing a more structured and methodical approach to its role within IT and is not shying away from owning and ensuring quality standards of business systems.  I find it interesting that while the industry is enamored with speed and the pure Agile aspect of self-managed teams, the enterprise realizes that quality doest’t just happen. Quality  must be supported with expertise, efficiency and the repeatable application of best practices.  Interesting also, is the realization that while standardization and best practices are a goal, it is also critical to grow QA skills to gain specialist expertise for the unique requirements of new technologies and transformations. 

 

QA success:  Standardized yet specialized

 

QA is streamlining and centralizing to get efficiency and cost optimization. Testing as a single stream across the entire org is being used by 26 percent of respondents, up from 8 percent in 2012 Organizations with a fully operational testing Center of Excellence (COE) is up to 19 percent from 6 percent last year (a three-fold increase)

 

But not all COE’s are vanilla:  While COE’s become efficient with standards; they also develop greater business and domain knowledge with 63 percent citing this as an important capability to add value to test execution. 

 

And augmenting testing resources with external teams continues to be a proven method to attain skilled resources:  Using a managed test service has increased from 13 percent to 20 percent.

 

The business realizes quality is essential and is walking the talk with budget:

 

The report has good news for QA in that businesses are increasing their proportion of their overall IT budgets. The amount allocated to quality increased from 18 percent in 2012 to 23 percent in 2013, and is expected to increase to 28 percent in 2015, underlining the opinion that quality is critical to the brand. 

In addition, a higher percentage of this budget is allocated to transformational projects instead of business as usual, which could indicate that we are able to maintain quality of existing IT systems with less effort due to adoption of test automation and improved regression testing. 

 

One key point made in the report, however is that QA and testing leaders must not rest on their success. They need to drive further efficiencies since on-going year-on-year budget increases are not sustainable.

 

But not all is quiet on the testing front:

 

While this sounds positive, however, the report also shows many businesses still engage in what could be described as reactive testing practices, involving QA too late in the delivery lifecycle.  

 

Testing is still too late to the party:

 

Nearly half of the organizations interviewed (45 percent) start testing during or after the development phase—too late to influence app quality beyond dealing with defects.   And 61 percent say they do not have plans to introduce quality earlier in the lifecycle.  

 

Metrics matter:  the analysis shows teams still rely on commonly used operational quality metrics—such as number of defects found and cost per test case executed.  While necessary, they are not sufficient.   Testers could be doing more to demonstrate tangible business value such as quantifying testing contribution to reduced time to market. 

 

So what does this mean?  Here’s my summary:

 

  • The good news is that the business is getting the value of quality to their customer experience and their brand, and voting with their pocketbooks
  • QA organizations are taking maturity seriously and are leveraging proven best practices such as adopting CoEs, instituting test automation and working to source additional domain expertise to better align with business and prepare for application transformation and new technologies
  • But we still have work to do to drive testing earlier in the lifecycle. We need to align testing teams with the teams defining user stories and requirements. We also need to ensure that dev/test collaboration and tools can help with ALM and Agile management platforms that help deliver visibility and traceability between the business analysts, testing teams and developer teams across the organization.
  • And there are still ample opportunities to increase efficiency, most noticeably in test lab management and automation test asset sharing for the COE and the continued application of automation for functional, performance, security and mobile testing.

 

What’s next?  The report goes on to highlight four other key trends each of which is meaty enough for another discussion:

 

  1. Mobile testing-- is increasing in importance and a key discipline but lacks specialized methods, expertise and environment. The report shows it is carried out by 55 percent of organizations up from 31 percent last year.   Read about HP mobile testing solutions here.
  2. Cloud-based testing and overall cloud adoption – some issues but continued progress--  the report shows this year, adoption has slowed somewhat as organizations continue to manage concerns about data security and performance.  But, overall executives remain positive on the growth in cloud based apps.
  3.  Test environment management continues to challenge-- While 40 percent of testing and QA budgets are allocated to test infrastructure and hardware and 28 percent is spent on testing tools, 67 percent of respondents say they don’t have the right or enough tools to support their test environment. This is all  despite the recent investment and 53 percent say they  struggle to maintain multiple test environments.
  4. Agile development is now widely adopted but still has problems for testing, especially with specific methodologies and expertise—83 percent of organizations report they use Agile methods for some or all of their app dev.  However 46 percent claim a lack of a consistent testing approach for Agile.  More to come on this timely subject in a follow-on blog.

You can read the press release hereand you can download the full report here and continue the conversation!

Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the Community Guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
Search
Showing results for 
Search instead for 
Do you mean 
About the Author
Kelly has over 20 years experience with enterprise systems and software in individual contributor and manager roles across product manageme...
Featured


Follow Us
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.