‘DevOp’ Mandate Sets High Bar For Automating QA in Pay TV

John  O’Brien, CEO, S3 Group

John O’Brien, CEO, S3 Group


New S3 Group Innovations Aim to Help Operators Cope with Increasing Complexities

By Fred Dawson

April 27, 2014 – As the once slow-moving manual testing processes associated with getting new set-tops into the field morph into a perpetual scramble to keep up with service expansion across an expanding portfolio of devices, the need to automate testing is obvious.

The question is whether automation of testing processes can deliver results that reflect true performance under real-world usage conditions on every device across every type of service category and application both before and after field deployments. As the pace of service innovation accelerates, certainty that subscribers are getting the quality of experience service providers expect is absolutely critical to preventing maintenance, customer service and replacement costs from soaring out of control.

“Before any feature change is made to an existing TV service, or any new service is introduced, it’s imperative that rigorous testing occurs to ensure that subscribers have a quality experience,” says Jason Blackwell, director of service provider strategies for the Digital Consumer Practice at Strategy Analytics. “Pay TV operators must now support set-top boxes, tablets, game consoles, smartphones and other devices, which requires managing a dizzying combination of possible software configurations and hardware platforms. As such, there is growing need for enhanced service validation solutions that provide the efficiency and reliability required to keep pace.”

One company focused on providing the MVPD marketplace the tools needed to accomplish these goals through highly automated processes is S3 Group. The Dublin-based company says its StormTest Development Center, a test automation platform that accelerates end-to-end delivery of multiscreen TV services to a wide variety of device platforms, is used to validate services delivered to more than 140 million viewers in 28 countries, including North American customers of Comcast, Mediacom, Time Warner Cable, Videotron and others.

“Test automation supporting engineering development and the continuous flow of new services and devices into the marketplace has become an industry priority,” says John O’Brien, CEO of S3 Group. Or, as John Maguire, director of strategy and marketing at S3 Group, puts it: “The distinction between operations and development is almost irrelevant.”

In other words, we’re now in a “DevOp” world where there needs to be seamless integration of testing processes and analysis from the development phase through commercial rollout to refurbishment of CPE returned from the field, Maguire says. Digital TV devices in the home are exposed to performance conditions that can be hard to replicate in the usual testing timeframe, he adds.

StormTest Development Center addresses this issue by simulating real-world usage conditions, enabling several generations of customer premises equipment to be simultaneously stress and reliability tested, along with capturing key performance metrics, Maguire says. Up to 64 devices can now be simultaneously tested on the platform, he adds.

The latest enhancements to the platform include expanded reporting options that use data generated by test automation to help development, operations and product management personnel assess the performance and service readiness of video services across connected devices. S3 Group has also enhanced HD and 4K image handling with a migration of the underlying StormTest software architecture to 64-bit processing. And it has streamlined test development with an improved “Navigator” graphical user interface, which the company says saves up to 30 percent in the test development effort by simplifying development and maintenance processes associated with test navigation frameworks.

“We’re seeing a massive acceleration in the complexity vector with traditional and new services like network DVR and catchup TV delivered to every type of device,” says John O’Brien. “Combining everything – content sources, device types, transport network – into a single complexity metric, we project a 400-fold increase in complexity over the 20-year period from 2000 to 2020.”

Moreover, time to market is accelerating as the complexity increases. “In the new DevOps model operators have to move new iterations of their services from development to deployment in a matter of days,” O’Brien says.

Echoing the point, Stefan Kastl, director for CPE at Sky Deutschland, says, “As we evolve our digital TV platform on a daily basis it is important that we can execute stress, reliability, performance and monitoring testing efficiently on a large number of set-top boxes simultaneously and with different releases at the same time.”  S3’s StormTest Development Center has given the company the ability to meet this goal with testing across all its DTH and cable set-top boxes, he adds.

O’Brien cites two other customers, BSkyB and an unnamed tier-one operator in Europe, which have begun implementing the latest components of the StormTest suite. “Sky is streamlining quality assurance across all phases of service development and operators, including CPE integration, screening, product launch and CPE screening from the field,” he says, adding that S3, along with supplying test automation support, was also instrumental in helping to define the unnamed European operator’s next-generation video platform.

In the case of BSkyB, along with extending test automation and CPE screening, the operator has tapped S3 Group’s services to assist with EPG development and CPE system integration, O’Brien notes. “They’ve seen significant reductions in faulty product reaching customers along with about a 55 percent savings in efforts expended on testing,” he says.

Extending S3’s automated lab-testing processes with the same level of thoroughness and at scales suited to monitoring the quality of millions of users’ experience is a top priority, Maguire says. “Network assurance is a solved problem,” he notes. “But the complexity of the multi-source and multi-targeted device environment poses a huge problem when it comes to assuring quality of experience at the app and service level.”

To extend the quality assurance monitoring capabilities of the automated testing platform to ongoing field operations S3 is exploring both hardware and software solutions, Maguire says. “Operators want to extend our automation environment as new devices go into operation,” he notes. “From a cohesiveness point of view what we’re doing at the field level is similar to what we’re doing in the labs with similar time bases and granularity. It’s a natural step from where we are today.”

Responding to rising demand in North America for its services, the company has opened a new office in Philadelphia, complementing its West Coast facilities in San Jose. As previously reported, S3 has taken a more prominent role in the U.S. market as the provider of code management and testing services for the official Reference Design Kit (RDK) code repository of RDK Management, LLC, the joint venture set up by Comcast and Time Warner Cable to create an open template for next-generation pay TV devices. This has become an ever more complex operation as new versions of RDK add new interfaces, functionalities and device types to the platform and as growing numbers of operators and manufacturers sign on to the initiative.