Nationwide Edge Datacenter Buildout Positions MSOs for Big Leap Forward

Tom Rutledge

Tom Rutledge

Distributed Cloud Architectures Will Drive Next-Gen Business Models

By Fred Dawson

November 27, 2017 – The largest U.S. cable MSOs have quietly built a springboard to the future anchored by an advanced edge infrastructure that could soon put them well ahead of competitors in the battle for residential and business customers.

Executing construction initiatives backed by Comcast, Charter, Cox and other entities, datacenter specialist EdgeConneX has built upwards of 40 video-optimized edge datacenters across the U.S., according to Phill Lawson-Shanks, chief architect and vice president of innovation at EdgeConneX. “These POPs (points of presence) are now the closest edge locations for 70 to 80 percent of the populations in these areas,” Lawson-Shanks says.

Their primary use so far has been to bring peering points between the Internet and ISPs closer to users in metro regions, some of which traditionally relied on long-haul private fiber links to peering points farther away. These are large multi-use facilities which host public CDN (content delivery network) services sold to Internet content providers while serving as private on/off ramps for cable operators in their dealings with Internet content suppliers.

New Cable Strategies

But, from the outset, cable operators backing the edge datacenter initiative had in mind another purpose as well. In contrast to strategies that dictated industry-wide headend centralization over the past ten years, new cable strategies require coordination between centralized and edge cloud facilities to execute on much-discussed IP-based transformations of their businesses.

Speaking of Comcast, Lawson-Shanks says, “They realized that with Xfinity they were going to need extremely low latency in the interactive data exchanges with customers.” The new edge datacenters are designed to accommodate much higher power loads for processing and cooling than a typical local headend can handle, which is essential to supporting the massive amount of data processing with minimal latency that allows operators to personalize and monetize user experiences across all services.

MSOs are beginning to move headend gear into these facilities, including new CCAP (Converged Cable Access Platform) appliances, with an eye toward migrating all processing onto the cloud at central and regional locations. How that workload is partitioned will vary from one MSO to the next, depending on specific service architectures. For example, Lawson-Shanks notes, “Charter does more transcoding in the edge datacenter while Comcast with its all-Xfinity approach to distribution does it centrally.”

While some MSOs have virtualized their stacks using virtual machine engines to operate transcoders, cloud DVR and other applications, they haven’t yet begun taking advantage of the dynamic shifting of workloads enabled by virtualization to maximize efficient utilization of hardware resources. “That will come,” Lawson-Shanks says.

These new edge datacenters are essential to executing new cable business strategies. A case in point is the strategic vision outlined by Charter CEO Tom Rutledge at the Society of Cable Telecommunications Engineers Cable-Tec Expo in Denver last month.

Declaring “We’re not in the video business,” Rutledge asserted Charter’s mandate is to optimize consumer choice with superior performance through sale of “capacity, connectivity and security.” He said this requires “advanced intelligent software” that can leverage a “high-capacity, low-latency, high-compute network” connecting consumers over wireless access points in and outside the home.

A Worldwide Trend

U.S. MSOs’ buildout of edge datacenters parallels efforts by other network service providers worldwide. Some of the other network operators investing heavily in this edge agenda include AT&T, Oath (Verizon’s new amalgam of Verizon Digital Media Services, AOL and Yahoo!), Liberty Global, Sky and several dozen telecoms participating in Ericsson’s global Unified Delivery Network (UDN) initiative.

Many are acting not only to buttress their own B2C service strategies but also to create new B2B wholesale opportunities through video-optimized CDN support for OTT providers’ service flows and dynamic advertising. So far, Comcast, through its Technology Solutions unit, is the only U.S. cable operator offering a wholesale CDN service, which was recently expanded to provide support for live direct-to-consumer services from TV programmers.

Demand for such facilities has triggered an outpouring of virtualized edge datacenter solutions from industry vendors that vastly expand on the capabilities of traditional CDNs, performing processing of content in support of virtually any live or on-demand multiscreen service model. Dynamic, targeted ad insertion, personalization of user experience, adherence to local blackout policies, support for time shifting and feature enhancements of every description can all be performed at the edge, leaving just a short hop for streaming at quality levels suited to display on big screens as well as handheld devices.

For example, Cisco Systems’ Open Media Distribution “cloud-ready” CDN platform, utilizing the company’s Virtualized Video Processing (V2P) technology, combines proprietary innovations with open-source CDN software to perform all the core elements of CDN management, including request routing, load balancing, caching, multi-format ABR (adaptive bitrate) streaming, CDN operations and viewing trend analytics and other advanced capabilities of the video-optimized CDN. As previously reported, Nokia and Imagine Communications are in the hunt as well.

Nokia, which became a provider of CDN technology by virtue of its acquisition of Alcatel-Lucent and that company’s previously acquired Velocix platform, is heavily promoting advances supporting TV caliber delivery and monetization of OTT. Imagine is touting a suite of edge components supporting just-in-time packaging, manifest manipulation and other advanced capabilities associated with ABR technology.

All three vendors say they provide software support for converting content delivered in IP mode to the edge to UDP/MPEG-2 transport mode for delivery over legacy pay TV conduits. By enabling operators to unify delivery of video over IP from the core to the edge, these solutions eliminate the need for legacy encoding, DAI and other hardware in regional and central headends.

The EdgeConnex Strategy

EdgeConneX’s role is to build and maintain high-grade colocation spaces for whatever platforms customers choose, as long as equipment measures up to company standards. Its investors include Comcast Ventures, Charter, Cox, Ciena, Brown Brothers Harriman, Providence Equity Partners and Akamai, according to industry reports.

Akamai has a strong CDN presence in many EdgeConneX facilities, Lawson-Shanks notes. Many other non-cable entities are using the facilities as well.

But cable utilization has been a focus from the outset. The first edge datacenter built by EdgeConneX provided support for Comcast operations in the Houston metro area starting about four years ago, well before the MSO introduced wholesale CDN services. As managers explored options for facilities locations there and elsewhere they realized it would be necessary to build new structures or convert existing buildings rather than use cable headend facilities, Lawson-Shanks says. While typical datacenter power requirements average around 7-10 Kw per cabinet, the types of servers required to handle processing for the video market require on the order of 20-30 Kw per cabinet, he notes.

“We found we could secure a warehouse or other type of building sitting on fiber where costs are relatively low and retrofit them as edge datacenters,” he explains. “Salt Lake City was next. We developed a whole new process to design buildings to absorb heat and operate securely without a need for people on premises. This became the cookie cutter we could use everywhere.”

With the template established, Lawson-Shanks says EdgeConneX was able to transform old buildings into 15,000 square-foot spaces suited for advanced datacenter operations in a matter of months, resulting in construction of 23 sites over the next 24 months. To streamline maintenance across these automated facilities EdgeConneX created the software platform EdgeOS to provide visibility to all buildings “through a single pane of glass.”

Now, with 40 sites operating in the U.S., EdgeConneX has expanded to other countries and is also creating much larger facilities suited for housing video-optimized datacenters here and abroad to serve as aggregation points for the booming OTT industry. Where the typical edge datacenter built for cable operators draws 2 megawatts of power, the larger datacenters can handle up to 8 Mw.

“We just announced an 8 Mw facility in Toronto and one in Buenos Aires,” Lawson-Shanks reports. “And we have campuses in Dublin and Amsterdam. Elsewhere in Europe you’ll see more traditional two-plus Mw edge sites in cities like Brussels, Milan and Manchester where they don’t have primary Internet peering points.”

The Charter Example

All of this activity points to what could be a very sudden transformation in what consumers are accustomed to getting from cable operators as the shift to DOCSIS 3.1 broadband capacity is completed. The capabilities of what Rutledge called the “high-compute network” allow operators to shift their focus from creating, bundling and marketing services that run separate from what’s available over the Internet to enabling blended services tailored to each subscribing household’s or business’s needs.

“We’ll be able to offer high-capacity two-way interactive products with massive bi-directional gigabit capabilities serving consumers and small businesses all on a single network,” Rutledge said. When it comes to video, Charter will integrate OTT apps and services like Netflix, Hulu and SlingTV “into our UI to make it easy for people to find what they want,” he said. He made clear that by expanding choice, Charter will contribute to breaking up “the big fat video package.”

Along with expanding the connectivity options within the managed service domain Charter will make security a primary focus in its pitch to residential and commercial customers. Under this new high-profile approach to security Charter will protect the privacy of its customers, add protections against theft for any content under its managed service control and protect products of other suppliers from theft through sharing of subscriber passwords, Rutledge said.

Like CDN services, aspects to this security strategy represent forays into new B2B monetization models enabled by the new network architecture. Speaking of the need for protections against password sharing, Rutledge said, “It’s a significant issue and it’s not well appreciated by the people who are new to distribution,” which he characterized as anyone with an app, including programmers operating in OTT mode. “They have an obligation to protect their product, and they don’t do a very good job, and that’s affecting the business,” he added.

The new intelligent networking platform also has major implications for operations, Rutledge noted. Charter personnel will have visibility into network performance and customer accounts through uniformly configured UIs across all its properties, including recently acquired Time Warner Cable and Bright House systems. “As the customer experience gets better, the average life of the customer relationship goes up and costs go down with less failure,” he said.

Lower costs mean more money can be spent on increasing value for customers. “It’s a virtuous circle,” he said.

The Cable Wireless Agenda

Rutledge also stressed the role wireless will be playing in Charter’s future. There are now some 200 million devices connecting to Wi-Fi in Charter subscribers’ homes with 80 percent of all subscribers’ mobile network traffic running over these Wi-Fi connections, he noted.

He said Charter will implement next-generation Wi-Fi over 802.11ax access points, which will expand premises wireless networking capacity from eight simultaneous streams in the home to 64. With the launch of branded mobile service as MVNOs (mobile virtual network operators), Charter and other MSOs following a similar Wi-Fi-first path will be fully invested as wireless providers in their own right.

“I’m calling it 6G,” Rutledge said, leaving open the possibility that Charter could acquire licensed spectrum to complement operations over unlicensed spectrum in a move to MNO (mobile network operator) status. Noting cable’s high bandwidth reach all the way to small cells wherever they’re positioned, Rutledge characterized 6G as “something we have and the phone guys don’t.”

The idea of adding licensed spectrum, which would enable deployment of 5G small cells, brings into play another aspect to the evolving edge datacenter scenario. “The next step for us is 5G,” says Lawson-Shanks.

Noting the need to aggregate massive amounts of traffic generated by small cells, he says EdgeConneX is working with operators to set up micro-datacenter facilities deeper in their networks to begin testing real-world requirements for utilizing 5G technology. These refrigerator-size or even smaller containers will be co-located near pole-mounted antennas communicating with a cluster of small cells.

The forthcoming trials will set parameters for massive quantities of micro-datacenters that will be deployed in the years ahead, Lawson-Shanks says, noting that how 5G will work in the real world has yet to be determined. “How many people can be served per micro location will be defined based on testing of 5G antennas to end users in multiple scenarios,” he says. “Right now all we have to go on is lab tests and some very limited field tests.”