WAN Op = NoOp : Part three : It’s the data

This is the third part of a series of posts on the background and current viability of WAN Optimization. Read the first two parts here and here.

In part two of this series we discussed how WAN Op provided the ability to take older WAN hostile applications and make them more WAN friendly, but as newer applications are designed to be WAN friendly the effectiveness of WAN Optimization is decreasing.

The data:

Another substantial benefit WAN optimization provides is data reduction.  Data reduction minimizes the amount of traffic transmitted across the network. Data reduction is particularly helpful for WANs because bits per second have been expensive.  Less bits means less need to buy more bandwidth, saving money. In general, WAN Optimization accomplishes data reduction with two approaches: data compression and data duplication.

Data compression is nothing new and most people have use zip utilities for decades.  Compression works best when it is compressing larger chunks of simple data that has a high amount of repetition.  Since the WAN Op appliance is intercepting and terminating the TCP and SMB sessions to create bigger block sizes for the WAN, it is in a prime location to compress data prior to transmission across the WAN.   Depending on the data, compression can provide substantial reduction of 80% or more.

In addition to compression, most WAN Optimization solutions use data duplication (DDUP) as well. DDUP works by remembering patterns of data within the data stream and storing copies of it on each appliance.  As data is transferred between a client and server across a WAN Op peers, the data is analyzed to see if the data contains any pre-stored pattern.  If the data does, the appliance will send only the signature of the stored data between the appliances instead of sending the actual data.  The peer appliance at the remote location sees the signature for the data and reconstructs the data from the local store.  If most of the data has been seen and is stored prior, the transfer is referred to as a warm transfer. If the WAN Optimization appliance has not seen the data prior, it will compress the data, transmit it to the peer, and store a local copy for future reference at each peer.  Transfers of this type are called cold transfer since the data does not yet exist.  DDUP’s beauty is that it is not file oriented like web caches of old but it is data oriented.  If you have a corporate logo in a power point slide, the same logo exists in a word document and a web page, it does not matter to WAN Optimization. The appliance will not have to send it across the WAN and can reconstruct it on the other side.  One document can be used to warm up most of the data for the other. The more files seen the more efficient the solution becomes even for previously unseen files.

The use of DDUP and compression, combined with the previously discussed SMB and TCP termination are the foundations of why WAN optimization has become so pervasive in so many networks in such a short time.

The problem is that the world is changing quickly making the techniques of data reduction less effective at the WAN Optimizations appliance.


WAN Optimization has never been an ideal solution for time sensitive applications.  Outside of reducing bulk data congestion on the link, traditional WAN Optimization does not do much to reduce the actual time it takes to transmit data from a one computer to another. Time sensitive applications like Voice over IP, remote desktop, desktop video conference, web meetings, and some types of web forms are highly sensitive to end to end network latency and WAN Op does not help since the real bottleneck is the network latency itself from the client to the server. This has always been the case for WAN Optimization.  Nothing new here.

What is new is that the percentage of network traffic of this type is growing substantially as compared to WAN Optimizable bulk traffic.   As unified communication platforms, such as Microsoft Lync, which provide high peer to peer desktop and tablet collaboration for voice, video, and desktop sharing become more pervasive the value of WAN Op is diminishing.

Software as a Service cloud applications are adding momentum to the trend. If users were using Outlook 2003 with an in-house Exchange server, then WAN Optimization provided value by improving the responsiveness of the application by greatly reducing the send and receive times.  On the other hand, if users are using Microsoft 365 and are using the secure web portal for access to email, WAN Op provides much less of an improvement.  Most of the user perceivable issues with responsiveness is driven not by the amount of bandwidth in use but with the latency of the entire network between the client and the server.

As desktops, laptops and tablets evolve to be less thick work stations and more thin communication and collaboration stations with most of the applications, data, and people located securely in private or public clouds, WAN Optimization will continue to be further diminished in its utility.

Pervasive Data reduction:

WAN Optimizations ability to reduce data, enhanced by its position in the network, has been the key to its success but also may be a source of its eventual demise.  Many of the same successful techniques used by WAN Optimization appliance do not need to reside at that location in the network and can be performed very adequately at the source and destination within the client and the server.  Data reduction techniques have been used for video (MPEG) and other equivalents for years.  As a result, WAN optimization has never been very useful for that type of communications since it adds little if any further reduction. The same holds true for voice over IP and other time sensitive compressed data streams.

What has changed recently is the amount of data reduction being done for non-time critical bulk applications that used to the bread and butter of WAN Optimization.  DDUP has been added directly to the file servers themselves.  Microsoft has added it to windows server 2012.  Network attached storage venders and distributed file systems have it included it as a standard feature.  Even DropBox and other cloud based storage applications have added compression and DDUP.  A true advantage of doing data reduction at the source and destination is the ability to reduce the traffic prior to encryption at the source and destination. This is a much simpler and secure option then using an add on solution such as WAN Optimization.

In summary, as  applications become more “WAN friendly” via protocol changes and pervasive data reduction they are becoming more impervious to WAN Optimization’s utility in your network.


Categories: Software Defined WAN (SD-WAN)