It is hard not to see the relationship forming between Big Data and the software defined network. As volumes increase due to smart technologies and machine-to-machine communications, it will take a great deal of flexibility to channel information to the right tools at the right times in order to make sense of complex data sets.
This, in fact, is one of the main selling points of SDN. Companies ranging from Cisco and HP to Microsoft, Oracle and SAP are scrambling to, in the words of Network World's Pete Bartolik, "figure out how to put the riches of Big Data into the hands of workers who are most positioned to fashion it into customer-facing uses." Achieving this will require networks of not only greater size and scale, but advanced management and orchestration capabilities that can assess where, when and how data is to be processed and analyzed.
Using Big Data to Enable SDN to…Enable Big Data
But exactly how will network management accomplish this feat? Curiously enough, the answer is Big Data. By employing the same techniques that organizations hope can divine patterns in customer behavior and complex transactional environments, the enterprise will be able to oversee the increasingly dynamic nature of its network environment, according to Delphi resident Gary Audin. This will become increasingly necessary as the response times of change management and network configuration processes drop from milliseconds to microseconds. This is faster than the human brain can analyze and respond to. And, depending on the size of the network and the interrelationships between processing points, it can easily generate volumes that qualify as Big Data. Leading standards groups like the European Telecommunications Standards Institute (ETSI) are already devising ways to unify SDN and NFV monitoring data.
This level of functionality will have to extend beyond the LAN into the wide area network if expectations for broadly federated cloud environments are to be realized, said Mushroom Networks CEO Dr. Cahit Akin. Current WAN orchestration systems are already tasked with complex processes like dynamic flow mapping, elastic IP address management and a wide range of QoS functions. As the enterprise comes to rely on the WAN for end-to-end data infrastructure support, these tools will need to employ Big Data as a means to maximize resource utilization and coordination, lest the cost of the dynamic, distributed architecture spins out of control. Big Data toolsets will help to not only collect network information and trigger alerts, but present ongoing operational conditions to IT in a way that is both understandable and actionable.
This is part of what Gartner is calling "advanced, pervasive and invisible analytics." The fact is, much of this analysis will take place outside the scope of human oversight as machines and automated systems take on increasing responsibility for the advanced data environments under development today. As the company's David Cearley put it in a recent outlook: "Every app now needs to be an analytic app." So not only will the sheer volume of networking data become so great as to overwhelm current management stacks, but the nature and context of that data will have to parsed and filtered at lightning speed in order to keep pace with rapidly changing network conditions.
Big Data is intended to bring clarity to complex systems without burdening the analyst, whether of computer or human variety, with stacks of raw data and disjointed information. Since modern networking already falls under the definition of a "complex system" and will become even more inscrutable as the mobile/cloud era unfolds, it is only natural that the enterprise turns to state-of-the-art technologies in order to drive both efficiency and cost-effectiveness.
Big Data is, in fact, the key to Big Networking.
Photo courtesy of Shutterstock.