Introducing a Novel Data Management Approach for Distributed Large Scale Data Processing in Future Computer Clouds

Abstract

Deployment of pattern recognition applications for large-scale data sets is an open issue that needs to be addressed. In this paper, an attempt is made to explore new methods of partitioning and distributing data, that is, resource virtualization in the cloud by fundamentally re-thinking the way in which future data management models will need to be developed on the Internet. The work presented here will incorporate content-addressable memory into Cloud data processing to entail a large number of loosely-coupled parallel operations resulting in vastly improved performance. Using a lightweight associative memory algorithm known as Distributed Hierarchical Graph Neuron (DHGN), data retrieval/processing can be modeled as pattern recognition/matching problem, conducted across multiple records and data segments within a single-cycle, utilizing a parallel approach. The proposed model envisions a distributed data management scheme for large-scale data processing and database updating that is capable of providing scalable real-time recognition and processing with high accuracy while being able to maintain low computational cost in its function.