Complex systems data mining and knowledge extraction
Luper, David Brandon
MetadataShow full item record
Complex systems appear across numerous disciplines, and analyzing them can be difficult. Standard analysis techniques fail to capture concepts such as emergent behavior, self-organization, or the entanglement among related components within a system. A better knowledge of complex systems could help avoid financial system collapse, predict terrorist network actions, and fight disease. One way to understand a complex system better is to leverage the information encapsulated within the higher order relationships of the system. A complex system is a set of interconnected compartments, and it is these connections that give rise to the characteristics and complexity of the system. These relationships define the structure of a network and the flow across them defines the function. The structure and function of a system encodes valuable information about the system, information that can be hard to find due to the massive amount of information contained within a complex system. In order to isolate important information, data analysis techniques must be implemented. The field of data mining is perfectly suited for this task. Data mining is a term used to describe a compilation of techniques including statistics, artificial intelligence, computational intelligence and database management used to discover and extract information in an automated fashion from large data sets. Though not universal, many forms of data mining are restricted to numerical input. This can be problematic when analyzing a system modeled as a graph, which is of a symbolic nature. Another problem with complex system analysis is a disconnect between higher level system function and lower level compositional elements within the system. The work herein proposes a methodology to solve these problems by presenting an encoding framework to map a complex system of connected symbols into a meaningful numeric feature space. This methodology will allow numerous techniques from the field of data mining to be applied in transformative ways, creating new possibilities in the field of systems research.