Gradual Class Evolution Detection Using Class Based Ensembles

Abstract

The recent advances in hardware and software have enabled the capture of different measurements of data in a wide range of fields. These measurements are generated continuously and in a very high fluctuating data rates. Mining data streams is concerned with extracting knowledge structures represented in models and patterns in non- stopping streams of information. The research in data stream mining has gained a high attraction due to the importance of its applications and the increasing generation of streaming information. Mining data streams is concerned with extracting knowledge structures represented in models and patterns in non-stopping streams of information. Data Stream classification poses major challenges than classifying static data because of several unique properties of data streams such as infinite length, concept drift, concept evolution and feature evolution. While extensive work has been done in the area of concept drift, concept evolution, a phenomena that induces concept drift has gained little recognition. Class evolution basically focuses on 3 aspects: the phenomenon of class emergence, disappearance and reoccurrence and is an important research topic for data stream mining. Most of the previous works implicitly regard class evolution as a transient change, which is not true for many real-world problems as in many real world applications class evolution is a gradual process. A class-based ensemble approach, namely Class-Based ensemble for Class Evolution (CBCE), is adopted to handle class evolution. By maintaining a base learner for each class and dynamically updating the base learners with new data, CBCE can rapidly adjust to class evolution. A novel under-sampling method for the base learners is used to handle the dynamic class- imbalance problem caused by the gradual evolution of classes. Based on the above concepts of gradual class evolution, a dataset containing records of tweets made in twitter is evaluated at different time stamps by converting the unstructured, dynamic dataset into a more compact form, to evaluate and analyse concept evolution.