Summarization Method and Timeline Generation of the Tweet

Abstract

Twitter is the most popular micro blogging web site. More than millions of tweets are posted along twitter every day. Tweets contains huge amount of noisy and redundant data. It is very important to summarize the huge amount of tweets by reducing the size of tweets and removing the noise, for improving the result accuracy. The operations over flood of tweets are not an easy task. There are so many tweets are unrelated, also arrival rate of tweets is fast. To handle these problems, there is a need of efficient and strong summarization algorithm. This algorithm should be flexible with random time duration. For topic evolution system should detect sub-topic and keeps track for any changes occur with the time. To achieve all these goals, proposed system performs three types of operations on tweets, named as clustering of tweets, summarization and topic evaluation over tweeter data. Framework has component is data duplication checking using SHA1 hashing strategy. Framework used clustering procedure it uses EM clustering and compare the EM clustering algorithm with K-means clustering algorithm. After this, tweets are summarized with greedy algorithm, which is more accuracy as compared to traditional summarization algorithm. Finally, the topic is detected for generated summary. Experimental results proves that the proposed system summarize the tweets more accurately and efficiently.