IMPLEMENTATION OF P-PIC ALGORITHM IN MAP REDUCE TO HANDLE BIG DATA

Jayalatchumy D, Thambidurai. P

Abstract: Clustering is a process of grouping objects that are similar among themselves but dissimilar to objects in others. Clustering large dataset is a challenging resource data intensive task. The key to scalability and performance benefits it to use parallel algorithms. Moreover the use of Big Data has become very crucial in almost all sectors nowadays. However analyzing Big data is a very challenging task. Google’s Mapreduce has attracted a lot of attention for such applications that motivate us to convert sequential algorithm to Mapreduce algorithm. This paper presents the p-PIC with Mapreduce, one of the newly developed clustering algorithms. P-PIC originated from PIC though scalable and effective finds it difficult to fit, and works well only for low end commodity computers. It performs clustering by embedding data points in a low dimensional data derived from the similarity matrix. The experimental results show that p-PIC can perform well in MR framework for handling big data. It is very fast and scalable. The results show that the accuracy in producing the clusters is almost the same in using Mapreduce framework. Hence the results produced by p-PIC in mapreduce are fast, scalable and accurate.

Keywords: p-PIC, MapReduce, Big data, clustering, HDFS

DOI: https://doi.org/10.15623/ijret.2014.0319022