Data Mining and Knowledge Engineering

Open Access Open Access  Restricted Access Subscription Access

Weather Forecasting Using Map Reduce Technique


   Subscribe/Renew Journal


The term Big Data basically refers to the huge volume of data. There are various sources that produce huge amount of data which includes facebook, twitter, weather station, sensors, airlines, hospitality data, newspaper etc.  Data that is collected from different sources will have different formats such as structured, semi-structured and unstructured, hence it has become difficult to process and manage by the traditional data management methods. Since there is enormous data generation, the important thing is how to store such a huge amount of data and how to manage the data. Big Data is characterized by 3V's size (volume), complexity (variety), and rate of growth (velocity) which make them difficult to process or analyze. There are some innovative technologies to capture, store and analyze petabytes of data. Hadoop manages such huge amount of data in an efficient manner. In this paper the input data is collected from NCDC (National Climatic Data Centre) then input data is stored in HDFS (Hadoop Distributed File System), later processed using Map Reduce technique.

 


Keywords

Hadoop, HDFS, Map Reduce, NCDC.
User
Subscription Login to verify subscription
Notifications
Font Size

  • M. Dhavapriya, N. Yasodha "Big Data Analytics: Challenges and Solutions Using Hadoop, Map Reduce" International Journal of Computer Science Trends and Technology (IJCST) – Volume 4 Issue 1, Jan - Feb 2016

  • Varsha B. Bobade "Survey Paper on Big Data and Hadoop International Research Journal of Engineering and Technology" (IRJET) Volume: 03 issue: 01 Jan-2016

  • M Ramya, ChetanBalaji, and L Girish. "Environment change prediction to adapt climate- smart agriculture using big data analytics". International Journal of Advanced Research in Computer Engineering & Technology (IJARCET) Volume, 4, May 2015.

  • Harshawardhan S. Bhosale A Review paper on Big Data and Hadoop International Journal of Scientific and Research Publications, Volume 4, Issue 10, October 2014 1 ISSN 2250-3153

  • Dagade, Mahesh Lagali. Big data weather analytics using Hadoop. International Journal of Emerging Technology in Computer Science & Electronics (IJETCSE), 14(2), APRIL 2015.

  • M Ramya, ChetanBalaji, and L Girish. Environment change prediction to adopt climate-smart agriculture using big data analytics. International Journal of Advanced Research in Computer Engineering & Technology (IJARCET) Volume, 4, May 2015.

  • Surekha Mariam Varghese Riyaz P.A. Leveraging map reduces with Hadoop for weather data analytics. IOSR Journal of Computer Engineering, 17(3), May-Jun 2015.

  • Gantz, John, and David Reinsel. "The digital universe in 2020: Big data, bigger digital shadows, and biggest growth in the far east", IDC iView: IDC Analyze the Future 2007, pp: 1-16, 2012.

  • Kaur, Anureet. "Big Data: A Review of Challenges, Tools, and Techniques. IJSRSET, 2 (2), 2016.

  • S. Vijayarani, S. Maria Sylviaa, A.Sakila, "Clustering Algorithms for Outlier Detection performance. Analysis” International Conference on Computing and Intelligence Systems, 04, 1213-1217, March 2015.

  • S.Vijayarani,S. Maria Sylviaa ,A.Sakila, “Clustering Algorithms for Outlier DetectionPerformance Analysis” International Conference on Computing and Intelligence Systems, 04, 1213-1217, March 2015.

  • Surekha Mariam Varghese Riyaz P.A. "Leveraging map reduce with hadoop for weather data analytics". IOSR Journal of Computer Engineering, 17(3), May- Jun 2015.

  • S.Vijayarani,S. Maria Sylviaa ,A.Sakila, “Clustering Algorithms for Outlier Detection Performance Analysis” International Conference on Computing and Intelligence Systems, 04, 1213-1217, March 2015


Abstract Views: 287




  • Weather Forecasting Using Map Reduce Technique

Abstract Views: 287  | 

Authors

Abstract


The term Big Data basically refers to the huge volume of data. There are various sources that produce huge amount of data which includes facebook, twitter, weather station, sensors, airlines, hospitality data, newspaper etc.  Data that is collected from different sources will have different formats such as structured, semi-structured and unstructured, hence it has become difficult to process and manage by the traditional data management methods. Since there is enormous data generation, the important thing is how to store such a huge amount of data and how to manage the data. Big Data is characterized by 3V's size (volume), complexity (variety), and rate of growth (velocity) which make them difficult to process or analyze. There are some innovative technologies to capture, store and analyze petabytes of data. Hadoop manages such huge amount of data in an efficient manner. In this paper the input data is collected from NCDC (National Climatic Data Centre) then input data is stored in HDFS (Hadoop Distributed File System), later processed using Map Reduce technique.

 


Keywords


Hadoop, HDFS, Map Reduce, NCDC.

References