This post shows how to install Orcale JDK using apt-get or by downloading from the Oracle site on Ubuntu. It also shows how to install OpenJDK. Installing Oracle JDK using apt-get For installing Java in Ubuntu first you need to add Oracle’s PPA (Personal Package Archive) so that Ubuntu knows…
This post shows what is Safemode in Namenode and what are the configurations for the safemode. You will also see the commands available to enter and leave the safemode explicitly. When the Namenode is started it loads the file system state into memory initially from the fsimage and then apply…
This post shows what is HDFS federation in Hadoop framework and what configuration changes are required for setting up HDFS federation. Problem with HDFS architecture In a Hadoop cluster namespace management and block management both are done by Namenode. So, essentially the Namenode has to perform the following tasks- 1-…
In this post there is a compilation of some of the frequently used HDFS commands with examples which can be used as reference. All HDFS commands are invoked by the bin/hdfs script. Running the hdfs script without any arguments prints the description for all commands. 1- HDFS command to create a…
In this post working of the HDFS components Namenode, Datanode and Secondary Namenode are explained in detail. Namenode in Hadoop HDFS works on a master/slave architecture. In HDFS cluster Namenode is the master and the centerpiece of the HDFS file system. Namenode manages the file system namespace. It keeps the…
As per the replica placement policy in Hadoop each HDFS block is replicated across different nodes. Default replication factor is 3 which means by default each HDFS block is replicated on three different nodes in order to make HDFS reliable and fault tolerant. Considerations for HDFS replica placement policy When…
HDFS (Hadoop Distributed File System) is a distributed file system, that is part of Hadoop framework. HDFS in Hadoop framework is designed to store and manage very large files. In HDFS large file is divided into blocks and then those blocks are distributed across the nodes of the cluster. When…
Big Data means a very large volume of data. Term big data is used to describe data so huge and ever growing that has gone beyond the storage and processing capabilities of traditional data management and processing tools. Some Examples of Big Data Facebook which stores data about your posts,…
In this post we’ll see how to install Hadoop in Pseudo-distributed mode (single node cluster). With the steps given here for Hadoop installation you will have Hadoop common, HDFS, MapReduce and YARN installed. Hadoop release used for installation is Hadoop 2.9.0 and it is installed on Ubuntu 16.04. Modes in…
Apache Hadoop is an open source framework for storing data and processing of data set of big data on a cluster of nodes (commodity hardware) in parallel. Hadoop framework is designed to scale up from single server to thousand of machines with each machine offering both storage and computation. It…