Session 3 (b) : Lab Demo - Interacting with HDFS

Session 3 (b)
In this session i will show you how you can interact with HDFS from Linux command line on a 'Single Node Apache Hadoop System'. Also we will look into the Basics of File System, Data Blocks and Disk Sectors that we discussed in Part a

Lab Demo:
-------------------------------------------------------------------------------------------------------

Login as root

Disk sector size
fdisk -l | grep 'sector size'

Filesystem type:
mount | grep /dev/sda3

Filesystem blocksize:
xfs_info /dev/sda3 | grep bsize

Switch to hdpuser
su hdpuser

Start hdfs:

Check if namenode WebUi is up
netstat -an | grep 50070

Show the Namenode Web UI to check the available and used space in hdfs

hadoop fs -ls /
hadoop fs -mkdir /hdpuser
hadoop fs -mkdir /others

Copy file from OS filesystem into hdfs

Check the hdfs blocks under
/hdfs/disk1/data (This directory was configured during installation and can be changed)

-------------------------------------------------------------------------------------------------------
You can email me on:

[amdara]

9 MB file saved in the HDD as 10 separate files. If so what is the benefit of the Hadoop ?