正文
利用hadoop自带程序运行wordcount
提示：扫一扫查出行【扫一扫了解最新限行尾号】
复制提示
1.启动hadoop守护进程
bin/start-all.sh
2.在hadoop的bin目录下建立一个input文件夹
JIAS-MacBook-Pro:hadoop-0.20. jia$ mkdir input
3.进入input目录之后，在input目录下新建两个文本文件，并想其写入内容
JIAS-MacBook-Pro:hadoop-0.20. jia$ cd input
JIAS-MacBook-Pro:input jia$ echo "hello excuse me fine thank you">text1.txt
JIAS-MacBook-Pro:input jia$ echo "hello how do you do thank you">text2.txt
4.进入hadoop的bin目录，输入jps命令，确认hadoop已经跑起来了
JIAS-MacBook-Pro:hadoop-0.20. jia$ cd bin
JIAS-MacBook-Pro:bin jia$ jps
SecondaryNameNode
NameNode
JobTracker
TaskTracker
DataNode
Jps
5.把input文件上传到hdfs上
JIAS-MacBook-Pro:hadoop-0.20.2 jia$ bin/hadoop dfs -put input in
6.查看hdfs上的项目
JIAS-MacBook-Pro:hadoop-0.20. jia$ bin/hadoop dfs -ls ./in/*
-rw-r--r-- 1 jia supergroup 31 2014-07-17 20:39 /user/jia/in/text1.txt
-rw-r--r-- 1 jia supergroup 30 2014-07-17 20:39 /user/jia/in/text2.txt
7.利用自带的wordcount执行，并把结果放在output文件夹上
JIAS-MacBook-Pro:hadoop-0.20. jia$ bin/hadoop jar hadoop-0.20.-examples.jar wordcount in output
// :: INFO input.FileInputFormat: Total input paths to process :
// :: INFO mapred.JobClient: Running job: job_201407172036_0001
// :: INFO mapred.JobClient: map % reduce %
// :: INFO mapred.JobClient: map % reduce %
// :: INFO mapred.JobClient: map % reduce %
// :: INFO mapred.JobClient: Job complete: job_201407172036_0001
// :: INFO mapred.JobClient: Counters:
// :: INFO mapred.JobClient: Map-Reduce Framework
// :: INFO mapred.JobClient: Combine output records=
// :: INFO mapred.JobClient: Spilled Records=
// :: INFO mapred.JobClient: Reduce input records=
// :: INFO mapred.JobClient: Reduce output records=
// :: INFO mapred.JobClient: Map input records=
// :: INFO mapred.JobClient: Map output records=
// :: INFO mapred.JobClient: Map output bytes=
// :: INFO mapred.JobClient: Reduce shuffle bytes=
// :: INFO mapred.JobClient: Combine input records=
// :: INFO mapred.JobClient: Reduce input groups=
// :: INFO mapred.JobClient: FileSystemCounters
// :: INFO mapred.JobClient: HDFS_BYTES_READ=
// :: INFO mapred.JobClient: FILE_BYTES_WRITTEN=
// :: INFO mapred.JobClient: FILE_BYTES_READ=
// :: INFO mapred.JobClient: HDFS_BYTES_WRITTEN=
// :: INFO mapred.JobClient: Job Counters
// :: INFO mapred.JobClient: Launched map tasks=
// :: INFO mapred.JobClient: Launched reduce tasks=
// :: INFO mapred.JobClient: Data-local map tasks=
JIAS-MacBook-Pro:hadoop-0.20. jia$
8.查看结果
JIAS-MacBook-Pro:hadoop-0.20. jia$ bin/hadoop dfs -ls
Found items
drwxr-xr-x - jia supergroup -- : /user/jia/in
drwxr-xr-x - jia supergroup -- : /user/jia/output
JIAS-MacBook-Pro:hadoop-0.20. jia$ bin/hadoop dfs -ls ./output
Found items
drwxr-xr-x - jia supergroup -- : /user/jia/output/_logs
-rw-r--r-- jia supergroup -- : /user/jia/output/part-r-
JIAS-MacBook-Pro:hadoop-0.20. jia$ bin/hadoop dfs -cat ./output/*
do 2
excuse 1
fine 1
hello 2
how 1
me 1
thank 2
you 3
cat: Source must be a file.
