Differences

This shows you the differences between two versions of the page.

--- deptlab:hadoop:gettingstarted [2017/01/15 20:31] – [Setting Up Input Files] localadmin
+++ deptlab:hadoop:gettingstarted [2025/04/23 16:40] (current) – [Getting Started with Hadoop] localadmin
@@ Line 2: / Line 2: @@
 This is a short tutorial on using Hadoop.
+Hadoop commands and formats change at times. If the **hadoop fs <command>** doesn't work. Try **hdfs dfs <command>**.
 We'll go through the process of compiling, packaging, and running a simple Hadoop program.
@@ Line 9: / Line 11: @@
 ===== Logging In =====
-First, make sure you can log in to the head node with SSH, currently at zoidberg.cs.ndsu.nodak.edu. You can log in to this server with your CS Domain password or your Blackboard password.
+First, make sure you can log in to the head node with SSH, currently at zoidberg.cs.ndsu.nodak.edu. You can log in to this server with your CS Domain password (The one you use for the Windows clusters found around campus or Campus Wi-Fi), and NOT your University System password.
 If you have trouble logging in:
@@ Line 66: / Line 68: @@
 In this case, we use the wc.jar JAR file, running the class 'WordCount' with two parameters, an input directory and an output directory. The output directory must not already exist in HDFS, it will be created by the program.
+===== View Output =====
+You can check the output directory with:
+<code>
+hadoop fs -ls /user/username/wordcount/output/
+</code>
+You should then see something similar to:
+<code>
+helsene@zoidberg:~$ hadoop fs -ls /user/helsene/wordcount/output
+Found 2 items
+-rw-r--r--   3 helsene nogroup         41 2011-11-08 11:23 /user/helsene/wordcount/output/part-r-00000
+</code>
+The 'part-r-00000' file contains the results of the word counting. You can look at the file using the 'cat' command.
+<code>
+helsene@zoidberg:~$ hadoop fs -cat /user/helsene/wordcount/output/part-r-00000
+Bye 1
+Goodbye	1
+Hadoop 2
+Hello 2
+World 2
+</code>
+==== Notes on Hadoop 2.8.5 ====
+May need to run: export HADOOP_CLASSPATH=${JAVA_HOME}/lib/tools.jar
+Compile: hadoop com.sun.tools.javac.Main WordCount.java
+Make Jar: jar cf wc.jar WordCount*.class
+Execute jar: hadoop jar wc.jar WordCount /user/ghokanso/wordcount/input /user/ghokanso/wordcount/output