Differences

This shows you the differences between two versions of the page.

--- deptlab:hadoop [2020/11/28 22:03] – [Status pages] localadmin
+++ deptlab:hadoop [2025/03/03 14:45] (current) – [Status pages] localadmin
@@ Line 1: / Line 1: @@
 ====== Using Hadoop in the CS Dept Lab ======
+====== Hadoop is OFFLINE until security Vulnerability can be fixed. ======
-==== Currently Offline ====
+Hadoop 2.10.2 is installed in the CS Dept. Lab in the folder: /usr/lib/hadoop. You'll find the Hadoop binaries and scripts at /usr/lib/hadoop/bin.
-  --- //[[support@cs.ndsu.edu|local admin]] 2020/11/15 03:17//
-Hadoop 2.7.1 is installed in the CS Dept. Lab in the folder: /usr/lib/hadoop. You'll find the Hadoop binaries and scripts at /usr/lib/hadoop/bin.
 If you are a member of the 'hadoop' group, you will be able to access hadoop to submit new jobs.
-Jobs must be submitted to the head node (zoidberg.cs.ndsu.nodak.edu). Jobs MUST be submitted at the head node.
+Jobs **MUST** be submitted to the head node (zoidberg.cs.ndsu.nodak.edu).
-Hadoop makes use of the HDFS (Hadoop Distributed File System). Hadoop node computers are configured to be a node in an HDFS cluster. There's a bit of space here so don't feel shy about using it, but please delete your large data files when they are no longer needed.
+Hadoop makes use of the HDFS (Hadoop Distributed File System). Hadoop node computers are configured to be a node in an HDFS cluster. There's a bit of space here (about 14TB) so don't feel shy about using it, but please delete your large data files when they are no longer needed.
+Credentials for the Hadoop cluster are your NDSU credentials.
 To become a member of the 'hadoop' group, email <labsupport@cs.ndsu.edu>. You will be added to the group and a folder in the HDFS filesystem will be made for you.
@@ Line 16: / Line 16: @@
 ===== Status pages =====
-[[http://zoidberg.cs.ndsu.nodak.edu:50070 | HDFS status ]]
+These pages are accessible only from ND state university system networks.
-<del>[[http://zoidberg.cs.ndsu.nodak.edu:50030 | Node/Job status]]</del> broken? FIXME:
+[[http://zoidberg.cs.ndsu.nodak.edu:50070 | HDFS status ]]
 [[http://zoidberg.cs.ndsu.nodak.edu:19888 | Job History]]
-[[http://zoidberg.cs.ndsu.nodak.edu:8088 | Resource Manager]]
+[[http://zoidberg.cs.ndsu.nodak.edu:8088 | Resource Manager]] **Only available from the computers in the 244 lab.**
-**These pages are accessible only from ND state networks**. /* , if they need to be accessible outside of these networks, contact <labsupport@cs.ndsu.edu> */
+**The Hadoop tracker pages are accessible only from ND state university system networks**. /* , if they need to be accessible outside of these networks, contact <labsupport@cs.ndsu.edu> */
 ===== Firewall info =====
-The head node is firewalled such that Hadoop is ONLY accessible from ND state networks. It can be accessed via SSH from anywhere, so job submission is possible from outside of these networks, but it is currently not possible to view the status pages.
+The head node is firewalled such that Hadoop status links are ONLY accessible from ND state networks. The head node, Zoidberg, can be accessed via SSH from anywhere, so job submission is possible from outside of these networks, but it is currently not possible to view the status pages.
 ====== Getting Started ======
@@ Line 63: / Line 63: @@
 ==== Monitoring ====
-You can monitor hadoop by visiting particular webpages on the head node. You can visit the [[http://zoidberg.cs.ndsu.nodak.edu:50030|Job Tracker]] and [[http://zoidberg.cs.ndsu.nodak.edu:50070|DFS Summary]] pages for information on jobs and HDFS. FIXME:
+You can monitor hadoop by visiting particular webpages on the head node. You can visit the [[http://zoidberg.cs.ndsu.nodak.edu:50030|Job Tracker]] and [[http://zoidberg.cs.ndsu.nodak.edu:50070|DFS Summary]] pages for information on jobs and HDFS.