NDSU CS Department Wiki

User Tools

Site Tools

Trace:
deptlab:hadoop

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
deptlab:hadoop [2023/03/15 14:52] – [Status pages] localadmindeptlab:hadoop [2025/03/03 14:45] (current) – [Status pages] localadmin
Line 1: Line 1:
 ====== Using Hadoop in the CS Dept Lab ====== ====== Using Hadoop in the CS Dept Lab ======
 +====== Hadoop is OFFLINE until security Vulnerability can be fixed. ======
  
-Hadoop 2.8.5 is installed in the CS Dept. Lab in the folder: /usr/lib/hadoop. You'll find the Hadoop binaries and scripts at /usr/lib/hadoop/bin. +Hadoop 2.10.2 is installed in the CS Dept. Lab in the folder: /usr/lib/hadoop. You'll find the Hadoop binaries and scripts at /usr/lib/hadoop/bin. 
  
 If you are a member of the 'hadoop' group, you will be able to access hadoop to submit new jobs.  If you are a member of the 'hadoop' group, you will be able to access hadoop to submit new jobs. 
Line 8: Line 9:
  
 Hadoop makes use of the HDFS (Hadoop Distributed File System). Hadoop node computers are configured to be a node in an HDFS cluster. There's a bit of space here (about 14TB) so don't feel shy about using it, but please delete your large data files when they are no longer needed. Hadoop makes use of the HDFS (Hadoop Distributed File System). Hadoop node computers are configured to be a node in an HDFS cluster. There's a bit of space here (about 14TB) so don't feel shy about using it, but please delete your large data files when they are no longer needed.
 +
 +Credentials for the Hadoop cluster are your NDSU credentials.
  
 To become a member of the 'hadoop' group, email <labsupport@cs.ndsu.edu>. You will be added to the group and a folder in the HDFS filesystem will be made for you.  To become a member of the 'hadoop' group, email <labsupport@cs.ndsu.edu>. You will be added to the group and a folder in the HDFS filesystem will be made for you. 
  
 ===== Status pages ===== ===== Status pages =====
 +
 +These pages are accessible only from ND state university system networks.
  
 [[http://zoidberg.cs.ndsu.nodak.edu:50070 | HDFS status ]] [[http://zoidberg.cs.ndsu.nodak.edu:50070 | HDFS status ]]
Line 17: Line 22:
 [[http://zoidberg.cs.ndsu.nodak.edu:19888 | Job History]] [[http://zoidberg.cs.ndsu.nodak.edu:19888 | Job History]]
  
-[[http://zoidberg.cs.ndsu.nodak.edu:8088 | Resource Manager]]+[[http://zoidberg.cs.ndsu.nodak.edu:8088 | Resource Manager]] **Only available from the computers in the 244 lab.**
  
-**These pages are accessible only from ND state university system networks**. /* , if they need to be accessible outside of these networks, contact <labsupport@cs.ndsu.edu> */+**The Hadoop tracker pages are accessible only from ND state university system networks**. /* , if they need to be accessible outside of these networks, contact <labsupport@cs.ndsu.edu> */
  
 ===== Firewall info ===== ===== Firewall info =====
deptlab/hadoop.1678909955.txt.gz · Last modified: by localadmin