Differences

This shows you the differences between two versions of the page.

--- deptlab:usingthelab [2025/01/20 18:13] – [File Transfer (SCP)] localadmin
+++ deptlab:usingthelab [2025/02/20 14:41] (current) – [Status] localadmin
@@ Line 7: / Line 7: @@
 ===== Status =====
-  * Hadoop is up.
+  * Hadoop is offline for security reasons.
   * Spark is up.
+  * Lab13 is down.
+  * Lan21 is down.
 ===== Accessing the Lab =====
@@ Line 46: / Line 48: @@
   * If using Python be sure to use Python Virtual Environments so you can install needed packages.  [[https://realpython.com/python-virtual-environments-a-primer/|Here's a tutorial]]. The system admin will NOT install packages for you.
+==== Hadoop ====
-===== GPU Computing =====
+The Apache Hadoop software library is a framework that allows for the distributed processing of large data sets across clusters of computers using simple programming models. The Hadoop head node is zoidberg.cs.ndsu.nodak.edu
+==== Spark ====
+Apache Spark™ is a multi-language engine for executing data engineering, data science, and machine learning on single-node machines or clusters. The spark head node is spark.cs.ndsu.edu
+==== GPU Computing ====
 Labs 02-05 contain NVIDIA 24GB Tesla M40 GPU's (These are past current Tensorflow version support) and Labs 06-09 contain NVIDIA 16GB Tesla P100 GPU's.
@@ Line 54: / Line 59: @@
 Use responsibly.
-===== MPI =====
+==== MPI ====
 MPICH is installed in the lab. MPICH is a high-performance and widely portable implementation of the  MPI-3.1 standard from the Argonne National Laboratory. It supports different computation and communication platforms including commodity clusters, SMPs, massively parallel systems, and high-speed networks.
@@ Line 60: / Line 65: @@
 ===== Storage =====
-Your home directory (/home/<username>) is stored on a Network File Server, and is mounted on all 21 machines, so your files are available everywhere. Quotas are not enforced, just try to be responsible with space usage.
+Your home directory (/home/<username>) is stored on a Network File Server (NFS), and is mounted on all 40 machines, so your files are available everywhere. Quotas are not enforced, just try to be responsible with space usage.
-If your program or research experiment requires high levels of file IO, we ask that you do not use the NFS server for this (you will destroy performance across the lab, plus it will be slow for your experiment).  If you require faster storage, contact [[support@cs.ndsu.edu]] and we can grant you access to local storage on each box.
+If your program or research experiment requires high levels of file IO, we ask that you **do not use** the NFS server for this (you will destroy performance across the lab, plus it will be slow for your experiment).  If you require faster storage, contact [[support@cs.ndsu.edu]] and we can grant you access to local storage on each box.
 ===== Fair Use of the 244 Lab =====
@@ Line 69: / Line 74: @@
   - Don't do anything illegal.  Just don't.
   - <color #ed1c24>**Log off when you're done**</color>. Don't just lock the screen unless you're returning shortly. Too many people doing that can waste a lot of resources and in-lab classes have priority access.
-  - Don't power off the workstations. Other users may be logged into the machines or want remote access.
+  - <color #ed1c24>**Don't power off the workstations.**</color> Other users may be logged into the machines or want remote access.
   - Don't unplug monitors, workstations, or other equipment. Students need the lab to function properly when you leave, so please leave the environment like you found it. Especially remember that students may be working on assignments remotely, so unplugging a machine will result in them losing their work.
   - Don't use up all the resources. Again, the lab is shared, so if you're running an experiment that uses up all the CPU/memory on the machines, you are making it difficult for students to complete their work. If this becomes a problem, the Administrator will lock your account. If you are a student and have large CPU/memory/disk requirements for your research, coordinate with your adviser and department Systems Administrator.
@@ Line 77: / Line 82: @@
 In the lab, you can put files in your public_html directory, they will be served out at http://students.cs.ndsu.nodak.edu/~yourusername/.
-You will use this directory for certain courses.  You're also welcome to use it for small scale private hosting.
+You will use this directory for certain courses.  You're also welcome to use it for small scale private hosting. Don't post Personal Identifiable Information (PII)!
@@ Line 88: / Line 93: @@
 To fix this you will need to log in via SSH and delete old x2go session information from your home directory.
-Maximum session idle time is **three** days.
+Maximum session idle time is **three** days, before sessions will be purged.
-**Note:** Some graphical applications do not like to work via x2go. GoogleEarth for example.
+**Note:** Some graphical applications do not like to work via x2go. GoogleEarth for example. Also GPU compute applications often run faster via a SSH text connection.
 **Note:** X2go usually works. If it doesn't...
@@ Line 99: / Line 104: @@
   * You can run x2go in debug mode to see a log file that may provide some clues as to what is wrong on your end. If you don't see debug mode, reinstall x2go.
+X2go install on a Mac is a bit different. It requires Xquartz to be installed. https://wiki.cs.ndsu.nodak.edu/doku.php?id=helpdocs:x2go:windows&s[]=x2go#x2go_on_mac It usually works but sometimes it doesn't and neither Google or I have been able to determine why.
+x2go is also available for Linux and can sometimes be made to work on a Chromebook.
 ===== Help! =====
 If you need help using the 244 lab. **Contact your instructor or academic advisor**.