Differences

This shows you the differences between two versions of the page.

--- deptlab:spark [2021/12/03 11:57] – [Firewall info] localadmin
+++ deptlab:spark [2024/09/05 11:04] (current) – [Firewall info] localadmin
@@ Line 1: / Line 1: @@
 ====== Using Apache Spark in the CS Dept Lab ======
-Spark 3.1.1 is installed in the CS Dept. Lab in the folder: /usr/lib/spark. You'll find the Spark binaries and scripts at /usr/lib/spark/bin.
+Spark 3.5.0 is installed in the CS Dept. Lab in the folder: /usr/lib/spark. You'll find the Spark binaries and scripts at /usr/lib/spark/bin.
 Spark is currently configured to use six worker nodes.
 If you are a member of the 'spark_users' group, you will be able to access Spark to submit new jobs. Jobs MUST be submitted to the head node (spark.cs.ndsu.edu).
+Spark documentation for 3.5.0 can be found here https://spark.apache.org/docs/3.5.0/
 ===== Status pages =====
@@ Line 14: / Line 17: @@
 ===== Firewall info =====
-The head node is firewalled such that Spark status links are ONLY accessible from ND state networks. The head node, Zoidberg, can be accessed via SSH from anywhere (**not being blocked from your home country or ISP**), so job submission is possible from outside of these networks, but it is currently not possible to view the status pages.
+The head node is firewalled such that Spark status links are ONLY accessible from /* ND state */ CS Dept networks. The Spark head node can be accessed via SSH from anywhere (**not being blocked from your home country or ISP**), so job submission is possible from outside of these networks, but it is currently not possible to view the status pages for security purposes.
 ===== Spark Components =====
 The current version of Spark installed does NOT support HDSF