Add Cloud Usage docs

source/cloud-usage.rst

@@ -0,0 +1,203 @@
+Cloud Usage
+===========
+
+This page covers how the lab uses cloud-based tools and how to properly
+access and manage shared resources.
+
+General Access
+--------------
+
+- All computing, whether on-premises or remote, should be done on
+  `BioHPC <https://portal.biohpc.swmed.edu/content/>`_.
+- New users should register for an account at
+  `BioHPC account registration <https://portal.biohpc.swmed.edu/accounts/register/>`_.
+- New users are required to attend mandatory BioHPC training on the first
+  Wednesday of every month. BioHPC accounts cannot be activated without
+  training attendance.
+- BioHPC access is provided mainly through
+  `web based visualization <https://portal.biohpc.swmed.edu/intranet/terminal/webgui/>`_.
+- It is recommended to download a
+  `VNC client <https://github.com/TurboVNC/turbovnc/releases>`_ for WebGUI
+  access.
+- Remote BioHPC access requires VPN access through Global Protect. Information
+  on how to set up remote access can be found
+  `here <https://www.utsouthwestern.edu/about-us/administrative-offices/information-resources/working-remotely.html>`_.
+
+Filesystem Navigation
+---------------------
+
+These are fundamental operations for moving and navigating to data on BioHPC
+nodes.
+
+- Print Working Directory (pwd): Displays the full path of the directory you
+  are currently in, to determine your location within the filesystem.
+
+.. code-block:: bash
+
+   pwd
+
+- List Directory Contents (ls): Shows the files and folders in your current
+  directory. Use ls -l for detailed information or ls -a to see hidden files.
+
+.. code-block:: bash
+
+   ls -la
+
+- Change Directory (cd): Navigates to a different directory. Use cd - to go
+  to the previous directory, cd ~ to go to your home directory, or provide a
+  path to navigate to.
+
+.. code-block:: bash
+
+   cd /path/to/data/
+
+- Move or Rename Data (mv): Moves files or directories from one location to
+  another. This same command is also used to rename files.
+
+.. code-block:: bash
+
+   mv data_file.csv /new/location/
+   mv old_name.txt new_name.txt
+
+- Copy Data (cp): Duplicates files. If you need to copy an entire directory
+  and all of its contents, be sure to use the recursive flag (-r).
+
+.. code-block:: bash
+
+   cp source_file.txt destination_file.txt
+   cp source_file.txt /destination/data_folder/
+   cp -r /source/data_folder/ /destination/data_folder/
+
+
+Data Processing
+---------------
+
+Microscopy data can now be processed through
+`Astrocyte <https://astrocyte.biohpc.swmed.edu/>`_. An existing BioHPC account
+is required for Astrocyte access. Astrocyte accounts also share the same
+credentials as BioHPC accounts.
+
+Project Creation
+^^^^^^^^^^^^^^^^
+
+Workflows can only be run through the usage of Astrocyte projects. Use these
+steps to create a project in Astrocyte and upload workflow input data.
+
+- Go to the My Project page in Astrocyte.
+
+.. image:: images/astrocyte-my-project-page.png
+  :alt: Screenshot: Astrocyte My Project page
+
+- Scroll to the create project section.
+- Enter a project name, then press Create.
+
+.. image:: images/astrocyte-create-project-section.png
+  :alt: Screenshot: Create project section with project name and Create button
+
+- After the project opens, find the Input area and click Upload
+
+.. image:: images/astrocyte-project-input-upload.png
+  :alt: Screenshot: Upload button under the project Input area.
+
+- Upload the input data through the upload channel you want to use.
+
+Data Staging
+^^^^^^^^^^^^
+
+Data can be uploaded to an Astrocyte project through a variety of provided
+channels. If you have not yet created a project, refer to the steps above.
+
+- Data can be directly uploaded through the WebGUI.
+
+.. image:: images/direct-upload.png
+  :alt: Screenshot: Astrocyte direct file upload
+
+- Data can be copied directly from the BioHPC portal via command line.
+
+.. image:: images/cli-upload.png
+  :alt: Screenshot: CLI upload instructions
+
+- Data can be uploaded through Lamella via share link.
+
+.. image:: images/lamella-upload.png
+  :alt: Screenshot: Astrocyte Lamella file upload
+
+
+Provided Workflows
+^^^^^^^^^^^^^^^^^^
+
+All workflows can be found at the
+`lab page <https://astrocyte.biohpc.swmed.edu/brand/Dean_lab/browse/>`_. View
+each workflow's attached documentation for additional information.
+
+- 3D GPU Deskew Workflow: This workflow normalizes selected ctASLM/light-sheet
+  microscopy image files to OME-Zarr and runs GPU-accelerated shear/rotation
+  operations.
+- 3D GPU Deconvolution Workflow: This workflow normalizes selected microscopy
+  image volumes to OME-Zarr, estimates a blind PSF, and runs GPU-accelerated
+  Richardson-Lucy deconvolution.
+- Neuroglancer Visualization: This workflow visualizes 3D OME-Zarr volumes.
+
+Resource Management
+-------------------
+
+Both BioHPC and Astrocyte bill for compute resource usage based on the time
+and type of nodes allocated. To ensure efficient use of your allocations and
+to avoid unnecessary charges, always try to keep your resource usage to a
+minimum. For information on BioHPC data management, view :doc:`data-management`.
+
+Generic Queues
+^^^^^^^^^^^^^^
+
+If you do not require a specific hardware configuration, you can use the
+generic queues to get your jobs running efficiently:
+
+- super: Generic high-performance CPU node.
+- GPU: Generic GPU node.
+
+These generic queues act as a sliding scale for resources. They will
+automatically attempt to allocate the weakest available node first. If those
+are fully occupied, the system will look to the next strongest node in the
+queue (e.g., if all 128GB nodes are busy, the queue will automatically try to
+place your job on an available 256GB node).
+
+Available BioHPC Nodes
+^^^^^^^^^^^^^^^^^^^^^^
+
+Below is a simplified overview of the available CPU and GPU nodes and what
+they are best suited for.
+
+CPU Nodes
+"""""""""
+
+- 128GB: Entry-level memory node, best for light data processing and basic
+  analysis.
+- 256GB / 256GBv1 / 256GBv2: Standard mid-tier memory nodes, ideal for
+  everyday bioinformatic workflows and average datasets.
+- 384GB: High-memory node, meant for memory-heavy jobs that exceed standard
+  system limits.
+- 512GB: Maximum-capacity memory node, reserved for massive datasets and
+  extreme processing tasks.
+
+GPU Nodes
+"""""""""
+
+- GPU2H200: Ultra-high-performance node with next-generation H200 GPUs for
+  maximum computation speed and massive AI or image processing workloads.
+- GPU4A100 / GPU4H100: Heavy-duty multi-GPU nodes with four top-tier cards,
+  ideal for intensive parallel computing and large-scale deep learning models.
+- GPU4v100: Multi-GPU node with four previous-generation V100 cards for
+  workflows optimized across multiple GPUs.
+- GPUA100 / GPUL4 / GPURTX6k: Modern single-GPU nodes ranging from lightweight
+  acceleration (L4) to high-memory image processing and heavy graphics
+  workflows (A100, RTX 6000).
+- GPUp100 / GPUp4 / GPUp40 / GPUv100s: Standard single and dual GPU nodes
+  suitable for everyday accelerated tasks, smaller image sets, and general
+  GPU-enabled pipelines.
+
+Related pages
+-------------
+
+- :doc:`digital-tools` for account and platform setup
+- :doc:`data-management` for information on handling BioHPC data
+- :doc:`policies` for lab expectations that intersect with record keeping

source/data-management.rst

@@ -12,6 +12,7 @@ Data Storage
   `BioHPC account registration <https://portal.biohpc.swmed.edu/accounts/register/>`_.
 - BioHPC access is a standard requirement for working in the lab, and new users
   are typically directed to an orientation or training session during setup.
+  See :doc:`cloud-usage` if this information has not yet been provided.
 - The primary archive location for lab data is ``/archive``.
 - Organize data routinely and remove material that no longer needs to be kept;
   storage has a real operational cost.
@@ -62,6 +63,7 @@ Brightness and contrast
 Related pages
 -------------
 
+- :doc:`cloud-usage` for BioHPC access, Astrocyte uploads, and compute resource expectations
 - :doc:`digital-tools` for account and platform setup
 - :doc:`figure-preparation` for manuscript-ready figure formatting expectations
 - :doc:`policies` for lab expectations that intersect with record keeping

source/departure.rst

@@ -11,6 +11,8 @@ Before Your Last Day
 - Make sure protocols and key procedures are documented in the appropriate shared system.
 - Organize published data in ``/archive/MIL/publications/`` using a clear project
   naming scheme that others can navigate later.
+- Confirm BioHPC and Astrocyte project materials follow :doc:`cloud-usage`
+  expectations before handing off cloud-based workflows.
 - Remove personal information from lab-managed computers and return any
   lab-owned devices that were assigned to you.
 - Return keys, badges, laptops, and other UTSW property as instructed.

source/digital-tools.rst

@@ -62,10 +62,21 @@ BioHPC
 - Register during onboarding and make sure you understand the storage structure
   used by your projects.
 - See :doc:`data-management` for storage expectations.
+- See :doc:`cloud-usage` for access, remote login, Astrocyte, and compute usage expectations.
+
+Astrocyte
+---------
+
+- Astrocyte is one of the channels offered by the lab for data analysis.
+- An existing BioHPC account is required for Astrocyte access. Login and
+  registration are the same as your BioHPC credentials.
+- See :doc:`data-management` for storage expectations.
+- See :doc:`cloud-usage` for project setup, upload options, provided workflows, and resource management.
 
 Related Pages
 -------------
 
 - :doc:`onboarding`
+- :doc:`cloud-usage`
 - :doc:`data-management`
 - :doc:`working-at-utsw`

source/index.rst

@@ -23,7 +23,7 @@ Explore the site through the public resources below.
       Recent work and publication resources.
 
    :doc:`Lab Resources <resources>`
-      Onboarding, policies, protocols, equipment, and data management.
+      Onboarding, policies, protocols, equipment, cloud usage, and data management.
 
 .. toctree::
    :caption: Contents

source/onboarding.rst

@@ -34,7 +34,9 @@ Computing and records
 - LabArchives is used for digital lab notebooks and shared lab information.
 - BioHPC is the primary storage and compute environment for lab data.
 
-See :doc:`digital-tools` for the operational details of these systems.
+See :doc:`digital-tools` for the operational details of these systems and
+:doc:`cloud-usage` for BioHPC training, VPN access, Astrocyte, and compute
+resource expectations.
 
 Contacts
 --------
@@ -103,4 +105,5 @@ Quick reference
 - Department IT: Hector.EncarnacionPetersen@UTSouthwestern.edu
 - Badge access: BadgeAccessControl@UTSouthwestern.edu
 - Core lab systems: see :doc:`digital-tools`
+- BioHPC, Astrocyte, and remote compute: see :doc:`cloud-usage`
 - Institutional logistics: see :doc:`working-at-utsw`

source/resources.rst

@@ -19,6 +19,7 @@ Overview
 - :doc:`meetings` for recurring meetings, attendance expectations, and presentation formats.
 - :doc:`digital-tools` for email, Slack, Outlook, GitHub, LabArchives, and BioHPC.
 - :doc:`working-at-utsw` for institution-facing logistics such as HR, IR, VPN, shipping, and travel.
+- :doc:`cloud-usage` for BioHPC, Astrocyte, remote access, data staging, and compute resource expectations.
 - :doc:`data-management` for storage expectations and microscopy image best practices.
 - :doc:`figure-preparation` for manuscript-ready figure expectations and formatting standards.
 - :doc:`protocols` for standard operating procedures and shared workflows.
@@ -33,6 +34,7 @@ Overview
    meetings
    digital-tools
    working-at-utsw
+   cloud-usage
    data-management
    figure-preparation
    equipment

source/working-at-utsw.rst

@@ -55,6 +55,7 @@ VPN and Remote Access
 - If you need remote access, set up the UTSW VPN while you are on campus if possible.
 - Limited intranet access is also available through the
   `Citrix Gateway <https://citrixgateway.utsouthwestern.edu/vpn/index.html>`_.
+- BioHPC and Astrocyte remote workflows are covered in :doc:`cloud-usage`.
 
 Equipment Surplus and Decontamination
 -------------------------------------

tests/test_cloud_usage_navigation.py

@@ -0,0 +1,33 @@
+from pathlib import Path
+
+
+ROOT = Path(__file__).resolve().parents[1]
+SOURCE = ROOT / "source"
+
+
+def read_doc(name):
+    return (SOURCE / name).read_text(encoding="utf-8")
+
+
+def test_cloud_usage_is_owned_by_lab_resources_navigation_only():
+    index = read_doc("index.rst")
+    resources = read_doc("resources.rst")
+
+    assert ":doc:`Cloud Usage <cloud-usage>`" not in index
+    assert "\n   cloud-usage\n" not in index
+    assert ":doc:`cloud-usage`" in resources
+    assert "\n   cloud-usage\n" in resources
+
+
+def test_cloud_usage_is_linked_from_resource_paths():
+    expected_links = {
+        "resources.rst": ":doc:`cloud-usage`",
+        "onboarding.rst": ":doc:`cloud-usage`",
+        "working-at-utsw.rst": ":doc:`cloud-usage`",
+        "digital-tools.rst": ":doc:`cloud-usage`",
+        "data-management.rst": ":doc:`cloud-usage`",
+        "departure.rst": ":doc:`cloud-usage`",
+    }
+
+    for filename, link in expected_links.items():
+        assert link in read_doc(filename)