diff --git a/about/free-trial.html.md b/about/free-trial.html.md
index 846df7a063..58bb89261b 100644
--- a/about/free-trial.html.md
+++ b/about/free-trial.html.md
@@ -30,7 +30,6 @@ These are **not included** in the free trial:
- Dedicated IPv4 addresses
- Access to performance-optimized vCPUs
-- GPU machines (To be deprecated as of 08/01/26)
You can add a credit card from the dashboard at any time during the trial. This lifts the resource limits and keeps your apps running without interruption. **Note: adding a card ends the free trial** and your usage starts counting toward your bill from that point on.
diff --git a/about/pricing.html.markerb b/about/pricing.html.markerb
index 501c3b355b..0868446675 100644
--- a/about/pricing.html.markerb
+++ b/about/pricing.html.markerb
@@ -21,7 +21,7 @@ All organizations (except for Linked Organizations) require a [credit card](/doc
## Compute
-We charge for started and stopped Machines differently. Attached GPUs are charged separately. For more details about how costs are calculated, see [Machine billing](/docs/about/billing/#machine-billing). To understand the difference between `performance` and `shared` CPU types in Machines, see [CPU performance](/docs/machines/cpu-performance).
+We charge for started and stopped Machines differently. For more details about how costs are calculated, see [Machine billing](/docs/about/billing/#machine-billing). To understand the difference between `performance` and `shared` CPU types in Machines, see [CPU performance](/docs/machines/cpu-performance).
### Started Fly Machines
@@ -236,30 +236,6 @@ You **will** be billed separately for data transfer to these external third-part
The [Machine price](#compute) and [volume price](#persistent-storage-volumes) for Fly Postgres are the same as any other Machine and volume you'd run on Fly.io. Assuming the Machines are running all the time, the cost for the preset configurations is about $2/month for a single node cluster for dev projects and from about $82 to $164/month for a three-node production cluster. You don't need to keep the preset configurations, you can [scale your Fly Postgres Machines](/docs/postgres/managing/scaling/) to suit your workload at any time.
-### GPU-enabled Fly Machines (Deprecated)
-
-
-**GPUs are deprecated and will be unavailable after August 1.**
-
-
-Pricing for a GPU-enabled Fly Machine is the price of a standard Fly Machine (see above) plus the price of the attached GPU. Like Machines, GPUs are billed by the second when the attached Machine is running.
-
-On-demand GPU pricing:
-
-* A10: $0.75/hr per GPU
-* L40S: $0.70/hr per GPU
-* A100 40G PCIe: $1.25/hr per GPU
-* A100 80G SXM: $1.50/hr per GPU
-
-Usage terms:
-
-* No minimum usage requirements.
-* Customizable CPU, RAM, and storage options.
-
-Reserved and dedicated options:
-
-* Discounted rates for reserved GPU Machines and dedicated hosts.
-
## Discontinued Plans
Fly.io no longer offers plans to new customers. If you purchased a Launch or Scale plan before October 7, 2024, you can remain on those plans unless you convert to Pay As You Go, delete your payment method, or otherwise stop using Fly.io.
diff --git a/about/support.html.md b/about/support.html.md
index 3f5830777d..5b97cc176b 100644
--- a/about/support.html.md
+++ b/about/support.html.md
@@ -94,7 +94,6 @@ Here are some things to include in your ticket:
**Networking**
**Machines**
-
**GPU Machines**-GPU support to end 08/01/26
**Managed Postgres** (MPG)
**Apps**
**Launch/Deploy** (UI & CLI)
diff --git a/blueprints/shared-nothing.html.markerb b/blueprints/shared-nothing.html.markerb
index c4b192029b..65e133e704 100644
--- a/blueprints/shared-nothing.html.markerb
+++ b/blueprints/shared-nothing.html.markerb
@@ -206,8 +206,6 @@ and the code for that article can be found on [GitHub as fly-apps/pdf-appliance]
This architectural pattern can be applied whenever there is a minority of requests that require an outsized amount
of resources. A second example that comes to mind: I've had requests for audio capture and transcription.
-Setting up a machine that [runs Whisper with Fly GPUs](https://news.ycombinator.com/item?id=39417197) is
-something I plan to explore.
## Backups
diff --git a/deep-dive/index.html.markerb b/deep-dive/index.html.markerb
index 3e4ff3495f..a5989f1432 100644
--- a/deep-dive/index.html.markerb
+++ b/deep-dive/index.html.markerb
@@ -9,7 +9,7 @@ toc: true
-Welcome to our deep dive demo, where you can explore Fly.io more thoroughly, but in a time-boxed way. In one hour or less: get a fully-functioning app running in the first few minutes, and then have enough time left over to understand what you just did, explore how the pieces fit together, and even integrate AI functionality that makes use of GPUs.
+Welcome to our deep dive demo, where you can explore Fly.io more thoroughly, but in a time-boxed way. In one hour or less: get a fully-functioning app running in the first few minutes, and then have enough time left over to understand what you just did and explore how the pieces fit together.
## Goals of the deep dive
diff --git a/elixir/advanced-guides/clustering-from-home-to-your-app-in-fly.html.md b/elixir/advanced-guides/clustering-from-home-to-your-app-in-fly.html.md
index ab1a5562a7..0264e8b2c0 100644
--- a/elixir/advanced-guides/clustering-from-home-to-your-app-in-fly.html.md
+++ b/elixir/advanced-guides/clustering-from-home-to-your-app-in-fly.html.md
@@ -20,14 +20,6 @@ Here we cover _why_ we might want to do this, _what_ is required to make it work
Besides being really cool that we can do this, there are some practical reasons as well.
-### AI/ML development
-
-With a local Elixir application clustered to an application on Fly.io with a GPU attached, we can keep our local development workflow without having a large GPU in our development machine.
-
-When we leverage [Nx](https://github.com/elixir-nx/nx) and [Bumblebee](https://github.com/elixir-nx/bumblebee), we can easily have the clustered application do all the GPU accelerated AI work and return the processing results seamlessly to our local application.
-
-It really does feel like the GPU is local when we work this way.
-
### Develop and debug a distributed application
Building a globally distributed application can be challenging to model locally. With [Fly.io Regions](https://fly.io/docs/reference/regions/), we can deploy our cluster-aware application where it makes sense. Then, our local application joins the global cluster, giving us a close-up view of how the application behaves in a truly globally distributed environment.
@@ -169,6 +161,6 @@ Now you're _really_ doing distributed Elixir!
## Summary
-When we couple Elixir's clustering ability with Fly.io's networking, VPN, and API discoverability, we can easily cluster a locally running Elixir application with a deployed Elixir application. This makes it easy to leverage hosted GPUs for developing AI/ML applications or working on distributed applications.
+When we couple Elixir's clustering ability with Fly.io's networking, VPN, and API discoverability, we can easily cluster a locally running Elixir application with a deployed Elixir application. This makes it easy to develop and debug distributed applications.
The ready-to-use script automates much of the process.
diff --git a/getting-started/launch-demo.html.markerb b/getting-started/launch-demo.html.markerb
index 9987b3c885..0d464e2c85 100644
--- a/getting-started/launch-demo.html.markerb
+++ b/getting-started/launch-demo.html.markerb
@@ -173,4 +173,3 @@ Read about some of the ways you can increase availability, capacity, and perform
* [Autoscale Machines based on load or custom metrics](/docs/reference/autoscaling/)
* [Scale Machine CPU and RAM](/docs/apps/scale-machine/)
* [Scale Machine count](/docs/apps/scale-count/)
-* Try out [Fly GPUs](/docs/gpus/)
diff --git a/getting-started/troubleshooting.html.md b/getting-started/troubleshooting.html.md
index 6be553045e..4ddc6532e2 100644
--- a/getting-started/troubleshooting.html.md
+++ b/getting-started/troubleshooting.html.md
@@ -104,7 +104,7 @@ fly auth login
1. **A Fly registry incident.** Check [status.flyio.net](https://status.flyio.net). If there's an active incident, wait it out or subscribe for updates.
-Note the image size limits: **8GB** for standard Machines, **50GB** for GPU Machines. If your image exceeds these limits, the push fails.
+Note the image size limit of **8GB**. If your image exceeds this limit, the push fails.
### 403 Forbidden
@@ -199,7 +199,7 @@ Secrets set with `fly secrets set` are available as environment variables at run
### Image size limit
-Standard (non-GPU) Machines have an **8GB rootfs limit**. GPU Machines allow up to **50GB**.
+Machines have an **8GB rootfs limit**.
If your image is too large:
diff --git a/gpus/getting-started-gpus.html.md b/gpus/getting-started-gpus.html.md
deleted file mode 100644
index f715dccef2..0000000000
--- a/gpus/getting-started-gpus.html.md
+++ /dev/null
@@ -1,111 +0,0 @@
----
-title: Getting Started with Fly GPUs
-layout: docs
-nav: firecracker
----
-
-
-
-
-
-
-**GPUs are deprecated and will be unavailable after August 1.**
-
-
-## How do I run a GPU Machine on Fly.io?
-
-A Fly GPU Machine works very similarly to a CPU-only (or "normal") Fly Machine, and has access to the same platform features by default. It boots up with GPU drivers installed and you can run `nvidia-smi` right away. So running a Fly App that uses CUDA compute is a slight variation on running any Fly App. In a nutshell:
-
-1. Make sure GPUs are enabled on your Fly Organization.
-1. Tell the Fly Platform to provision your Machines on Fly GPU hosts.
-1. Provide a Docker image or Dockerfile for your project that installs the NVIDIA libraries your project uses, alongside the rest of your project.
-1. Carry on as you would for any Fly App, tailoring it to the needs of your application.
-
-
-**Aside:** Firecracker doesn't do GPUs, so GPU-enabled Machines run under Cloud Hypervisor instead. This means you'll see some different output in logs, but in general you don't have to think about this.
-
-
-## Placing your Machines on GPU hosts
-
-Not all of Fly.io's physical servers have GPUs attached to them, so you have to tell the platform your requirements before the Machine is created on a host.
-
-Configuration is done by different means depending on whether you're using Fly Launch to manage your app, or running Machines individually using the API or the [`fly machine run` command](/docs/machines/run/).
-
-### Specify the GPU model
-
-Assuming that you're going the [Fly Launch](/docs/launch/) way and configuring your Machines app-wide, you can include a line like the following in your `fly.toml` file before running `fly deploy` for the first time:
-
-```
-vm.size = "a100-40gb"
-```
-
-The `a100-40gb` preset is the `a100-pcie-40gb` GPU with the `performance-8x` CPU config and 32GB RAM. You can exert more granular control over resources using the [`[[vm]]` table](/docs/reference/configuration/#the-vm-section) in `fly.toml`.
-
-Run `fly platform vm-sizes` to get the list of presets, and check out the [pricing page](/docs/about/pricing/#gpus-and-fly-machines) to help you plan ahead.
-
-### Specify the region
-
-But you're not done! For `fly deploy` to succeed in provisioning your Machine(s), the app's `primary_region` must match a region that hosts the kind of GPU you're asking for. At the time of writing, `a100-40gb` GPUs are only available in `ord`. Set the initial region in `fly.toml`:
-
-```
-primary_region = "ord" # If you change this, ensure it's to a region that offers the GPU model you want
-```
-
-Currently GPUs are available in the following regions:
-
-- `a10`: `ord`
-- `l40s`: `ord`
-- `a100-40gb`: `ord`
-- `a100-80gb`: `iad`, `sjc`, `syd`, `ams`
-
-
-### Change GPU model
-
-GPU models are located in different regions, so you'll likely have to destroy any existing Machines before redeploying using a different GPU spec.
-
-
-## Choosing a Docker base image
-
-Many open-source ML projects have ready-made Docker images; for example: [Ollama](https://ollama.ai/blog/ollama-is-now-available-as-an-official-docker-image+external), [Basaran](https://github.com/hyperonym/basaran+external), [LocalAI](https://localai.io/basics/getting_started/+external). Check out [GitHub `fly-apps` repos with the `gpu` topic](https://github.com/orgs/fly-apps/repositories?q=topic%3Agpu+external) for some examples.
-
-If you're building your own image, choose a base image that [NVIDIA supports](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#system-requirements+external); `ubuntu:22.04` is a solid choice that's compatible with the CUDA driver GPU Machines use. You can launch a vanilla Ubuntu image and run `nvidia-smi` with no further setup.
-
-From there, install whatever packages your project needs. This will include some libraries from NVIDIA if you want to do something more than admire the output of `nvidia-smi`.
-
-## Installing NVIDIA libraries
-
-You don't need to install any `cuda-drivers` packages in the Docker image, but you'll want some subset of [NVIDIA's GPU-accelerated libraries](https://developer.nvidia.com/gpu-accelerated-libraries+external). `libcublas-12-2` (linear algebra) and `libcudnn8` (deep-learning primitives) are a common combination, along with [`cuda-nvcc`](https://developer.nvidia.com/cuda-llvm-compiler+external) for compiling stuff with CUDA support.
-
-In general, you'll install NVIDIA libs using your Linux package manager. In a Python environment, it's possible to skip system-wide installation and use pip packages instead.
-
-Tips:
-
-- Be deliberate about how much cruft you put in your Docker image. Avoid meta-packages like `cuda-runtime-*`.
-- `cuda-libraries-12-2` is a convenient, but bulky, start. Once you know what libs are needed at build and runtime, pick accordingly to optimize final image size.
-- Use multi-stage docker builds as much as possible.
-
-To install packages from NVIDIA repos, you'll need the `ca-certificates` and `cuda-keyring` packages first.
-
-## Where to store data
-
-Machine learning tends to involve large quantities of data. We're working with a few constraints:
-
-- Large Docker images (many gigabytes) can be very unwieldy to push and pull, particularly over large geographical distances.
-- The root file system of a Fly Machine is ephemeral -- it's reset from its Docker image on every restart. It's also limited to 50GB on GPU-enabled Machines.
-- Fly Volumes are limited to 500GB, and are attached to a physical server. The Machine must run on the same hardware as the volume it mounts.
-
-Unless you've got a constant workload, you'll likely want to shut down GPU Machines when they're not needed—you can do this manually with `fly machine stop`, [have the main process exit](/docs/launch/autostop-autostart/#apps-that-shut-down-when-idle) when idle, or use the Fly Proxy [autostop and autostart](/docs/launch/autostop-autostart/) features—to save money. Saving data on a persistent Fly Volume means you don't have to download large amounts of data, or reconstitute a large Docker image into a rootfs, whenever the Machine restarts. You'll probably want to store models, at least, on a volume.
-
-## Using swap
-
-Designing your workload and provisioning appropriate resources for it are the first line of defense against losing work by running out of memory (system RAM or VRAM).
-
-You can also enable swap for system RAM on a Fly Machine, simply by including a line like the following in `fly.toml`:
-
-```
-swap_size_mb = 8192 # This enables 8GB swap
-```
-
-Keep in mind that this consumes the commensurate amount of space on the Machine's root file system, leaving less capacity for whatever else you want to store there.
-
-If you need more system RAM and faster performance, also scale up with `fly scale memory`.
diff --git a/gpus/gpu-quickstart.html.markerb b/gpus/gpu-quickstart.html.markerb
deleted file mode 100644
index 90b4eba634..0000000000
--- a/gpus/gpu-quickstart.html.markerb
+++ /dev/null
@@ -1,108 +0,0 @@
----
-title: Fly GPUs quickstart
-layout: docs
-toc: false
-nav: firecracker
----
-
-
-**GPUs are deprecated and will be unavailable after August 1.**
-
-
-1. You can use any base image for your Dockerfile, but it is convenient to base it on `ubuntu:22.04` and install libraries from NVIDIA's official apt repository: `RUN apt install -y cuda-nvcc-12-2 libcublas-12-2 libcudnn8` is usually enough.
-
-
- **Notes**:
- - Do not install meta packages like: `cuda-runtime-*`
- - `cuda-libraries-12-2` is good, but a bulky start. Once you know what libs are needed at build and runtime, please pick accordingly to optimize final image size.
- - Use multi-stage docker builds as much as possible.
-
-
-1. From flyctl, create an app using either `fly launch` or `fly apps create`.
-
-
- **Note**: GPUs are not available in all regions. There are these GPU types available: Nvidia A10, L40S, A100-PCIe-40GB, and A100-SXM4-80GB.
-
- Currently GPUs are available in the following regions:
- - `a10`: `ord`
- - `l40s`: `ord`
- - `a100-40gb`: `ord`
- - `a100-80gb`: `iad`, `sjc`, `syd`, `ams`
-
-
-1. Create or modify the `fly.toml` config file in the project source directory, replacing values with your own:
-
- ```toml
- app = "my-gpu-app"
- primary_region = "ord"
- vm.size = "a100-40gb"
-
- # Use a volume to store LLMs or any big file that doesn't fit in a Docker image
- [[mounts]]
- source = "data"
- destination = "/data"
-
- [http_service]
- internal_port = 8080
- auto_stop_machines = false
- ```
-
-
- **Notes**:
- - Make sure `vm.size` is set in `fly.toml`, valid values are `a10`, `l40s`, `a100-40gb` and `a100-80gb`.
- - Make sure to include a `[[mounts]]` section in `fly.toml`.
- - The volume gets created automatically by `fly deploy`.
- - Use the volume to store the models and large files that can't be shipped as a docker image.
-
-
-
-1. Deploy your app:
-
- ```cmd
- fly deploy
- ```
-
-That's pretty much it to get an app running with a Machine on a GPU.
-
-## Volumes and GPU Machines
-
-
-**Important**: If you create any additional volumes, they need to be created with the same constraints as your Machine.
-
-
-Here's an example of creating a new one hundred gigabyte volume for storing ML models in the `ord` region, on a machine with a GPU:
-
-```cmd
-fly volumes create models \
- --size 100 \
- --vm-gpu-kind a100-40gb \
- --region ord
-```
-
-Example Dockerfile:
-
-```
-FROM ubuntu:22.04 as base
-RUN apt update -q && apt install -y ca-certificates wget && \
- wget -qO /cuda-keyring.deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb && \
- dpkg -i /cuda-keyring.deb && apt update -q
-
-FROM base as builder
-RUN apt install -y --no-install-recommends git cuda-nvcc-12-2
-RUN git clone --depth=1 https://github.com/nvidia/cuda-samples.git /cuda-samples
-RUN cd /cuda-samples/Samples/1_Utilities/deviceQuery && \
- make && install -m 755 deviceQuery /usr/local/bin
-
-FROM base as runtime
-#RUN apt install -y --no-install-recommends libcudnn8 libcublas-12-2
-COPY --from=builder /usr/local/bin/deviceQuery /usr/local/bin/deviceQuery
-CMD ["sleep", "inf"]
-```
-
-## Examples using Fly GPUs
-
-- Elixir Llama2-13b on Fly GPUs: https://gist.github.com/chrismccord/59a5e81f144a4dfb4bf0a8c3f2673131
-- Github fly-apps repos with the `gpu` topic: https://github.com/orgs/fly-apps/repositories?q=topic%3Agpu
-- Fly.io CUDA example: https://gist.github.com/dangra/f8123001fe0f2453a8cd638b89738465
-- Deploying CLIP on Fly.io: https://gist.github.com/simonw/52c7734e34cac2b26ea1378845674edc
-
diff --git a/gpus/index.html.md b/gpus/index.html.md
deleted file mode 100644
index e38a1626eb..0000000000
--- a/gpus/index.html.md
+++ /dev/null
@@ -1,50 +0,0 @@
----
-title: Fly GPUs
-layout: docs
-nav: firecracker
-toc: false
----
-
-
-**GPUs are deprecated and will be unavailable after August 1.**
-
-
-Fly.io has GPUs! If you have workloads that would benefit from GPU acceleration, Fly GPU Machines may be for you.
-
-
-
-
-
-## What can I use Fly GPUs for?
-
-Four models of GPU are available: A10, L40S, NVIDIA A100 40G PCIe and A100 80G SXM.
-
-A100 units are all about the tensor cores, and are positioned for inference, model training, and intensive high-precision computation tasks like scientific simulations. As their names suggest, they have 40GB and 80GB of GPU memory. ([A100 datasheet](https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/a100/pdf/nvidia-a100-datasheet-nvidia-us-2188504-web.pdf+external))
-
-L40S cards are all-rounders; they've got tensor cores, RT cores, and NVENC/NVDEC, and have 48GB of GPU RAM. Choose the L40S to accelerate graphics or video workloads, as well as for inference. ([L40S datasheet](https://resources.nvidia.com/en-us-l40s/l40s-datasheet-28413+external))
-
-A10 cards are all-arounders with less GPU RAM. They've got tensor cores, shader cores, NVENC/NVDEC, and can run Llama 3 8B at float16 without breaking the bank. Choose the A10 when you don't need more than 8 billion parameters. This works great for smaller large language models, Stable Diffusion, and other such workflows. ([A10 datasheet](https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/a10/pdf/a10-datasheet.pdf+external))
-
-
-Right now each Fly GPU Machine uses a single full GPU. A single GPU is well suited to rendering, encoding/decoding, inference, and a smidgen of fine tuning. Training large models from scratch requires much, much beefier resources.
-
-Go to the [GPU Quickstart](https://fly.io/docs/gpus/gpu-quickstart/) to get off the ground fast, or read more practicalities in [Getting started with Fly GPUs](/docs/gpus/getting-started-gpus/).
-
-## Regions with GPUs
-
-Currently GPUs are available in the following regions:
-
-- `a10`: `ord`
-- `l40s`: `ord`
-- `a100-40gb`: `ord`
-- `a100-80gb`: `iad`, `sjc`, `syd`, `ams`
-
-## Examples
-
-Here's some more inspiration for your GPU Machines project:
-
-- [Python GPU Dev Machine](/docs/gpus/python-gpu-example/)
-- [Elixir Llama2-13b on Fly.io GPUs](https://gist.github.com/chrismccord/59a5e81f144a4dfb4bf0a8c3f2673131)
-- [Fly.io CUDA example](https://gist.github.com/dangra/f8123001fe0f2453a8cd638b89738465)
-- [Deploying CLIP on Fly.io](https://gist.github.com/simonw/52c7734e34cac2b26ea1378845674edc)
-- [GitHub `fly-apps` repos with the `gpu` topic](https://github.com/orgs/fly-apps/repositories?q=topic%3Agpu)
diff --git a/gpus/python-gpu-example.html.md b/gpus/python-gpu-example.html.md
deleted file mode 100644
index a04c6ff7f4..0000000000
--- a/gpus/python-gpu-example.html.md
+++ /dev/null
@@ -1,250 +0,0 @@
----
-title: Python GPU Dev Machine
-layout: docs
-nav: firecracker
----
-
-
-**GPUs are deprecated and will be unavailable after August 1.**
-
-
-This is a worked example for a CUDA-enabled Python template environment on a Fly GPU Machine, for working with ML models.
-
-We'll start with a minimal Ubuntu Linux, add a non-root user, and set up a Python virtual environment for a project, with Jupyter Notebook installed. NVIDIA libraries that the project uses can be installed to the persistent disk as needed, using pip.
-
-## Deployment to Fly.io
-### Create a Fly App
-```cmd
-fly apps create
-```
-
-### Clone the example repo
-```cmd
-git clone git@github.com:fly-apps/python_gpu_example.git && cd python_gpu_example
-```
-
-### Edit app configuration in `fly.toml`
-* Change `app` to match the name of the app you just created.
-* Optionally, `primary_region`—make sure to choose a [region in which GPU Machines are available](/docs/gpus/#regions-with-gpus).
-* Optionally, change the NONROOT_USER `build.arg`—if you do, also edit the `destination` of the volume mount in `mounts` to match.
-* Optionally, tweak `swap_size_mb`.
-
-### Deploy
-
-```cmd
- fly deploy
-```
-
- The `fly deploy` command launches the app, provisioning initial resources on first run. In this case, it creates one Fly Machine VM and one Fly Volume. When an app is configured to use a volume, `fly deploy` does not create a redundant or standby Machine.
-
-## Using the Machine
-
-### Jupyter notebook
-
-The easiest way to visit a private Fly App with the browser is with the `fly proxy` command, which proxies a local port to a Machine.
-
-```
-fly proxy 8888:8888
-```
-
-Run `fly logs` to find a line like
-
-```
-http://127.0.0.1:8888/tree?token=c5fe8a87d8c00dd16637f0d4a1d8df7e3590c6a6064bbb6b
-```
-
-from Jupyter. Visit that link in the browser and you can start up a notebook.
-
-
-### Terminal
-
-Connect to the Machine using `fly ssh console`. To activate the configured venv, do the following:
-
-```
-# su pythonuser
-$ cd ~/project/
-$ source ~/project-venv/bin/activate
-```
-
-Then you can install new pip packages to the persistent volume, download models, and run the Python REPL or execute scripts.
-
-To deactivate the venv, type `deactivate`. To drop back to the root user, hit CTRL-D.
-
-As the root user, you can use apt to manage system-wide software. Anything you install with apt is installed to the Machine's root file system, which means it disappears when the Machine next shuts down, so if you find yourself doing this, remember to add any new packages to the Dockerfile, ready to be built into the image on the next deployment.
-
-## Fly.io-specific things
-Fly Launch doesn't have a scanner that will set this up just how we want, so the prep looks a lot like preparing a Docker container. We're configuring and running a Fly Machine instead, of course.
-* We'll use `fly deploy` to launch the Machine using configuration stored in the Fly Launch app config file, `fly.toml`
-* Persistent storage is provided by a Fly Volume attached to the Machine
-* Fly GPU Machines come configured to use their GPU hardware, with NVIDIA drivers installed. You can launch a vanilla Ubuntu image and run nvidia-smi with no further setup
-* This project makes use of Fly.io IPv6 private networking. It could also be configured with a Fly Proxy service so it's available via a public Anycast or private Flycast IP address
-
-## General considerations
-There's no one right way to set up a project like this. Here are some of the considerations that went into the example:
-
-### Data storage
-Machine learning tends to involve large quantities of data. We're working with a few constraints:
-* Large Docker images (many gigabytes) can be very unwieldy to push and pull, particularly over large geographical distances.
-* The root file system of a Fly Machine is ephemeral—it's reset from its Docker image on every restart. It's also limited to 50GB on GPU-enabled Machines.
-* Fly Volumes are limited to 500GB, and are attached to a physical server. The Machine must run on the same hardware as the volume it mounts.
-
-We want to shut down GPU Machines when they're not needed, either manually with `fly machine stop`, or using the Fly Proxy autostop and autostart features, so it's not desirable to download many GB of models or libraries whenever the Machine restarts.
-
-The compromise we use here is to generate a sub-1GB Docker image and store the project's pip packages and any downloaded data on the Fly Volume. This keeps all pip dependencies together in one venv for easy coordination and flexibility. With a well-established workload, you might make a different calculation; maybe all the projects deps actually fit in a manageable Docker image, and you can dispense with the volume storage, or some packages can be installed system-wide with apt, leaving less to manage with pip.
-
-### Compute
-The GPUs available at this time are `a100-sxm4-80gb`, `a100-pcie-40gb` and `l40s`; you can use one GPU per Machine. We're not currently looking at model training on a massive scale; with careful design we can certainly do some reasonable inference on a single one of these cards. Here we're looking at running models manually so we'll stick with a single Machine, but an obvious use for Fly GPU Machines is as a "stateless" service for an app whose front end and any other components run on cheaper CPU-only Machines. This allows for independent horizontal scaling of front and back ends, as well as traffic-based capacity scaling by starting and stopping Machines.
-
-At this time, Fly GPU Machines are provisioned with the `performance-8x` CPU config and 32GB RAM by default. Playing very crudely with a language model, I found it easy to out-of-memory kill my Jupyter (Python) kernel with 32GB of RAM. If you need more RAM and fastest performance, scale up with `fly scale memory`, but losing work is annoying, so it's worth enabling swap.
-
-## Implementation specifics
-
-### App configuration
-The example `fly.toml` file does the following:
-
-* Sets the name of the app to deploy to.
-* Sets the app's primary region, where `fly deploy` will put the initial Machine. Deployment will fail if this region doesn't have GPUs available (or is out of capacity).
-* Specifies Machine resources (most crucially a GPU) using a preset (`a100-40gb`).
-* Configures swap.
-* Sets a build argument that the Dockerfile uses to set the name of the non-root user.
-* Configures a volume mount. `fly deploy` provisions a new volume on first run (or when there's no Machine or Volume present on the app). You can specify the size for the initial volume here if desired.
-
-```toml
-app = "cgpu-allinone" # Change this to your app's name
-primary_region = "ord" # If you change this, ensure it's to a region that offers GPUs
-vm.size = "a100-40gb" # A shorthand for the size preset in the [[vm]] section
-swap_size_mb = 32768 # This enables 32GB swap
-
-[build]
- [build.args]
- NONROOT_USER = "pythonuser" # Access this value in the Dockerfile using `ARG NONROOT_USER`
-
-# Use a volume to store LLMs or any big file that doesn't fit in a Docker image
-# This whole volume will be the non-root user's home directory
-[mounts]
-source = "data"
-destination = "/home/pythonuser" # Make sure this matches the value of the NONROOT_USER build arg
-# initial_size = "50gb" # Uncomment to set the size for the volume created on first deployment
-```
-
-### The Docker image
-
-Ready-made Docker images exist for many ML-related projects. There is a CUDA-enabled Jupyter Docker image, but it's kind of a black box and it's not maintained by the Jupyter folks, though they link to it.
-
-Here we'll go step by step from a small Ubuntu image, installing and configuring a Python development environment that can use the GPU's CUDA capabilities, kind of like we would with a normal computer.
-
-Here's a summary of what the Dockerfile does;
-
-* Uses Ubuntu 22.04 as a base image, as it's compatible with the NVIDIA drivers on Fly Machines.
-* Installs system-wide packages with the apt package manager: python3, python3-pip, python3-venv, python3-wheel, git, nano (substitute your favourite terminal-compatible text editor here).
-* Adds a non-root user to own the Python venv and run things. Gets the name for this user from a `[build.arg]` set in `fly.toml`.
-* Copies the scripts for root and the user to run at startup.
-* CMD invokes the first script with the non-root user name as argument.
-
-```Dockerfile
-FROM ubuntu:22.04
-RUN apt update -q && apt install -y python3 python3-pip python3-venv python3-wheel git nano && \
- apt clean && rm -f /var/lib/apt/lists/*_*
-
-ARG NONROOT_USER
-RUN echo "User will be $NONROOT_USER"
-ENV PYTHON_USER=$NONROOT_USER
-
-# Create unprivileged user with a home dir and using bash
-RUN useradd -ms /bin/bash $PYTHON_USER
-
-COPY --chmod=0755 ./entrypoint.sh ./entrypoint.sh
-COPY --chown=$PYTHON_USER:$PYTHON_USER --chmod=0755 ./post-initialization.sh ./post-initialization.sh
-# If you have a requirements.txt for the project, uncomment this and
-# adjust post-initialization.sh to use it
-# COPY --chown=$PYTHON_USER:$PYTHON_USER requirements.txt .
-
-# CMD ["sleep", "inf"]
-CMD ["/bin/bash", "-c", "./entrypoint.sh $PYTHON_USER"]
-```
-
-### Entrypoint script
-When this Machine boots, it runs the script `entrypoint.sh` as root.
-
-This script gives the non-root user (whose username is set to `pythonuser` via a build argument in `fly.toml`) ownership of the non-root user's home directory. This is necessary, because we're mounting a Fly Volume over that point in the Machine file system.
-
-Then it runs `nvidia-smi` to make sure the GPU drivers are loaded and the device is ready for the non-root user to use.
-
-Then it runs the next script (`post-initialization.sh`) as the non-root (`pythonuser`) user.
-
-You could instead have it run `sleep inf` here, and `fly ssh console` into the Machine after deployment to finish setting up the environment on the persistent volume.
-
-```bash
-#!/bin/bash
-
-USERNAME=$1
-
-echo "Inside entrypoint script."
-chown $USERNAME:$USERNAME /home/$USERNAME
-nvidia-smi # This ensures the driver is initialized so that non-root user can use the GPU
-
-echo "About to run post-initialization script as $USERNAME."
-su -c "bash ./post-initialization.sh" $USERNAME
-```
-
-### Post-initialization script
-This script, `post-initialization.sh`, runs as the non-root user. It activates a virtual environment for the project and runs Jupyter from a project directory. It also puts some things on the persistent home directory, if they're absent. More specifically, it does this:
-
-* Ensures there's a dir called `~/project` with a Python virtual environment created
-* Activates this venv
-* Uses the presence or absence of the `jupyter` pip package as a proxy for whether it's the first run or not (if there was a venv, then it's not the first run, but it checks anyway). If Jupyter isn't installed, it installs it. Tailor this to whatever pip packages you want. If you want to run Jupyter on boot, `jupyter` is the only package you absolutely need here. You can install more pip packages persistently straight from the Jupyter notebook interface.
-* Starts a Jupyter server on the Machine's private IPv6 address so it's only accessible using Fly.io private networking; that is, over a WireGuard connection (including user-mode WireGuard with the `fly proxy` command)
-
-```bash
-#!/bin/bash
-
-PROJECT_DIR="project"
-VENV_DIR="$PROJECT_DIR-venv"
-
-echo "Running post-initialization script as $USER."
-echo "Entering $USER's home dir"
-cd ~
-echo "Creating venv dir for $PROJECT_DIR if it doesn't exist"
-mkdir -p $VENV_DIR
-echo "Creating venv in $VENV_DIR if it doesn't exist"
-
-if [ ! -f "$VENV_DIR/pyvenv.cfg" ]; then
- echo "No 'pyvenv.cfg' file found in $VENV_DIR; creating a venv"
- python3 -m venv $VENV_DIR
-fi
-
-echo "Activating venv in $VENV_DIR"
-source $VENV_DIR/bin/activate
-
-echo "Creating dir $PROJECT_DIR if it doesn't exist"
-mkdir -p $PROJECT_DIR && cd $PROJECT_DIR
-# If you want to get project Python deps using requirements.txt, uncomment this and
-# adjust the Dockerfile to copy the file into the image
-# cp /requirements.txt .
-
-# The following only installs pip packages if Jupyter isn't yet installed;
-# essentially on first boot.
-if pip show jupyter &> /dev/null; then
- echo "Jupyter is installed with pip."
-else
- echo "Installing packages with pip"
- # Uncomment to use requirements.txt. You can also install more packages
- # after deployment. This Python venv lives on the persistent Fly Volume.
- # pip install -r requirements.txt
-
- # Install from scratch without a requirements.txt; some examples:
- pip install jupyter
- # pip install numpy torch # numpy isn't getting installed as a dep of torch so do it explicitly
- # pip install diffusers transformers accelerate # HuggingFace libs for specific projects
-fi
-
-echo "Starting Jupyter notebook server!"
-jupyter notebook --ip $FLY_PRIVATE_IP --no-browser
-
-# If you don't want Jupyter, use the `sleep inf` command instead, and
-# `fly ssh console` into the Machine to interact with it.
-# sleep inf
-```
-
-
-This example project is meant to be a transparent template you can build on. Tailor this script to install different packages, use different directories, install from `requirements.txt`, or even skip Jupyter and use the `sleep inf` command to keep the Machine running so you can `fly ssh console` in and just work from the terminal or the Python REPL.
diff --git a/kubernetes/using-gpus.html.markerb b/kubernetes/using-gpus.html.markerb
deleted file mode 100644
index 77d0cb160a..0000000000
--- a/kubernetes/using-gpus.html.markerb
+++ /dev/null
@@ -1,47 +0,0 @@
----
-title: Using GPUs with FKS
-layout: docs
-toc: false
-nav: firecracker
----
-
-
-**GPUs are deprecated and will be unavailable after August 1.**
-
-
-
-Fly Kubernetes is in closed beta and not recommended for critical production usage. To report issues or provide feedback, email us at beta@fly.io.
-
-
-GPUs are available on Fly Kubernetes. Information about our GPUs can be found in our [Fly GPUs documentation](https://fly.io/docs/gpus/#what-can-i-use-fly-gpus-for).
-
-GPUs are consumed by requesting a GPU resource, similarly to requesting `cpu` or `memory`. There is a custom resource `gpu.fly.io/` that is used to request
-GPUs. Note that:
-
-* The available GPU types are: `a10`, `l40s`, `a100-pcie-40gb` and `a100-sxm4-80gb`. Check the [documentation](https://fly.io/docs/gpus/#regions-with-gpus) for which regions they are available in.
-* Pods are deployed in the same region as your cluster. You can place your Pod in a particular region by adding a `fly.io/region: ` annotation to your Pod's metadata.
-* GPU resource requests should only specify the `limits` section.
-* You can specify CPU and memory resources alongside GPU resources. The minimum number of cores supported is 2 and the minimum amount of memory is 4096 MiB. The VMs
-deployed will always be performance Machines.
-* The valid number of GPUs you can request are: 1, 2, 4 and 8.
-* You can only request one type of GPU at a time
-
-Below is an example of a Pod with a GPU:
-
-```yaml
-apiVersion: v1
-kind: Pod
-metadata:
- name: ollama
- annotations:
- fly.io/region: ams # optional
-spec:
- containers:
- - name: ollama
- image: ollama/ollama:latest
- resources:
- limits:
- gpu.fly.io/a100-80gb: 1
-```
-
-This will deploy a GPU Machine with the size `a100-80gb` with 1 GPU core in the region `ams`.
diff --git a/partials/_firecracker_nav.html.erb b/partials/_firecracker_nav.html.erb
index 1e146282e7..4fe76c8738 100644
--- a/partials/_firecracker_nav.html.erb
+++ b/partials/_firecracker_nav.html.erb
@@ -81,16 +81,6 @@
{ text: "Supported Postgres Extensions", path: "/docs/mpg/extensions/" }
]
},
- {
- title: "Fly GPUs",
- path: "/docs/gpus/gpu-quickstart",
- open: false,
- links: [
- { text: "GPU Quickstart", path: "/docs/gpus/gpu-quickstart/" },
- { text: "Getting Started with GPU Machines", path: "/docs/gpus/getting-started-gpus/" },
- { text: "Python GPU Dev Machine", path: "/docs/gpus/python-gpu-example/" }
- ]
- },
{
title: "Databases & Storage",
path: "/docs/database-storage-guides/",
@@ -122,7 +112,6 @@
{ text: "Create an FKS Cluster", path: "/docs/kubernetes/clusters/" },
{ text: "Connect to an FKS Cluster", path: "/docs/kubernetes/connect-clusters/" },
{ text: "Configure FKS Services", path: "/docs/kubernetes/services/" },
- { text: "Use GPUs with FKS", path: "/docs/kubernetes/using-gpus/" },
{ text: "Use Volumes with FKS", path: "/docs/kubernetes/using-volumes/" }
]
},
diff --git a/reference/suspend-resume.html.md b/reference/suspend-resume.html.md
index f9f187bba1..66d94999b1 100644
--- a/reference/suspend-resume.html.md
+++ b/reference/suspend-resume.html.md
@@ -86,7 +86,6 @@ A machine can use suspend if it has:
- **≤ 2 GB** memory (For larger memory sizes, suspend is discouraged due to increased suspend times)
- **No** [**swap**](https://fly.io/docs/reference/configuration/#swap_size_mb-option) **configured**
- **No** [**schedule**](https://fly.io/docs/machines/flyctl/fly-machine-run/#start-a-machine-on-a-schedule) **configured**
-- **No GPU configured** (GPUs will be deprecated as of 08/01/26)
- Been updated since **June 20, 2024 20:00 UTC**
If you have an older machine, or you’re not sure when it was last updated, you can bring it up to date with:
diff --git a/volumes/overview.html.markerb b/volumes/overview.html.markerb
index e74043837c..1709c1612c 100644
--- a/volumes/overview.html.markerb
+++ b/volumes/overview.html.markerb
@@ -53,7 +53,7 @@ Volumes exist on a single server in a single [region](/docs/reference/regions/).
To prevent a single server hardware failure from taking down your app, it is better if each volume is placed in a separate hardware zone. A separate hardware zone is just another way of saying a different server. We default to separate hardware zones when you create volumes with the [`fly volumes create`](/docs/flyctl/volumes-create/) command, but this is not guaranteed when creating Fly Volumes through other methods. Note that having each volume in a separate hardware zone limits the number of volumes your app can have in a region. If you need more volumes in a region than there are distinct hardware zones, you can set `--require-unique-zone` to `false` when you run [`fly volumes create`](/docs/flyctl/volumes-create/).
-You can hint what type of compute load you expect to use the volume for by using the `--vm-XXX` options with the [`fly volumes create`](/docs/flyctl/volumes-create/) command. For example, to create a volume you want to use with an L40S GPU Machine, set `--vm-gpu-kind=l40s` to ensure that your new volume is placed on GPU hardware.
+You can hint what type of compute load you expect to use the volume for by using the `--vm-XXX` options with the [`fly volumes create`](/docs/flyctl/volumes-create/) command. For example, if you plan to attach a `performance-4x` Machine to the volume, set `--vm-size=performance-4x` to place the volume on a host that has capacity for that Machine. Without a hint, your volume can end up on a host with room for the volume but not for the Machine you want to attach to it.
## Volume encryption