Skip to content

RFC: ARM64 (aarch64) Architecture Support for Cloud Foundry#1530

Open
vighes wants to merge 9 commits into
cloudfoundry:mainfrom
vighes:rfc-arm64-support
Open

RFC: ARM64 (aarch64) Architecture Support for Cloud Foundry#1530
vighes wants to merge 9 commits into
cloudfoundry:mainfrom
vighes:rfc-arm64-support

Conversation

@vighes

@vighes vighes commented Jun 24, 2026

Copy link
Copy Markdown

This RFC proposes adding ARM64 architecture support to Cloud Foundry,
enabling deployments on ARM64-based infrastructure (AWS Graviton, Ampere
Altra, etc.) with 20-40% better price/performance.

A proof-of-concept validated all major CF components (Diego, Garden-runC,
Gorouter, Loggregator, Cloud Controller, UAA, BOSH) compile and execute
on ARM64 with zero source code changes.

Looking forward to community feedback on the proposed workstreams and
open questions.

Preview link

cc @beyhan

@linux-foundation-easycla

linux-foundation-easycla Bot commented Jun 24, 2026

Copy link
Copy Markdown

CLA Not Signed

@beyhan
beyhan requested review from a team, Gerg, beyhan, cweibel, rkoster and stephanme and removed request for a team June 25, 2026 06:17
@beyhan beyhan added toc rfc CFF community RFC labels Jun 25, 2026
Comment thread toc/rfc/rfc-draft-arm64-support.md Outdated
Comment thread toc/rfc/rfc-draft-arm64-support.md Outdated

@rkoster rkoster left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the impact on the ecosystem I think it would only make sense to also include apple silicon support as in scope. Because this would bring a lot of value to the CF Core developers who have MacBooks and now way of easily validating component changes locally.

Comment thread toc/rfc/rfc-draft-arm64-support.md Outdated
Comment thread toc/rfc/rfc-draft-arm64-support.md Outdated
@beyhan beyhan moved this from Inbox to In Progress in CF Community Jun 30, 2026

1. **Stemcell naming convention:** How should ARM64 stemcells be identified? Proposed: `ubuntu-jammy-arm64` alongside existing `ubuntu-jammy` (implicitly x86_64). Open to community input on naming and metadata approach.

2. **CI/CD long-term ownership:** ARM64 CI workers will be provided initially to unblock development. Should long-term ownership transfer to the CF Foundation, or remain as a community-contributed resource?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally eventually we end up with CF Foundation ownership and get some AWS credits to help with offsetting the increase in CI spend.

Also related, these workers are these concourse workers running on ARM?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, these would be Concourse workers running on ARM64. For Go cross-compilation tasks, existing x86_64 workers can produce ARM64 binaries via GOARCH=arm64. But for native compilation (runC with seccomp, Ruby gem native extensions, stemcell building) and integration testing, ARM64 workers are required.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@taylorsilva to my knowledge Concourse is supporting ARM64 and having concourse workers running on ARM64 won't be an issue. Is this correct?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for confirming Concourse ARM64 support.

@vighes

vighes commented Jul 1, 2026

Copy link
Copy Markdown
Author

Given the impact on the ecosystem I think it would only make sense to also include apple silicon support as in scope. Because this would bring a lot of value to the CF Core developers who have MacBooks and now way of easily validating component changes locally.

Our POC cross-compiled all Go components from Apple Silicon (M4 Pro Mac), However full local integration (runC with seccomp, container creation) still requires Linux ARM64 either via Docker/Lima on Mac or a remote ARM64 instance. The Linux ARM64 work directly benefits Apple Silicon developers for compilation and unit testing, though not for full local container deployment. I'll add a note acknowledging this developer workflow benefit.

- Support mixed-architecture cells (ARM64 Diego cells + x86_64 control plane) following the Windows cell model, removed as non-goal
- Add binary blob handling strategy for components shipping pre-compiled binaries (e.g., JDK in uaa-release)
- Add Apple Silicon developer workflow benefit
Comment thread toc/rfc/rfc-draft-arm64-support.md Outdated
@norman-abramovitz

Copy link
Copy Markdown
Contributor

I totally agree with @rkoster on its value for CF developers. It would be much easier to have a local CF where I can generate test CF environments rather than using Docker to create local test environments. It would also help promote Korifi as a development environment, where I am currently trying to get Stratos to run in a Korifi environment.


The App Runtime Deployments Working Group owns `cf-deployment`. Work includes:

- Set up cf-deployment validation against the ARM64 stemcell (similar to existing Jammy/Noble validation)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting up another validation pipeline for ARM64 is not a problem. But we should clarify the cost increase in advance.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, We can provide more details on expected compute hours once we scope the pipeline configurations.

Comment thread toc/rfc/rfc-draft-arm64-support.md
Comment thread toc/rfc/rfc-draft-arm64-support.md Outdated
Comment thread toc/rfc/rfc-draft-arm64-support.md
Comment thread toc/rfc/rfc-draft-arm64-support.md
vighes added 2 commits July 6, 2026 16:32
- Add Paketo CNBs to existing ARM64 support list
- Add BOSH Docker CPI on Apple Silicon to testing scope (ref bosh-deployment#497)
- Add buildpack automation challenges (storage, size, pipelines) to Phase 3
Comment thread toc/rfc/rfc-draft-arm64-support.md
Comment thread toc/rfc/rfc-draft-arm64-support.md Outdated
@wayneeseguin

Copy link
Copy Markdown
Contributor

Given the impact on the ecosystem I think it would only make sense to also include apple silicon support as in scope. Because this would bring a lot of value to the CF Core developers who have MacBooks and now way of easily validating component changes locally.

Our POC cross-compiled all Go components from Apple Silicon (M4 Pro Mac), However full local integration (runC with seccomp, container creation) still requires Linux ARM64 either via Docker/Lima on Mac or a remote ARM64 instance. The Linux ARM64 work directly benefits Apple Silicon developers for compilation and unit testing, though not for full local container deployment. I'll add a note acknowledging this developer workflow benefit.

This comes to mind as a new possibility for running locally on Mac: https://github.com/apple/container

- Add BOSH Director architecture-aware compilation consideration for mixed AMD64/ARM64 environments to Foundational Infrastructure WG scope
- Update stemcell naming open question to reference FIPS naming precedent and add architecture metadata field proposal
@beyhan

beyhan commented Jul 10, 2026

Copy link
Copy Markdown
Member

I went through the comments and resolved the ones that appeared to be clarified. To summarize, there are two "major" questions that still need clarification before we can proceed with the RFC:

  1. How are we going to cover the increased CI/CD costs?
  2. What should the long-term CI/CD ownership model be?

I'll raise these questions during the TOC meeting to work out how we can resolve them.

Comment thread toc/rfc/rfc-draft-arm64-support.md Outdated
The App Runtime Interfaces Working Group owns Cloud Controller (CAPI) and UAA. Work includes:

- Validate Cloud Controller Ruby application runs on ARM64 stemcells (POC validated gem compilation)
- Validate UAA Java WAR deploys and runs on ARM64 JVM (Eclipse Temurin, SAP Machine, or equivalent ARM64 JDK)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UAA is owned by Foundational Infrastructure WG.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Stephan, Corrected — moved UAA validation to the Foundational Infrastructure WG section. Updated in latest commit.

Comment thread toc/rfc/rfc-draft-arm64-support.md Outdated
The ARI Working Group owns the classical buildpacks which have higher adoption in CF today. Note: ARM64 support for Cloud Native Buildpacks (CNBs) is already being addressed by the Paketo WG separately (see [ARM64 Paketo Buildpacks](https://www.cloudfoundry.org/blog/arm64-paketo-buildpacks/)). Work for classical buildpacks includes:

- Produce ARM64 variants of classical buildpack dependencies (JDK, Node.js, Python, Ruby, Go, .NET, PHP, nginx runtimes)
- Produce an ARM64 variant of the `cflinuxfs4` root filesystem

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cflinuxfs5 (not 4).

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed — updated to cflinuxfs5 in the buildpacks section. Thanks for catching the remaining references.

@beyhan

beyhan commented Jul 14, 2026

Copy link
Copy Markdown
Member

I went through the comments and resolved the ones that appeared to be clarified. To summarize, there are two "major" questions that still need clarification before we can proceed with the RFC:

  1. How are we going to cover the increased CI/CD costs?
  2. What should the long-term CI/CD ownership model be?

I'll raise these questions during the TOC meeting to work out how we can resolve them.

We discussed the two points from above and current decision is:

  1. Ask the WG leads to provide rough cost estimation.
  2. Suggest creation of two new areas one for the new ARM64 stemcell and one for the CF stack and buildpacks. By this we can lower entry barrier for the contribution and have clear responsibilities for the beginning.

…ences

- UAA is owned by FI WG, not ARI WG (per @stephanme)
- Fix remaining cflinuxfs4 references to cflinuxfs5
@vighes

vighes commented Jul 14, 2026

Copy link
Copy Markdown
Author

I went through the comments and resolved the ones that appeared to be clarified. To summarize, there are two "major" questions that still need clarification before we can proceed with the RFC:

  1. How are we going to cover the increased CI/CD costs?
  2. What should the long-term CI/CD ownership model be?

I'll raise these questions during the TOC meeting to work out how we can resolve them.

We discussed the two points from above and current decision is:

1. Ask the WG leads to provide rough cost estimation.

2. Suggest creation of two new areas one for the new ARM64 stemcell and one for the CF stack and buildpacks. By this we can lower entry barrier for the contribution and have clear responsibilities for the beginning.

Thanks for the update. The two-area approach (ARM64 stemcell area + CF stack/buildpacks area) makes sense for clear ownership boundaries.

@beyhan

beyhan commented Jul 15, 2026

Copy link
Copy Markdown
Member

I went through the comments and resolved the ones that appeared to be clarified. To summarize, there are two "major" questions that still need clarification before we can proceed with the RFC:

  1. How are we going to cover the increased CI/CD costs?
  2. What should the long-term CI/CD ownership model be?

I'll raise these questions during the TOC meeting to work out how we can resolve them.

We discussed the two points from above and current decision is:

1. Ask the WG leads to provide rough cost estimation.

2. Suggest creation of two new areas one for the new ARM64 stemcell and one for the CF stack and buildpacks. By this we can lower entry barrier for the contribution and have clear responsibilities for the beginning.

Thanks for the update. The two-area approach (ARM64 stemcell area + CF stack/buildpacks area) makes sense for clear ownership boundaries.

I asked about cost estimation in the TOC Slack channel. @vighes it would be great if you could incorporate the second point about the new WG areas into the RFC.

@wayneeseguin wayneeseguin left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added comments, suggest adding the Windows ARM64 as a non-goal explicitly at this time (could always be added later).

Comment thread toc/rfc/rfc-draft-arm64-support.md Outdated
- **Author(s):** Sachin Vighe
- **Status:** Draft
- **Related RFCs:** [rfc-0026-noble-os](https://github.com/cloudfoundry/community/blob/main/toc/rfc/rfc-0026-noble-os.md)
- **Affected Component(s):** bosh-linux-stemcell-builder, Diego, Garden-runC, Gorouter, Loggregator, Cloud Controller, UAA, cf-deployment, buildpacks

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't put my finger on it at the moment but I feel there are some more affected components. bosh-dns? others?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added bosh-agent, bosh-dns, BPM and NATS to the Affected Components. Also added their validation to the relevant workstreams. The Affected Repositories section at the bottom provides the full breakdown.


### Developer Workflow Benefit

ARM64 Linux support also benefits CF core developers on Apple Silicon MacBooks (M1/M2/M3/M4). All Go-based CF components can be cross-compiled locally using `GOOS=linux GOARCH=arm64`, enabling faster development and unit test iteration without needing remote infrastructure. Additionally, an ARM64 stemcell would enable running BOSH with the Docker CPI natively on Apple Silicon (see [bosh-deployment#497](https://github.com/cloudfoundry/bosh-deployment/issues/497)), giving developers a fully local CF environment without Rosetta emulation issues. Validating this local BOSH-lite workflow on Apple Silicon is included in the testing scope.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be a fantastic outcome indeed. We would be able to get a 'cf local' up and running again most likely, especially with apple releasing their containers project.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed! Once ARM64 stemcells exist, BOSH-lite on Docker CPI becomes native on Apple Silicon no more Rosetta issues. Validating this workflow is included in the testing scope.

Comment thread toc/rfc/rfc-draft-arm64-support.md Outdated
- BOSH Acceptance Tests (BATS) pass on ARM64 stemcells
- CF Acceptance Tests (CATs) pass on ARM64 deployment
- Applications can be deployed via standard buildpacks on ARM64 cells
- Performance ≥ 100% of x86_64 baseline for equivalent instance classes

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this always guaranteed? 🤔

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point, not guaranteed across all workload types. Reworded to "comparable" with benchmarking deferred to Phase 2. POC data and public Graviton benchmarks suggest parity or better for Go/Java/Ruby workloads.

Comment thread toc/rfc/rfc-draft-arm64-support.md Outdated

## Non-Goals

1. **Not breaking x86_64 support** — existing deployments continue unchanged

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit confusing, wouldn't it be better to state this as a direct goal vs non-goal, e.g. it is a goal to have no impact on existing?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean maybe it's fine, I find the logic of this section a bit confusing on first read.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I still feel that this section should be rewritten to directly state goals of not breaking things with opt-in support etc...

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restructured, Non-Goals section now focuses purely on what's out of scope. Thanks for flagging.

Comment thread toc/rfc/rfc-draft-arm64-support.md Outdated
BOSH Director (bosh)
Diego (diego-release) + Garden (garden-runc-release) ← critical path

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the whole tree really the 'critical path' ? it either all works or it doesn't.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's right, the whole chain is the critical path. Removed the misleading annotation from the diagram and clarified while the entire chain must work, Garden-runC carries the highest implementation risk specifically due to C code and seccomp.

- Validate the BOSH agent on ARM64 (POC has demonstrated this works)
- Set up CI pipelines producing ARM64 stemcell builds (ARM64 CI workers will be provided)
- Validate the BOSH Director (Ruby) operates correctly on ARM64 (POC validated 134 gems)
- For mixed AMD64/ARM64 environments: address BOSH Director's current single-architecture compilation VM limitation (e.g., support multiple compilation VM types or architecture-aware compilation routing)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is key, for example being able to run Diego Cells in isolation segments on say split intel vs arm would be quite valuable.

- Performance ≥ 100% of x86_64 baseline for equivalent instance classes
- Zero impact on existing x86_64 deployments

## Non-Goals

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps should declare Windows ARM64 support is a non-goal for this effort?

**Testing (1):**
- `cf-acceptance-tests` — Full platform validation (Go)

Remaining repositories (P1/P2) are primarily Go-based and are expected to cross-compile with minimal effort. A detailed repository inventory will be shared alongside this RFC.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might address my initial comments.

- Clarify critical path language in dependency chain diagram
- Rewrite Non-Goals section to separate goals from non-goals clearly
- Add Windows ARM64 as explicit non-goal
- Soften performance success criteria to "comparable" with Phase 2 benchmarking
- Add bosh-dns, bosh-agent, BPM, NATS to affected components
- Add "Proposed New Areas" section for ARM64 Stemcell and CF Stack areas
- Expand workstream details for BOSH DNS and BPM validation
@vighes

vighes commented Jul 15, 2026

Copy link
Copy Markdown
Author

I went through the comments and resolved the ones that appeared to be clarified. To summarize, there are two "major" questions that still need clarification before we can proceed with the RFC:

  1. How are we going to cover the increased CI/CD costs?
  2. What should the long-term CI/CD ownership model be?

I'll raise these questions during the TOC meeting to work out how we can resolve them.

We discussed the two points from above and current decision is:

1. Ask the WG leads to provide rough cost estimation.

2. Suggest creation of two new areas one for the new ARM64 stemcell and one for the CF stack and buildpacks. By this we can lower entry barrier for the contribution and have clear responsibilities for the beginning.

Thanks for the update. The two-area approach (ARM64 stemcell area + CF stack/buildpacks area) makes sense for clear ownership boundaries.

I asked about cost estimation in the TOC Slack channel. @vighes it would be great if you could incorporate the second point about the new WG areas into the RFC.

Hi @beyhan Done — see the "Proposed New Areas" section which proposes two new areas with clear boundaries and scope.

@vighes

vighes commented Jul 15, 2026

Copy link
Copy Markdown
Author

Added comments, suggest adding the Windows ARM64 as a non-goal explicitly at this time (could always be added later).

Added as explicit non-goal #1. There's no Windows Server ARM64 offering from major cloud providers today, so this is firmly out of scope.

Comment thread toc/rfc/rfc-draft-arm64-support.md Outdated
- Run component-level integration tests on ARM64 (Diego inigo tests, Gorouter integration tests)
- Validate the Loggregator pipeline on ARM64
- Validate container networking (Silk CNI, cf-networking) on ARM64
- Validate NATS and BPM on ARM64

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BPM is not part of the ARP working group

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — moved BPM to the Foundational Infrastructure WG section. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

rfc CFF community RFC toc

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.