Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
24 changes: 17 additions & 7 deletions .bazelrc
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
# Bazel 7.7.0 remains pinned in .bazelversion. Bzlmod is TensorBoard's only
# supported dependency-resolution entry point. WORKSPACE.bzlmod is an internal
# compatibility bridge for dependencies that do not yet expose module APIs.
# Bazel must still evaluate that bridge, so --noenable_workspace is not yet
# supported even though --noenable_bzlmod is intentionally unsupported.
# Bazel 8.7.0 remains pinned in .bazelversion. Bzlmod is TensorBoard's only
# supported dependency-resolution entry point. Keep WORKSPACE evaluation
# disabled so new dependencies cannot silently bypass MODULE.bazel.
common --enable_bzlmod
common --noenable_workspace

# Keep local and CI builds from reporting ruleset usage through telemetry
# introduced by newer Aspect build rules.
common --repo_env=DO_NOT_TRACK=1
# Let the OS-specific sections below (`common:linux`, `common:macos`, etc.)
# take effect so one `.bazelrc` can carry the required per-platform C++ flags.
common --enable_platform_specific_config
Expand All @@ -18,19 +21,26 @@ test --incompatible_default_to_explicit_init_py
build --define=use_fast_cpp_protos=true

# Use Bazel's hermetic JDK 21 for Java targets and build tools. The default
# target runtime is the host JDK, which can be newer than Bazel 7's Java
# target runtime is the host JDK, which can be newer than Bazel's Java
# compiler and make otherwise unrelated Java compilation fail.
build --java_runtime_version=remotejdk_21
build --tool_java_runtime_version=remotejdk_21

# Protobuf 6.31.1 requires C++17 in this build. Keep the host and target
# rules_nodejs 5.8.1's concatjs rules still return legacy struct providers.
# Remove this compatibility flag with the planned rules_js/ts_project migration.
build --incompatible_disallow_struct_provider_syntax=false

# The current protobuf/gRPC stack requires C++17. Keep the host and target
# language level aligned across supported platforms.
common:linux --cxxopt=-std=c++17
common:linux --host_cxxopt=-std=c++17
common:macos --cxxopt=-std=c++17
common:macos --host_cxxopt=-std=c++17
common:windows --cxxopt=/std:c++17
common:windows --host_cxxopt=/std:c++17
# Protobuf requires Bazel-on-Windows users to acknowledge that upstream may
# remove MSVC support in a future release.
common:windows --define=protobuf_allow_msvc=true

# Local shells and virtualenvs can leak Python import state into Bazel tests,
# which then import from the wrong environment instead of the test runfiles.
Expand Down
2 changes: 1 addition & 1 deletion .bazelversion
Original file line number Diff line number Diff line change
@@ -1 +1 @@
7.7.0
8.7.0
5 changes: 5 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1,11 +1,16 @@
* text=auto
*.bzl text eol=lf
*.py text eol=lf
*.sh text eol=lf
*.toml text eol=lf
.bazel* text eol=lf
BUILD text eol=lf
BUILD.bazel text eol=lf
*.BUILD text eol=lf
Cargo.lock text eol=lf
WORKSPACE text eol=lf
MODULE.bazel text eol=lf
MODULE.bazel.lock text eol=lf
*.proto text eol=lf
tensorboard/data/server/*.pb.rs text eol=lf

Expand Down
37 changes: 14 additions & 23 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@ permissions:

env:
# Keep this Bazel version in sync with .bazelversion and MODULE.bazel's
# bazel_compatibility declaration. Bazel 8 is intentionally unsupported.
BAZEL_VERSION: '7.7.0'
BAZEL_SHA256SUM: 'fe7e799cbc9140f986b063e06800a3d4c790525075c877d00a7112669824acbf'
# bazel_compatibility declaration.
BAZEL_VERSION: '8.7.0'
BAZEL_SHA256SUM: 'd7606e679b78067c811096fb3d6cf135225b528835ca396e3a4dddf957859544'
BUILDTOOLS_VERSION: '3.0.0'
BUILDIFIER_SHA256SUM: 'e92a6793c7134c5431c58fbc34700664f101e5c9b1c1fcd93b97978e8b7f88db'
BUILDOZER_SHA256SUM: '3d58a0b6972e4535718cdd6c12778170ea7382de7c75bc3728f5719437ffb84d'
Expand Down Expand Up @@ -91,6 +91,8 @@ jobs:
sudo mv ~/bazel /usr/local/bin/bazel
sudo chmod +x /usr/local/bin/bazel
cp ./ci/bazelrc ~/.bazelrc
- name: 'Validate Bzlmod lockfile'
run: bazel mod graph --lockfile_mode=error
- name: 'Install TensorFlow'
run: |
python -m pip install -U pip
Expand Down Expand Up @@ -178,11 +180,11 @@ jobs:
matrix:
mode: ['native']
platform: ['ubuntu-22.04']
rust_version: ['1.65.0']
rust_version: ['1.85.0']
include:
- mode: 'universal'
platform: 'ubuntu-22.04'
rust_version: '1.65.0'
rust_version: '1.85.0'
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6.3.0
Expand Down Expand Up @@ -221,10 +223,10 @@ jobs:
setuptools wheel
- name: 'Build'
if: matrix.mode == 'native'
run: cd tensorboard/data/server/ && cargo build --release
run: cd tensorboard/data/server/ && cargo build --locked --release
- name: 'Test'
if: matrix.mode == 'native'
run: cd tensorboard/data/server/ && cargo test --release
run: cd tensorboard/data/server/ && cargo test --locked --release
- name: 'Package (native)'
if: matrix.mode == 'native'
run: |
Expand Down Expand Up @@ -305,8 +307,8 @@ jobs:
runs-on: ubuntu-22.04
strategy:
matrix:
rust_version: ['1.65.0']
cargo_raze_version: ['0.16.1']
# Keep this version in sync with the rules_rust toolchain in MODULE.bazel.
rust_version: ['1.85.0']
steps:
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: 'Cache Cargo artifacts'
Expand All @@ -319,29 +321,18 @@ jobs:
~/.cargo/registry/index/
~/.cargo/registry/cache/
~/.cargo/git/db/
# Needed for installing binaries (`cargo-raze`) with cache
~/.cargo/.crates.toml
~/.cargo/.crates2.json
key: lint-rust-${{ runner.os }}-cargo-${{ matrix.rust_version }}-${{ matrix.cargo_raze_version }}-${{ hashFiles('**/Cargo.lock', '.github/workflows/ci.yml') }}
key: lint-rust-${{ runner.os }}-cargo-${{ matrix.rust_version }}-${{ hashFiles('**/Cargo.lock', '.github/workflows/ci.yml') }}
- name: 'Install Rust toolchain'
uses: dtolnay/rust-toolchain@5f2e2a7aff63d8cbdacb4832218a30fa7a37ee6e # current HEAD as of 1/19/2023
with:
toolchain: ${{ matrix.rust_version }}
components: rustfmt, clippy
- name: 'Install cargo-raze'
run: cargo install cargo-raze --version ${{ matrix.cargo_raze_version }} --locked
- name: 'Run Rustfmt'
run: (cd tensorboard/data/server/ && cargo fmt -- --check)
- name: 'Run Clippy'
# You can run `cargo clippy --all-targets --manifest-path tensorboard/data/server/Cargo.toml --fix` to fix all Clippy complaints.
# This will only apply `MachineApplicable` fixes (https://doc.rust-lang.org/nightly/nightly-rustc/rustc_lint_defs/enum.Applicability.html), so some modifications may need to be done manually.
run: cargo clippy --all-targets --manifest-path tensorboard/data/server/Cargo.toml -- -D warnings
- name: 'Check cargo-raze freshness'
run: |
rm -rf third_party/rust/
(cd tensorboard/data/server/ && cargo fetch && cargo raze)
git add .
git diff --staged --exit-code
run: cargo clippy --locked --all-targets --manifest-path tensorboard/data/server/Cargo.toml -- -D warnings

lint-frontend:
runs-on: ubuntu-22.04
Expand Down Expand Up @@ -390,7 +381,7 @@ jobs:
- name: 'Lint BUILD files'
# TODO(tensorboard-team): address all lint warnings and remove the exemption.
run:
git ls-files -z '*BUILD' third_party/js.bzl third_party/workspace.bzl WORKSPACE | xargs -0 buildifier --mode=check --lint=warn
git ls-files -z '*BUILD' third_party/js.bzl MODULE.bazel | xargs -0 buildifier --mode=check --lint=warn
--warnings=-native-py,-native-java
- run: ./tensorboard/tools/mirror_urls_test.sh
- name: 'Lint for no py2 BUILD targets'
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@
*.egg-info/
*.swp
node_modules
/.bazel-user-root
15 changes: 15 additions & 0 deletions BUILD
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
load("@aspect_rules_ts//ts:defs.bzl", rules_ts_config = "ts_config")
load("@npm//@bazel/typescript:index.bzl", "ts_config")

licenses(["notice"])
Expand All @@ -13,3 +14,17 @@ ts_config(
],
deps = [],
)

# rules_ts uses its own TsConfigInfo provider. Keep these targets separate from
# the legacy @bazel/typescript configuration during the incremental migration.
rules_ts_config(
name = "tsconfig-rules-ts",
src = "tsconfig.json",
visibility = ["//tensorboard:internal"],
)

rules_ts_config(
name = "tsconfig-lax-rules-ts",
src = "tsconfig-lax.json",
visibility = ["//tensorboard:internal"],
)
70 changes: 23 additions & 47 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,44 +24,23 @@ $ source tf/bin/activate
```

TensorBoard builds are done with [Bazel](https://bazel.build). The supported
version is pinned in `.bazelversion` (currently Bazel 7.7.0) so local and CI
version is pinned in `.bazelversion` (currently Bazel 8.7.0) so local and CI
builds use the same incompatible-flag behavior and the same validated Bzlmod
dependency graph. Bazelisk reads this file and selects that version
automatically. Bazel 8 is intentionally unsupported. TensorBoard uses Bzlmod
for dependency resolution; legacy builds with `--noenable_bzlmod` are not
supported. Some dependencies still use legacy repository macros through the
transitional `WORKSPACE.bzlmod` file. Normal development must not add a second
dependency graph to `WORKSPACE`; new dependencies should use a BCR module, a
module extension, or `use_repo_rule` in that order.

Until those transitional repositories are migrated, Bazel must still evaluate
`WORKSPACE.bzlmod`; therefore, builds must not pass `--noenable_workspace`.
This compatibility requirement does not make legacy `--noenable_bzlmod` builds
supported.

The remaining `WORKSPACE.bzlmod` entries are intentional migration exceptions:

- The pinned Closure snapshot requires TensorBoard's Protobuf 6/Soy patches.
The module-published Closure release is currently validated upstream with
Bazel 8 rather than TensorBoard's supported Bazel 7.7.0.
- rules_nodejs 5.8.1 supplies the legacy `yarn_install`, concatjs, TypeScript,
and esbuild APIs used by the frontend build. The module-native rules_nodejs
dependency used by `aspect_rules_js` does not provide those removed APIs.
- rules_sass has no BCR module, and replacing it means moving Sass compilation
into the modern JavaScript rule stack.
- The Rust data server uses an old rules_rust toolchain and a checked-in
cargo-raze graph. Moving it requires a coordinated Rust toolchain and
crate-universe migration.
- TensorBoard's font/JavaScript archives, Closure Java artifacts, and local
compatibility repositories still depend on the exceptions above.

`WORKSPACE.bzlmod` should shrink as those projects are completed. A completely
WORKSPACE-independent build is reached when the following command succeeds;
it is a migration diagnostic that is expected to fail today, not a supported
presubmit command:
automatically. TensorBoard uses Bzlmod exclusively for dependency resolution;
legacy builds with `--noenable_bzlmod` are not supported and WORKSPACE
evaluation is explicitly disabled in `.bazelrc`. New dependencies should use a
BCR module, a module extension, or `use_repo_rule` in that order.

The frontend still consumes the legacy rules_nodejs 5.8.1 concatjs APIs, but
its Node, Yarn, npm, and esbuild repositories are instantiated by a module
extension. This keeps the build WORKSPACE-independent while the TypeScript
targets are incrementally migrated to rules_js/rules_ts.

The supported strict build can be exercised directly with:

```sh
(tf)$ bazel test //tensorboard/... --enable_bzlmod --noenable_workspace
(tf)$ bazel test //tensorboard/... --noenable_workspace
```

The Bazel build and test configuration is currently validated on Linux only.
Expand Down Expand Up @@ -345,29 +324,27 @@ Sample upgrade: https://github.com/tensorflow/tensorboard/pull/5977
there are occasionally special instructions, especially for major releases.
Make a mental note of any of these special instructions.

2. Update the `build_bazel_rules_nodejs` target in `WORKSPACE.bzlmod` as
described in the rules_nodejs release notes. This dependency still uses a
legacy repository macro behind the Bzlmod entry point. Also remove or
comment out now-stale links to "http://mirror.tensorflow.org/".
2. Update the `build_bazel_rules_nodejs` `http_archive` in `MODULE.bazel` as
described in the rules_nodejs release notes. Review
`patches/rules_nodejs_5_8_1_bzlmod.patch` and
`third_party/nodejs_extensions.bzl` for compatibility changes that can be
removed or must be updated with the archive.

3. Update npm packages scoped with `@bazel` in package.json using yarn.
* Use the same version as the rules_nodejs version.
* See the previous section for instructions on how to use yarn.

4. Update the `rules_sass` target in `WORKSPACE.bzlmod`.
* Examine https://github.com/bazelbuild/rules_sass/tags to see the list
of rules_sass releases.
* Pick a tag (the most recent is likely good enough) and use that version to
modify the `rules_sass` target in `WORKSPACE.bzlmod`.
4. Update the `gzgz_rules_sass` dependency and Sass toolchain in
`MODULE.bazel`. Review TensorBoard's Sass adapter and patches whenever its
Starlark or npm integration changes.

5. Update the minimum bazel version to match the one supported by rules_nodejs:
* Examine https://github.com/bazelbuild/rules_nodejs/blob/stable/index.bzl
and find the SUPPORTED_BAZEL_VERSIONS constant.
* Compare the supported Bazel version from rules_nodejs with the version
pinned in `.bazelversion` and constrained by `MODULE.bazel`. Update those
files and `.github/workflows/ci.yml` together if the project deliberately
changes Bazel versions. Do not move to Bazel 8 without a separate
compatibility migration.
changes Bazel versions.

6. Attempt to rebuild and test TensorBoard to make sure it works:
* `rm -rf node_modules; bazel clean --expunge; yarn`
Expand All @@ -379,8 +356,7 @@ Sample upgrade: https://github.com/tensorflow/tensorboard/pull/5977
the rules_nodejs release notes (from Step 1) might be helpful.

8. Generate mirrors for the new versions of rules_nodejs and rules_sass and
update `WORKSPACE.bzlmod` with the new "http://mirror.tensorflow.org/"
URLs.
update their `MODULE.bazel` source URLs.
Googlers, see information at go/tensorboard-tf-mirror.

## Updating Angular
Expand Down
4 changes: 2 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ RUN apt update && apt install -y wget unzip python3 python3-pip python3-dev pyth
RUN python3 -m pip install -U pip

# Setup build environment
ENV BAZEL_VERSION='4.2.2'
ENV BAZEL_SHA256SUM='11dea6c7cfd866ed520af19a6bb1d952f3e9f4ee60ffe84e63c0825d95cb5859'
ENV BAZEL_VERSION='8.7.0'
ENV BAZEL_SHA256SUM='d7606e679b78067c811096fb3d6cf135225b528835ca396e3a4dddf957859544'
ENV BUILDTOOLS_VERSION='3.0.0'
ENV BUILDIFIER_SHA256SUM='e92a6793c7134c5431c58fbc34700664f101e5c9b1c1fcd93b97978e8b7f88db'
ENV BUILDOZER_SHA256SUM='3d58a0b6972e4535718cdd6c12778170ea7382de7c75bc3728f5719437ffb84d'
Expand Down
Loading
Loading