From f5a4e74016c8cdca47ce944ae3d1ca0e80384867 Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Thu, 23 Apr 2026 18:38:19 -0400 Subject: [PATCH 1/4] chore: Add a root AGENTS.md context file and consolidate shared info --- .agents/skills/java-development/SKILL.md | 75 ++++---------- sdk-platform-java/GEMINI.md | 120 ++--------------------- 2 files changed, 28 insertions(+), 167 deletions(-) diff --git a/.agents/skills/java-development/SKILL.md b/.agents/skills/java-development/SKILL.md index 0b929dec6cd0..06d07c77ee14 100644 --- a/.agents/skills/java-development/SKILL.md +++ b/.agents/skills/java-development/SKILL.md @@ -1,67 +1,30 @@ --- name: java-development -description: General guidance on Java development practices, building, testing, and style in the monorepo. Use this skill when working on Java code across the repository. +description: Workflow guidance for performing Java development tasks in this repository. --- -# Java Development Guide +# Java Development Workflow -This skill provides general guidelines for Java development inside the monorepo. It covers building, formatting, testing, and style conventions to ensure consistency across modules. +Use this skill when tasked with modifying Java code, fixing bugs, or adding features. It guides you through the typical development lifecycle in this repo. -## Workflow +## Workflow Steps -### 1. Building the Project +### Step 1: Understand the Scope +- Determine if you are working on **Core Components** (`sdk-platform-java`) or a specific **Client Library** (`java-`). +- Review the root [AGENTS.md](../../AGENTS.md) for the Style Guide and Dependency Management rules that apply repo-wide. -The repository uses Maven as its primary build system. +### Step 2: Make Changes and Build +- Apply your code changes. +- To build efficiently, use the recommended flags or scoped builds detailed in [AGENTS.md](../../AGENTS.md#building-the-project). -* **Build All Modules**: To build all modules from the root of the repository, run: - ```bash - mvn install -T 1C -P quick-build - ``` - > [!TIP] - > Use `-T 1C` to build modules in parallel (one thread per CPU core) and `-P quick-build` to skip unnecessary plugins for faster builds. -* **Build a Specific Module**: You can also run Maven commands within a specific module directory (e.g., `java-bigquery`) to build only that module. +### Step 3: Determine and Run Tests +- If working on a **Client Library**, run its unit tests (`mvn test` in the module directory) and check its README for integration test procedures. +- If working on **Core Components** (`sdk-platform-java`), you **must** follow the testing heuristics in [sdk-platform-java/DEVELOPMENT.md](../../../sdk-platform-java/DEVELOPMENT.md#testing-guide) to know which tests are required. +- If golden files need updating, follow the specific procedures referenced in [AGENTS.md](../../AGENTS.md#specialized-development-guides). -### 2. Code Formatting +### Step 4: Format Code +- You **must** ensure the code is formatted before completing a task. Use the formatting commands listed in [AGENTS.md](../../AGENTS.md#code-formatting). -Code formatting is enforced using the `fmt-maven-plugin`. - -* **Check Formatting**: To check for formatting issues without modifying files, run: - ```bash - mvn fmt:check -T 1C - ``` -* **Apply Formatting**: To automatically format the code according to the project style, run: - ```bash - mvn fmt:format -T 1C - ``` - > [!TIP] - > To save time, run `mvn fmt:format` within the specific module directory you are working on, rather than at the root. - > [!NOTE] - > Always run `mvn fmt:format` before committing changes to avoid build failures due to formatting. - -### 3. Testing Strategy - -* **Unit Tests**: Traditional unit tests should be added for individual classes and methods. Run them using: - ```bash - mvn test -T 1C - ``` -* **Integration Tests**: Many modules have integration tests that run against live services or emulators. These may require specific profiles or environment variables. Refer to the specific module's README for details. - -### 4. Style Guide - -Follow these general rules to maintain code quality and consistency: - -1. **Minimize Visibility**: Default to the most restrictive access level possible. Avoid using `public` unless the class or method is intended to be part of the public API. -2. Avoid Fully Qualified Names: Use imports to keep class names short and readable, rather than using fully qualified names in the code. -3. **Avoid Obsolete APIs**: Do not call methods marked with `@ObsoleteApi` or `@Deprecated` unless there are no viable alternatives. -4. **Clean Diffs**: Avoid unnecessary formatting changes or whitespace modifications to keep diffs clean and easy to review. - -### 5. Dependency Management - -* **Version Bumps**: Try not to bump any external dependency versions unless there is a known security vulnerability (CVE) or a critical bug fix. -* **New Dependencies**: Avoid introducing new external dependencies. If a new dependency is required, provide a strong justification in the pull request. -* **Standard Library First**: Prefer to use features from the Java standard library, followed by existing dependencies in the project (preferably Google-managed dependencies). - -### 6. Contribution Guidelines - -* **Commit Messages**: Follow the [Conventional Commits](https://www.conventionalcommits.org/) specification. Include the module as the scope (e.g., `feat(spanner): ...`, `fix(bigquery): ...`). -* **Pull Requests**: All code changes must be submitted via a pull request and require review. Ensure you pull the latest changes from `main` and resolve any conflicts before submitting. +### Step 5: Verify and Document +- Verify that all tests pass and formatting is correct. +- Follow the Contribution Guidelines in [AGENTS.md](../../AGENTS.md#7-contribution-guidelines) for creating commits and PRs. diff --git a/sdk-platform-java/GEMINI.md b/sdk-platform-java/GEMINI.md index aed84e060b35..d7a8c36425a4 100644 --- a/sdk-platform-java/GEMINI.md +++ b/sdk-platform-java/GEMINI.md @@ -2,7 +2,9 @@ ## 1. Overview -This directory, sdk-platform-java, contains the foundational components for building Java client libraries for Google Cloud Platform services. It includes the GAPIC (Generated API Client) generator for Java, the GAX (Google API Extensions) runtime library, and other shared modules. +This directory, `sdk-platform-java`, contains the foundational components for building Java client libraries for Google Cloud Platform services. It includes the GAPIC (Generated API Client) generator for Java, the GAX (Google API Extensions) runtime library, and other shared modules. + +For general project rules, style guides, dependency management, and contribution guidelines, please refer to the root [AGENTS.md](../AGENTS.md). ## 2. Project Structure @@ -28,58 +30,11 @@ The repository is structured into several key modules: * **`java-shared-dependencies`**: Manages shared Maven dependencies for all Google Cloud Java client libraries. This ensures consistency and helps avoid dependency conflicts. -## 3. Getting Started - -### 3.1. Prerequisites - -To build and work with this project, you will need to install: - -* Java 11+ -* Maven - -### 3.2. Building the Project - -The project uses Maven for its primary build system. To build all modules, run the following command from the root of the repository: - -```sh -mvn install -``` - -### 3.3. Code Formatting - -Code formatting is enforced using the `fmt-maven-plugin`. To check for formatting issues, run: - -```sh -mvn fmt:check -``` - -To format the code, run: - -```sh -mvn fmt:format -``` - -## 4. Testing - -### 4.1. Testing Strategy - -The repository employs a multi-layered testing strategy to ensure the quality and correctness of the generated code: - -* **Unit Tests:** Traditional unit tests for individual classes and methods. See [unit testing best practices](https://engdoc.corp.google.com/eng/doc/devguide/testing/unit/best-practices.md?cl=head&polyglot=java). -* **Golden Unit Tests:** These tests generate code from test protos and compare the output to "golden" files, which are pre-approved versions of the generated code. These test cases exist inside the `gapic-generator-java` module. See [GrpcServiceStubClassComposerTest](https://github.com/googleapis/google-cloud-java/blob/main/sdk-platform-java/gapic-generator-java/src/test/java/com/google/api/generator/gapic/composer/grpc/GrpcServiceStubClassComposerTest.java) for an example of golden unit tests. The steps for golden unit tests include creating a test proto, loading the test proto, generating from the test proto, and comparing the result with a golden file. -* **Showcase Integration Tests:** These tests run the generated Showcase client against a local Showcase server to verify end-to-end functionality. This is the preferred way of testing integration tests. -* **Golden Integration Tests:** These tests generate full client libraries for real Google Cloud APIs and compare them to golden versions. This is an older test strategy and showcase testing is preferred. +## 3. Testing -Based on where the code changes are, we should add different tests, in general +For detailed testing strategy and heuristics on which tests to run based on code changes, please refer to [DEVELOPMENT.md](DEVELOPMENT.md). -* If the changes are in `gax` or `api-common` only, you _must_ add traditional unit tests, you _may_ add showcase integration tests if you are modifying a public setting. -* If the changes are in `gapic-generator-java` only, showcase integration tests are most likely not needed - * If they are in the `composer` module, you _must_ add golden unit tests, you _may_ add traditional unit tests for better coverage and easier testing. - * If they are in the `parser` module, you _should_ add traditional unit tests with a test proto if possible, you _may_ add golden unit tests to easily see the changes. For example, [routing_rule_parser_testing](https://github.com/googleapis/google-cloud-java/blob/main/sdk-platform-java/gapic-generator-java/src/test/proto/routing_rule_parser_testing.proto) that is only used for testing [RoutingRuleParser](https://github.com/googleapis/google-cloud-java/blob/main/sdk-platform-java/gapic-generator-java/src/main/java/com/google/api/generator/gapic/protoparser/RoutingRuleParser.java). - * If they are in `other modules(ast, model, writer etc.)`, you _must_ add traditional unit tests, you _may_ add golden unit tests to easily see the changes. -- If the changes are in both `gax` and `gapic-generator-java`, you _must_ add all test layers, including traditional unit tests, golden unit tests and showcase integration tests. - -### 4.2. Running Unit Tests +### 3.1. Running Unit Tests Unit tests can be tested via this command: @@ -87,7 +42,7 @@ Unit tests can be tested via this command: mvn test ``` -### 4.3. Running Golden Integration Tests +### 3.2. Running Golden Integration Tests Golden integration tests are run using Bazel. To run all golden integration tests, use the following command from the root of the repository: @@ -95,63 +50,6 @@ Golden integration tests are run using Bazel. To run all golden integration test bazelisk test //... ``` -Specific integration tests can be run by specifying the target, for example: - -```sh -bazelisk test //test/integration:redis -``` - -### 4.4. Updating Golden Files - -If you make changes that affect the generated code, you will need to update the golden files. This can be done using the following command: - -```sh -bazelisk run //test/integration:update_asset && bazelisk run //test/integration:update_credentials && bazelisk run //test/integration:update_iam && bazelisk run //test/integration:update_kms && bazelisk run //test/integration:update_pubsub && bazelisk run //test/integration:update_logging && bazelisk run //test/integration:update_redis && bazelisk run //test/integration:update_storage && bazelisk run //test/integration:update_library && bazelisk run //test/integration:update_compute && bazelisk run //test/integration:update_bigtable && bazelisk run //test/integration:update_apigeeconnect -``` - -### 4.5. Running Showcase Integration Tests - -Showcase integration tests are run against a local server that implements the Showcase API. The `java-showcase/README.md` file provides detailed instructions on how to run these tests. The general steps are: - -1. **Install the Showcase server:** - - ```sh - go install github.com/googleapis/gapic-showcase/cmd/gapic-showcase@latest - ``` - -2. **Run the Showcase server:** - - ```sh - gapic-showcase run - ``` - -3. **Run the integration tests:** - - ```sh - cd java-showcase - mvn verify -P enable-integration-tests - ``` - -## 5. Style Guide - -1. Minimize visibility scopes by defaulting to the most restrictive access level, avoiding the `public` modifier unless required. -2. Use short names over fully qualified names. -3. Avoid calling `@ObsoleteApi` or `@Deprecated` methods unless there are no alternatives. -4. Avoid unnecessary formatting changes to keep diffs clean. -5. Use `mvn` for everything other than the `test/integration` folder. - -## 6. Dependency Management - -- Try not to bump any external dependency version unless there is a known CVE (security or vulnerability issue) or a critical bug fix. -- Try to avoid introducing new external dependencies. If a new dependency is required, please state the reason. -- Prefer to use features from the Java standard library, then try to use features from any existing dependencies (preferably from Google managed dependencies). - -## 7. Contribution Guidelines +### 3.3. Running Showcase Integration Tests -- **Commits:** Commit messages should follow the [Conventional Commits](https://www.conventionalcommits.org/) - specification. The format is `: `. The type should be one of the following: fix, feat, - build, chore, docs, test, or refactor. -- **Issues:** All significant changes should start with a GitHub issue. -- **Pull Requests:** All code changes must be submitted via a pull request and require review. Before creating a PR, always pull latest from main, merge main to local branch and resolve any conflicts. -- **Testing:** All new logic should be accompanied by tests. -- For more details, see `CONTRIBUTING.md`. \ No newline at end of file +Showcase integration tests are run against a local server that implements the Showcase API. The `java-showcase/README.md` file provides detailed instructions on how to run these tests. \ No newline at end of file From c6046d21dffd9fe1ec7acefb695a62f77f18f592 Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Thu, 23 Apr 2026 21:03:43 -0400 Subject: [PATCH 2/4] chore: Add missing AGENTS.md file for context --- AGENTS.md | 76 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 76 insertions(+) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 000000000000..c6b424d35aaf --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,76 @@ +# Agents Guide for google-cloud-java + +## 1. Overview +This repository, `google-cloud-java`, contains the Java client libraries for Google Cloud Platform services, as well as core components in `sdk-platform-java`. + +## 2. Project Structure +The repository is a monorepo containing both generated and handwritten libraries, as well as core platform components and parent POMs. + +### Core Modules +- **`sdk-platform-java/`**: Contains foundational components for building client libraries. + - **Note**: This directory has its own `GEMINI.md` file with detailed instructions specific to core development (GAPIC generator, GAX). + - Includes `gapic-generator-java` (the generator) and `gax-java` (Google API Extensions). +- **`google-auth-library-java/`**: The Google Auth Library for Java. This is a **handwritten** library used for authentication and credential management across all Google Cloud clients. It is a critical dependency for all client libraries. + +### Parent Modules and BOMs +- **`google-cloud-pom-parent/`**: The top-level parent POM for all modules in the repository. It manages plugin versions and common configuration. +- **`google-cloud-jar-parent/`**: The parent POM for all client library JAR modules in the repository. It inherits from `google-cloud-pom-parent` and manages shared dependencies. +- **`gapic-libraries-bom/`**: The Bill of Materials (BOM) that manages versions of all client libraries to ensure compatibility when used together. +- **`java-shared-dependencies/`** (inside `sdk-platform-java`): Manages shared Maven dependencies for all Google Cloud Java client libraries to ensure consistency and avoid conflicts. + +### Client Libraries (`java-/`) +Directories starting with `java-` are client libraries for specific Google Cloud services. +- **Generated Clients**: The majority of these are automatically generated from service definitions (protos) using the GAPIC generator in `sdk-platform-java`. +- **Handwritten & Split Repositories**: Some major libraries are either entirely handwritten or are maintained as "split repos" (they have their own standalone repositories in the `googleapis` GitHub organization but are also managed here). When working on these, be aware that changes may need to be synchronized with their respective split repos. Key examples include: + - **BigQuery**: [java-bigquery](java-bigquery) + - **BigQuery Storage**: [java-bigquerystorage](java-bigquerystorage) + - **Spanner**: [java-spanner](java-spanner) + - **Spanner JDBC**: [java-spanner-jdbc](java-spanner-jdbc) + - **Storage**: [java-storage](java-storage) + - **Storage NIO**: [java-storage-nio](java-storage-nio) + - **Datastore**: [java-datastore](java-datastore) + - **Logging**: [java-logging](java-logging) + +## 3. Getting Started +### Prerequisites +- Java 11+ +- Maven +- Bazelisk (for integration tests in `sdk-platform-java`) + +### Building the Project +- Build all modules: `mvn install` from root. +- Recommended for faster builds: `mvn install -T 1C -P quick-build`. +- Build specific module: Run `mvn` commands within the module directory. + +### Code Formatting +- Check: `mvn fmt:check` +- Format: `mvn fmt:format` +- Tip: Run in specific module to save time. + +## 4. Testing Strategy +- **Unit Tests**: `mvn test`. +- **Integration Tests**: Module specific, may require emulators or live services. +- **Core Components Testing**: See [sdk-platform-java/DEVELOPMENT.md](sdk-platform-java/DEVELOPMENT.md) for a detailed explanation of the 4 layers of testing (Traditional Unit, Golden Unit, Showcase, Golden Integration). + +## 5. Style Guide +1. Minimize visibility scopes. Default to most restrictive access level. +2. Use short names over fully qualified names. +3. Avoid calling `@ObsoleteApi` or `@Deprecated` methods. +4. Avoid unnecessary formatting changes to keep diffs clean. +5. Use `mvn` for everything other than the `test/integration` folder. + +## 6. Dependency Management +- Do not bump external dependency versions unless for CVE or critical bug fix. +- Avoid introducing new external dependencies. +- Prefer Java standard library, then existing dependencies. + +## 7. Contribution Guidelines +- **Commits**: Conventional Commits `(): `. +- **Pull Requests**: Submitted via PR, require review. Pull latest from main and resolve conflicts. +- **Testing**: All new logic should be accompanied by tests. + +## 8. Specialized Development Guides +For development on core components, refer to the following guides in `sdk-platform-java`: +- **GAPIC Generator**: [sdk-platform-java/gapic-generator-java/DEVELOPMENT.md](sdk-platform-java/gapic-generator-java/DEVELOPMENT.md) +- **GAX**: [sdk-platform-java/gax-java/DEVELOPMENT.md](sdk-platform-java/gax-java/DEVELOPMENT.md) +- **Hermetic Build**: [sdk-platform-java/hermetic_build/DEVELOPMENT.md](sdk-platform-java/hermetic_build/DEVELOPMENT.md) From 5a634d9f6e018983ae3693957363f97b37cf7b3d Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Thu, 23 Apr 2026 21:28:58 -0400 Subject: [PATCH 3/4] chore: fix skill relative paths and refine prerequisites in AGENTS.md --- .agents/skills/java-development/SKILL.md | 10 +++++----- AGENTS.md | 12 ++++++------ 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/.agents/skills/java-development/SKILL.md b/.agents/skills/java-development/SKILL.md index 06d07c77ee14..1b5ed8ce6566 100644 --- a/.agents/skills/java-development/SKILL.md +++ b/.agents/skills/java-development/SKILL.md @@ -11,20 +11,20 @@ Use this skill when tasked with modifying Java code, fixing bugs, or adding feat ### Step 1: Understand the Scope - Determine if you are working on **Core Components** (`sdk-platform-java`) or a specific **Client Library** (`java-`). -- Review the root [AGENTS.md](../../AGENTS.md) for the Style Guide and Dependency Management rules that apply repo-wide. +- Review the root [AGENTS.md](../../../AGENTS.md) for the Style Guide and Dependency Management rules that apply repo-wide. ### Step 2: Make Changes and Build - Apply your code changes. -- To build efficiently, use the recommended flags or scoped builds detailed in [AGENTS.md](../../AGENTS.md#building-the-project). +- To build efficiently, use the recommended flags or scoped builds detailed in [AGENTS.md](../../../AGENTS.md#building-the-project). ### Step 3: Determine and Run Tests - If working on a **Client Library**, run its unit tests (`mvn test` in the module directory) and check its README for integration test procedures. - If working on **Core Components** (`sdk-platform-java`), you **must** follow the testing heuristics in [sdk-platform-java/DEVELOPMENT.md](../../../sdk-platform-java/DEVELOPMENT.md#testing-guide) to know which tests are required. -- If golden files need updating, follow the specific procedures referenced in [AGENTS.md](../../AGENTS.md#specialized-development-guides). +- If golden files need updating, follow the specific procedures referenced in [AGENTS.md](../../../AGENTS.md#8-specialized-development-guides). ### Step 4: Format Code -- You **must** ensure the code is formatted before completing a task. Use the formatting commands listed in [AGENTS.md](../../AGENTS.md#code-formatting). +- You **must** ensure the code is formatted before completing a task. Use the formatting commands listed in [AGENTS.md](../../../AGENTS.md#code-formatting). ### Step 5: Verify and Document - Verify that all tests pass and formatting is correct. -- Follow the Contribution Guidelines in [AGENTS.md](../../AGENTS.md#7-contribution-guidelines) for creating commits and PRs. +- Follow the Contribution Guidelines in [AGENTS.md](../../../AGENTS.md#7-contribution-guidelines) for creating commits and PRs. diff --git a/AGENTS.md b/AGENTS.md index c6b424d35aaf..f74ba554a856 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -30,11 +30,12 @@ Directories starting with `java-` are client libraries for specific Google Cloud - **Storage NIO**: [java-storage-nio](java-storage-nio) - **Datastore**: [java-datastore](java-datastore) - **Logging**: [java-logging](java-logging) + - **Logging Logback**: [java-logging-logback](java-logging-logback) ## 3. Getting Started ### Prerequisites -- Java 11+ -- Maven +- Java 11+ (build targets Java 8 bytecode compatibility) +- Maven 3.0+ - Bazelisk (for integration tests in `sdk-platform-java`) ### Building the Project @@ -50,19 +51,18 @@ Directories starting with `java-` are client libraries for specific Google Cloud ## 4. Testing Strategy - **Unit Tests**: `mvn test`. - **Integration Tests**: Module specific, may require emulators or live services. -- **Core Components Testing**: See [sdk-platform-java/DEVELOPMENT.md](sdk-platform-java/DEVELOPMENT.md) for a detailed explanation of the 4 layers of testing (Traditional Unit, Golden Unit, Showcase, Golden Integration). +- **Core Components Testing**: Components should have adequate unit and integration tests to ensure coverage and correctness. ## 5. Style Guide 1. Minimize visibility scopes. Default to most restrictive access level. 2. Use short names over fully qualified names. -3. Avoid calling `@ObsoleteApi` or `@Deprecated` methods. +3. Avoid calling `@ObsoleteApi` or `@Deprecated` methods and classes. 4. Avoid unnecessary formatting changes to keep diffs clean. 5. Use `mvn` for everything other than the `test/integration` folder. ## 6. Dependency Management - Do not bump external dependency versions unless for CVE or critical bug fix. -- Avoid introducing new external dependencies. -- Prefer Java standard library, then existing dependencies. +- Avoid introducing new external dependencies if possible. Prefer Java standard library, then opt for existing dependencies. ## 7. Contribution Guidelines - **Commits**: Conventional Commits `(): `. From 12cfa77aaabead7f7ca296f3196f082a9dec60fd Mon Sep 17 00:00:00 2001 From: Lawrence Qiu Date: Fri, 24 Apr 2026 10:55:09 -0400 Subject: [PATCH 4/4] chore: Update sdk-platform-java/GEMINI.md Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- sdk-platform-java/GEMINI.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sdk-platform-java/GEMINI.md b/sdk-platform-java/GEMINI.md index d7a8c36425a4..4a024b1d752b 100644 --- a/sdk-platform-java/GEMINI.md +++ b/sdk-platform-java/GEMINI.md @@ -52,4 +52,4 @@ bazelisk test //... ### 3.3. Running Showcase Integration Tests -Showcase integration tests are run against a local server that implements the Showcase API. The `java-showcase/README.md` file provides detailed instructions on how to run these tests. \ No newline at end of file +Showcase integration tests are run against a local server that implements the Showcase API. The [java-showcase/README.md](java-showcase/README.md) file provides detailed instructions on how to run these tests. \ No newline at end of file