diff --git a/Documentation/README.md b/Documentation/README.md index 07cb638..f6bdd5a 100644 --- a/Documentation/README.md +++ b/Documentation/README.md @@ -280,6 +280,8 @@ Closed: ML_Out (.\ML_Out.1.sds). The ML model delivers four float values for each category `PAPER`, `ROCK`, `SCISSORS`, and `UNKNOWN`. The inference interval is about 8 Hz. This information is recorded in the `ML_Out..sds` files in binary format. The file `RockPaperScissors/SDS_Metadata/ML_Out.sds.yml` describes the content and using this file with SDS utilities allows you to view the content. +A utility script to auto-generate this `ML_Out.sds.yml` from a `model_config.h` header is available [here](../RockPaperScissors/AppKit-E8_USB/scripts/README.md). + **Example output:** ```txt diff --git a/Documentation/creating-an-example-using-rps-template.md b/Documentation/creating-an-example-using-rps-template.md new file mode 100644 index 0000000..00b5f74 --- /dev/null +++ b/Documentation/creating-an-example-using-rps-template.md @@ -0,0 +1,133 @@ +### Creating an New Use Example from the RockPaperScissors Template + +This guide explains how to create a new image classification use case by reusing the existing `RockPaperScissors` application as the template project. + +The `RockPaperScissors` directory contains the reusable embedded application structure. When a user selects another use case in ModelNova Fusion Studio, such as `VehicleClassification` or `ToolClassification`, Fusion Studio uses the existing `RockPaperScissors` project directory and replaces the model-specific artifacts for the selected use case. + +#### Repository Structure + +Open the ModelNova repository root. The reusable template project is located at: + +```text + ++-- RockPaperScissors + +-- AppKit-E8_USB + | +-- algorithm + | | +-- ML + | | +-- ai_layer + | | +-- cil_layer + | | | +-- include + | | | +-- src + | | +-- model_config.h + | +-- Board + | +-- sdsio + | +-- SDS.csolution.yml + +-- RPS_cls_dataset +``` + +Fusion Studio updates the model-related content inside this existing project when replacing the use case artifacts. + +#### Supported Model Categories + +Fusion Studio supports generating model artifacts for image classification models based on the supported model architectures. + +Choose the architecture that best meets your application requirements, considering factors such as accuracy, latency, memory footprint, and target hardware. + +#### Prerequisites + +Before updating the use case, make sure you have: + +- ModelNova Fusion Studio installed and configured. +- A dataset prepared for the selected use case. +- Keil Studio for VS Code. +- A working build of the existing `RockPaperScissors` example. +- Access to the target board and flashing/debugging setup. + +#### Step 1: Use the Existing RockPaperScissors Project + +Use the existing `RockPaperScissors` directory as the template project for the selected use case. + +The project path remains: + +```text +RockPaperScissors\AppKit-E8_USB +``` + +Fusion Studio replaces the model artifacts in this existing directory for the use case selected by the user. For example, a user may select a vehicle classification, tool classification, food classification, or defect classification use case, but the embedded project directory remains `RockPaperScissors`. + +#### Step 2: Generate Model Artifacts Using Fusion Studio + +Follow the [Create ML model](./README.md#create-ml-model) section above to train, validate, and deploy your model in Fusion Studio. When generating artifacts, select the existing `RockPaperScissors` project as the target for artifact replacement. + +Export the generated `.cilpkg` package if manual replacement is needed. + +For example, a vehicle classification dataset might contain classes such as: + +```text +car +bus +truck +motorcycle +``` + +> [!NOTE] +> Fusion Studio replaces the model artifacts automatically. Verify that the required files are present in the existing `RockPaperScissors` project. + +#### Step 3 (Optional): Replace the Model Artifacts Manually + +If you choose to update the artifacts manually, unzip or extract the `.cilpkg` package generated by Fusion Studio. + +The extracted package should contain the required Common Inference Library artifacts: + +```text +ai_layer +cil.a +cil.h +model_config.h +``` +> **For details about the `.cilpkg` package, refer to the [CIL Package Documentation](../RockPaperScissors/AppKit-E8_USB/algorithm/ML/README.md)**. + +These files replace the template model artifacts in the existing `RockPaperScissors` project. + +Copy the extracted artifacts into the following locations: + +| Artifact | Destination | +| --- | --- | +| `ai_layer` | `RockPaperScissors\AppKit-E8_USB\algorithm\ML\ai_layer` | +| `model_config.h` | `RockPaperScissors\AppKit-E8_USB\algorithm\ML\model_config.h` | +| `cil.a` | `RockPaperScissors\AppKit-E8_USB\algorithm\ML\cil_layer\src\cil.a` | +| `cil.h` | `RockPaperScissors\AppKit-E8_USB\algorithm\ML\cil_layer\include\cil.h` | + +Expected paths: + +```text +RockPaperScissors\AppKit-E8_USB\algorithm\ML\ai_layer +RockPaperScissors\AppKit-E8_USB\algorithm\ML\model_config.h +RockPaperScissors\AppKit-E8_USB\algorithm\ML\cil_layer\src\cil.a +RockPaperScissors\AppKit-E8_USB\algorithm\ML\cil_layer\include\cil.h +``` + +#### Step 4: Review Application Configuration + +After replacing the model artifacts, review the application configuration for any use case specific changes. + +Check the following areas: + +- Class labels and output post-processing. +- Input image size and preprocessing requirements. +- Number of output classes. +- Dataset-specific test files or SDS recordings. +- Any user-facing log messages that still mention Rock/Paper/Scissors. + +The exact files to update depend on the model output and application logic. Start by reviewing files under: + +```text +RockPaperScissors\AppKit-E8_USB\algorithm +RockPaperScissors\AppKit-E8_USB\algorithm\ML +``` + +#### Notes + +- Fusion Studio uses the existing `RockPaperScissors` project directory for replacing artifacts for the selected use case. +- Do not create a separate directory for each selected use case. +- Regenerate and replace all four model artifacts together when changing the model. diff --git a/README.md b/README.md index df4a7eb..d30bcb9 100644 --- a/README.md +++ b/README.md @@ -74,6 +74,8 @@ The diagram below illustrates the RPS application architecture. During algorithm While Fusion Studio supports multiple workspace creation methods (Scratch, Starter Pack, AI Assist), the **ML workflow remains identical for all workspaces**. +> The `RockPaperScissors` project also serves as a [template for creating new examples](./Documentation/README.md#creating-a-new-use-case-from-the-rps-template). + ## Analyze Timing with SystemView [SEGGER SystemView](https://www.segger.com/products/development-tools/systemview/) can be used for timing analysis. The application includes annotations that let you measure the timing of different compute blocks, as shown below. diff --git a/RockPaperScissors/AppKit-E8_USB/algorithm/ML/image_processing_func.c b/RockPaperScissors/AppKit-E8_USB/algorithm/ML/image_processing_func.c index 7b262b9..b393cd3 100644 --- a/RockPaperScissors/AppKit-E8_USB/algorithm/ML/image_processing_func.c +++ b/RockPaperScissors/AppKit-E8_USB/algorithm/ML/image_processing_func.c @@ -21,6 +21,11 @@ #include "cmsis_compiler.h" #include "image_processing_func.h" +#ifdef USE_SEGGER_SYSVIEW +#include "SEGGER_SYSVIEW.h" +#include "sysview_markers.h" +#endif + /* Clamp a value to a specified range. @@ -323,6 +328,9 @@ void crop_resize_rgb565_to_rgb888( int dst_width, int dst_height) { +#ifdef USE_SEGGER_SYSVIEW + SEGGER_SYSVIEW_MarkStart(SYSVIEW_MARKER_RESIZE_IMAGE); +#endif int crop_size = src_height; // 720 for 1280x720 int crop_x = (src_width - crop_size) / 2; // center horizontally int crop_y = 0; @@ -339,6 +347,13 @@ void crop_resize_rgb565_to_rgb888( int sx = (src_x_fp >> FP_SHIFT) + crop_x; int idx = (sy * src_width + sx) * 2; +#ifdef USE_SEGGER_SYSVIEW + SEGGER_SYSVIEW_MarkStop(SYSVIEW_MARKER_RESIZE_IMAGE); +#endif + +#ifdef USE_SEGGER_SYSVIEW + SEGGER_SYSVIEW_MarkStart(SYSVIEW_MARKER_CONVERT_IMAGE); +#endif uint16_t pixel = src[idx] | (src[idx + 1] << 8); uint8_t r5 = (pixel >> 11) & 0x1F; @@ -350,6 +365,9 @@ void crop_resize_rgb565_to_rgb888( dst_pixel[0] = (r5 << 3) | (r5 >> 2); dst_pixel[1] = (g6 << 2) | (g6 >> 4); dst_pixel[2] = (b5 << 3) | (b5 >> 2); +#ifdef USE_SEGGER_SYSVIEW + SEGGER_SYSVIEW_MarkStop(SYSVIEW_MARKER_CONVERT_IMAGE); +#endif } } } diff --git a/RockPaperScissors/AppKit-E8_USB/algorithm/data_in_user.c b/RockPaperScissors/AppKit-E8_USB/algorithm/data_in_user.c index d662bc2..64d1714 100644 --- a/RockPaperScissors/AppKit-E8_USB/algorithm/data_in_user.c +++ b/RockPaperScissors/AppKit-E8_USB/algorithm/data_in_user.c @@ -182,18 +182,6 @@ int32_t GetInputData (uint8_t *buf, uint32_t max_len) { SEGGER_SYSVIEW_MarkStop(SYSVIEW_MARKER_CAPTURE_IMAGE); #endif -#ifdef USE_SEGGER_SYSVIEW - SEGGER_SYSVIEW_MarkStart(SYSVIEW_MARKER_CONVERT_IMAGE); -#endif - -#ifdef USE_SEGGER_SYSVIEW - SEGGER_SYSVIEW_MarkStop(SYSVIEW_MARKER_CONVERT_IMAGE); -#endif - -#ifdef USE_SEGGER_SYSVIEW - SEGGER_SYSVIEW_MarkStart(SYSVIEW_MARKER_RESIZE_IMAGE); -#endif - /* Resize RGB image to fit ML model expected size */ crop_resize_rgb565_to_rgb888(inFrame, CAMERA_FRAME_WIDTH, @@ -202,10 +190,6 @@ int32_t GetInputData (uint8_t *buf, uint32_t max_len) { ML_IMAGE_WIDTH, ML_IMAGE_HEIGHT); -#ifdef USE_SEGGER_SYSVIEW - SEGGER_SYSVIEW_MarkStop(SYSVIEW_MARKER_RESIZE_IMAGE); -#endif - /* Release input frame */ if (vStream_VideoIn->ReleaseBlock() != VSTREAM_OK) { SDS_PRINTF("Failed to release video input frame\n"); diff --git a/RockPaperScissors/AppKit-E8_USB/scripts/README.md b/RockPaperScissors/AppKit-E8_USB/scripts/README.md new file mode 100644 index 0000000..195d5e3 --- /dev/null +++ b/RockPaperScissors/AppKit-E8_USB/scripts/README.md @@ -0,0 +1,84 @@ +# ML Out SDS Generator + +`generate_ml_out_sds_from_header.py` reads model class information from a C header file and generates an ML OUT SDS YAML file. + +## Requirements + +- Python 3.9 or later +- PyYAML + +Install the dependency with: + +```bash +python3 -m pip install PyYAML +``` + +## Input file + +The input is a C header file, typically named `model_config.h` in RockPaperScissors\AppKit-E8_USB\algorithm\ML Directory. + +## Usage + +Example using all defaults: + +```bash +python3 generate_ml_out_sds_from_header.py model_config.h +``` + +Example with a custom frequency: + +```bash +python3 generate_ml_out_sds_from_header.py model_config.h --frequency 10.0 +``` + +Example with a custom output path: + +```bash +python3 generate_ml_out_sds_from_header.py model_config.h --output generated/ML_Out.sds.yml +``` + +Example using short option names: + +```bash +python3 generate_ml_out_sds_from_header.py model_config.h -f 10.0 -o generated/ML_Out.sds.yml +``` + +## Command-line arguments + +| Argument | Required | Default | Description | +|---|---:|---:|---| +| `model_config` | Yes | None | Path to the input `model_config.h` file. | +| `-o`, `--output` | No | `./ML_Out.sds.yml` | Path of the generated YAML file. Parent directories are created automatically. | +| `-f`, `--frequency` | No | `8.33` | Frequency written to `sds.frequency`. The value must be greater than zero. | +| `-h`, `--help` | No | N/A | Displays command-line help. | + +To display help: + +```bash +python3 generate_ml_out_sds_from_header.py --help +``` + +## Output file + +By default, the script generates: + +```text +ML_Out.sds.yml +``` + +For the example input above, the generated file will look like: + +```yaml +sds: + name: ML output + description: ML classification results + frequency: 8.33 + content: + - value: CAT + type: float + - value: DOG + type: float + - value: BIRD + type: float +``` +**Note:** All labels are automatically converted to uppercase in the generated `ML_Out.sds.yml` file. diff --git a/RockPaperScissors/AppKit-E8_USB/scripts/generate_ml_out_sds_from_header.py b/RockPaperScissors/AppKit-E8_USB/scripts/generate_ml_out_sds_from_header.py new file mode 100644 index 0000000..aa3d056 --- /dev/null +++ b/RockPaperScissors/AppKit-E8_USB/scripts/generate_ml_out_sds_from_header.py @@ -0,0 +1,180 @@ +#!/usr/bin/env python3 +"""Generate ML_Out.sds.yml from MODEL_LABELS in model_config.h.""" + +import argparse +import ast +import re +import sys +from pathlib import Path + +import yaml + + +NUM_CLASSES_PATTERN = re.compile( + r"^\s*#\s*define\s+MODEL_NUM_CLASSES\s+\(?\s*(\d+)\s*\)?", + re.MULTILINE, +) +LABEL_ARRAY_PATTERN = re.compile( + r"static\s+const\s+char\s*\*\s*const\s+MODEL_LABELS\s*" + r"\[\s*MODEL_NUM_CLASSES\s*\]\s*=\s*\{(?P.*?)\}\s*;", + re.DOTALL, +) +C_STRING_PATTERN = re.compile(r'"(?:\\.|[^"\\])*"') + + +def parse_arguments() -> argparse.Namespace: + parser = argparse.ArgumentParser( + description=( + "Read MODEL_NUM_CLASSES and MODEL_LABELS from model_config.h, " + "then generate ML_Out.sds.yml." + ) + ) + parser.add_argument( + "model_config", + type=Path, + help="Path to model_config.h", + ) + parser.add_argument( + "-o", + "--output", + type=Path, + default=Path("ML_Out.sds.yml"), + help="Output YAML path (default: ./ML_Out.sds.yml)", + ) + parser.add_argument( + "-f", + "--frequency", + type=float, + default=8.33, + help="SDS output frequency (default: 8.33)", + ) + return parser.parse_args() + + +def remove_c_comments(text: str) -> str: + """Remove C/C++ comments while preserving string literals.""" + return re.sub( + r'("(?:\\.|[^"\\])*")|(/\*.*?\*/|//[^\r\n]*)', + lambda match: match.group(1) or "", + text, + flags=re.DOTALL, + ) + + +def decode_c_string(token: str) -> str: + """Decode a standard C string literal using Python-compatible escapes.""" + try: + value = ast.literal_eval(token) + except (SyntaxError, ValueError) as exc: + raise ValueError(f"Unable to parse label string: {token}") from exc + + if not isinstance(value, str): + raise ValueError(f"Label is not a string: {token}") + return value + + +def load_model_labels(header_path: Path) -> tuple[int, list[str]]: + if not header_path.is_file(): + raise FileNotFoundError(f"Header file not found: {header_path}") + + source = header_path.read_text(encoding="utf-8") + source_without_comments = remove_c_comments(source) + + num_classes_match = NUM_CLASSES_PATTERN.search(source_without_comments) + if not num_classes_match: + raise ValueError("Could not find numeric MODEL_NUM_CLASSES definition") + num_classes = int(num_classes_match.group(1)) + + labels_match = LABEL_ARRAY_PATTERN.search(source_without_comments) + if not labels_match: + raise ValueError( + "Could not find 'static const char * const " + "MODEL_LABELS[MODEL_NUM_CLASSES]' initializer" + ) + + array_body = labels_match.group("body") + labels = [decode_c_string(token) for token in C_STRING_PATTERN.findall(array_body)] + + if not labels: + raise ValueError("MODEL_LABELS does not contain any labels") + + if any(not label.strip() for label in labels): + raise ValueError("MODEL_LABELS contains an empty label") + + if len(labels) != num_classes: + raise ValueError( + f"MODEL_NUM_CLASSES is {num_classes}, but MODEL_LABELS contains " + f"{len(labels)} labels" + ) + + return num_classes, labels + + +def generate_sds_yaml(labels: list[str], output_path: Path, frequency: float) -> None: + document = { + "sds": { + "name": "ML output", + "description": "ML classification results", + "frequency": frequency, + "content": [ + {"value": label.upper(), "type": "float"} + for label in labels + ], + } + } + + output_path.parent.mkdir(parents=True, exist_ok=True) + with output_path.open("w", encoding="utf-8", newline="\n") as output_file: + yaml.safe_dump( + document, + output_file, + sort_keys=False, + default_flow_style=False, + allow_unicode=True, + width=4096, + ) + + +def validate_generated_yaml(output_path: Path, expected_count: int) -> None: + with output_path.open("r", encoding="utf-8") as output_file: + generated = yaml.safe_load(output_file) + + try: + content = generated["sds"]["content"] + except (TypeError, KeyError) as exc: + raise ValueError("Generated YAML does not contain sds.content") from exc + + if not isinstance(content, list): + raise ValueError("Generated YAML sds.content is not a list") + + if len(content) != expected_count: + raise ValueError( + f"Generated YAML contains {len(content)} labels, expected " + f"{expected_count}" + ) + + +def main() -> int: + args = parse_arguments() + + try: + if args.frequency <= 0: + raise ValueError("Frequency must be greater than 0") + + num_classes, labels = load_model_labels(args.model_config) + generate_sds_yaml(labels, args.output, args.frequency) + validate_generated_yaml(args.output, num_classes) + except (OSError, ValueError, yaml.YAMLError) as exc: + print(f"Error: {exc}", file=sys.stderr) + return 1 + + print( + f"Generated {args.output} with {len(labels)} labels at " + f"frequency={args.frequency}; count matches " + f"MODEL_NUM_CLASSES={num_classes}" + ) + return 0 + + +if __name__ == "__main__": + sys.exit(main())