Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Documentation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,8 @@ Closed: ML_Out (.\ML_Out.1.sds).

The ML model delivers four float values for each category `PAPER`, `ROCK`, `SCISSORS`, and `UNKNOWN`. The inference interval is about 8 Hz. This information is recorded in the `ML_Out.<n>.sds` files in binary format. The file `RockPaperScissors/SDS_Metadata/ML_Out.sds.yml` describes the content and using this file with SDS utilities allows you to view the content.

A utility script to auto-generate this `ML_Out.sds.yml` from a `model_config.h` header is available [here](../RockPaperScissors/AppKit-E8_USB/scripts/README.md).

**Example output:**

```txt
Expand Down
133 changes: 133 additions & 0 deletions Documentation/creating-an-example-using-rps-template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
### Creating an New Use Example from the RockPaperScissors Template

This guide explains how to create a new image classification use case by reusing the existing `RockPaperScissors` application as the template project.

The `RockPaperScissors` directory contains the reusable embedded application structure. When a user selects another use case in ModelNova Fusion Studio, such as `VehicleClassification` or `ToolClassification`, Fusion Studio uses the existing `RockPaperScissors` project directory and replaces the model-specific artifacts for the selected use case.

#### Repository Structure

Open the ModelNova repository root. The reusable template project is located at:

```text
<ModelNova repository root>
+-- RockPaperScissors
+-- AppKit-E8_USB
| +-- algorithm
| | +-- ML
| | +-- ai_layer
| | +-- cil_layer
| | | +-- include
| | | +-- src
| | +-- model_config.h
| +-- Board
| +-- sdsio
| +-- SDS.csolution.yml
+-- RPS_cls_dataset
```

Fusion Studio updates the model-related content inside this existing project when replacing the use case artifacts.

#### Supported Model Categories

Fusion Studio supports generating model artifacts for image classification models based on the supported model architectures.

Choose the architecture that best meets your application requirements, considering factors such as accuracy, latency, memory footprint, and target hardware.

#### Prerequisites

Before updating the use case, make sure you have:

- ModelNova Fusion Studio installed and configured.
- A dataset prepared for the selected use case.
- Keil Studio for VS Code.
- A working build of the existing `RockPaperScissors` example.
- Access to the target board and flashing/debugging setup.

#### Step 1: Use the Existing RockPaperScissors Project

Use the existing `RockPaperScissors` directory as the template project for the selected use case.

The project path remains:

```text
RockPaperScissors\AppKit-E8_USB
```

Fusion Studio replaces the model artifacts in this existing directory for the use case selected by the user. For example, a user may select a vehicle classification, tool classification, food classification, or defect classification use case, but the embedded project directory remains `RockPaperScissors`.

#### Step 2: Generate Model Artifacts Using Fusion Studio

Follow the [Create ML model](./README.md#create-ml-model) section above to train, validate, and deploy your model in Fusion Studio. When generating artifacts, select the existing `RockPaperScissors` project as the target for artifact replacement.

Export the generated `.cilpkg` package if manual replacement is needed.

For example, a vehicle classification dataset might contain classes such as:

```text
car
bus
truck
motorcycle
```

> [!NOTE]
> Fusion Studio replaces the model artifacts automatically. Verify that the required files are present in the existing `RockPaperScissors` project.

#### Step 3 (Optional): Replace the Model Artifacts Manually

If you choose to update the artifacts manually, unzip or extract the `.cilpkg` package generated by Fusion Studio.

The extracted package should contain the required Common Inference Library artifacts:

```text
ai_layer
cil.a
cil.h
model_config.h
```
> **For details about the `.cilpkg` package, refer to the [CIL Package Documentation](../RockPaperScissors/AppKit-E8_USB/algorithm/ML/README.md)**.

These files replace the template model artifacts in the existing `RockPaperScissors` project.

Copy the extracted artifacts into the following locations:

| Artifact | Destination |
| --- | --- |
| `ai_layer` | `RockPaperScissors\AppKit-E8_USB\algorithm\ML\ai_layer` |
| `model_config.h` | `RockPaperScissors\AppKit-E8_USB\algorithm\ML\model_config.h` |
| `cil.a` | `RockPaperScissors\AppKit-E8_USB\algorithm\ML\cil_layer\src\cil.a` |
| `cil.h` | `RockPaperScissors\AppKit-E8_USB\algorithm\ML\cil_layer\include\cil.h` |

Expected paths:

```text
RockPaperScissors\AppKit-E8_USB\algorithm\ML\ai_layer
RockPaperScissors\AppKit-E8_USB\algorithm\ML\model_config.h
RockPaperScissors\AppKit-E8_USB\algorithm\ML\cil_layer\src\cil.a
RockPaperScissors\AppKit-E8_USB\algorithm\ML\cil_layer\include\cil.h
```

#### Step 4: Review Application Configuration

After replacing the model artifacts, review the application configuration for any use case specific changes.

Check the following areas:

- Class labels and output post-processing.
- Input image size and preprocessing requirements.
- Number of output classes.
- Dataset-specific test files or SDS recordings.
- Any user-facing log messages that still mention Rock/Paper/Scissors.

The exact files to update depend on the model output and application logic. Start by reviewing files under:

```text
RockPaperScissors\AppKit-E8_USB\algorithm
RockPaperScissors\AppKit-E8_USB\algorithm\ML
```

#### Notes

- Fusion Studio uses the existing `RockPaperScissors` project directory for replacing artifacts for the selected use case.
- Do not create a separate directory for each selected use case.
- Regenerate and replace all four model artifacts together when changing the model.
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,8 @@ The diagram below illustrates the RPS application architecture. During algorithm

While Fusion Studio supports multiple workspace creation methods (Scratch, Starter Pack, AI Assist), the **ML workflow remains identical for all workspaces**.

> The `RockPaperScissors` project also serves as a [template for creating new examples](./Documentation/README.md#creating-a-new-use-case-from-the-rps-template).

## Analyze Timing with SystemView

[SEGGER SystemView](https://www.segger.com/products/development-tools/systemview/) can be used for timing analysis. The application includes annotations that let you measure the timing of different compute blocks, as shown below.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,11 @@
#include "cmsis_compiler.h"
#include "image_processing_func.h"

#ifdef USE_SEGGER_SYSVIEW
#include "SEGGER_SYSVIEW.h"
#include "sysview_markers.h"
#endif

/*
Clamp a value to a specified range.

Expand Down Expand Up @@ -323,6 +328,9 @@ void crop_resize_rgb565_to_rgb888(
int dst_width,
int dst_height)
{
#ifdef USE_SEGGER_SYSVIEW
SEGGER_SYSVIEW_MarkStart(SYSVIEW_MARKER_RESIZE_IMAGE);
#endif
int crop_size = src_height; // 720 for 1280x720
int crop_x = (src_width - crop_size) / 2; // center horizontally
int crop_y = 0;
Expand All @@ -339,6 +347,13 @@ void crop_resize_rgb565_to_rgb888(
int sx = (src_x_fp >> FP_SHIFT) + crop_x;

int idx = (sy * src_width + sx) * 2;
#ifdef USE_SEGGER_SYSVIEW
SEGGER_SYSVIEW_MarkStop(SYSVIEW_MARKER_RESIZE_IMAGE);
#endif

#ifdef USE_SEGGER_SYSVIEW
SEGGER_SYSVIEW_MarkStart(SYSVIEW_MARKER_CONVERT_IMAGE);
#endif
uint16_t pixel = src[idx] | (src[idx + 1] << 8);

uint8_t r5 = (pixel >> 11) & 0x1F;
Expand All @@ -350,6 +365,9 @@ void crop_resize_rgb565_to_rgb888(
dst_pixel[0] = (r5 << 3) | (r5 >> 2);
dst_pixel[1] = (g6 << 2) | (g6 >> 4);
dst_pixel[2] = (b5 << 3) | (b5 >> 2);
#ifdef USE_SEGGER_SYSVIEW
SEGGER_SYSVIEW_MarkStop(SYSVIEW_MARKER_CONVERT_IMAGE);
#endif
}
}
}
Expand Down
16 changes: 0 additions & 16 deletions RockPaperScissors/AppKit-E8_USB/algorithm/data_in_user.c
Original file line number Diff line number Diff line change
Expand Up @@ -182,18 +182,6 @@ int32_t GetInputData (uint8_t *buf, uint32_t max_len) {
SEGGER_SYSVIEW_MarkStop(SYSVIEW_MARKER_CAPTURE_IMAGE);
#endif

#ifdef USE_SEGGER_SYSVIEW
SEGGER_SYSVIEW_MarkStart(SYSVIEW_MARKER_CONVERT_IMAGE);
#endif

#ifdef USE_SEGGER_SYSVIEW
SEGGER_SYSVIEW_MarkStop(SYSVIEW_MARKER_CONVERT_IMAGE);
#endif

#ifdef USE_SEGGER_SYSVIEW
SEGGER_SYSVIEW_MarkStart(SYSVIEW_MARKER_RESIZE_IMAGE);
#endif

/* Resize RGB image to fit ML model expected size */
crop_resize_rgb565_to_rgb888(inFrame,
CAMERA_FRAME_WIDTH,
Expand All @@ -202,10 +190,6 @@ int32_t GetInputData (uint8_t *buf, uint32_t max_len) {
ML_IMAGE_WIDTH,
ML_IMAGE_HEIGHT);

#ifdef USE_SEGGER_SYSVIEW
SEGGER_SYSVIEW_MarkStop(SYSVIEW_MARKER_RESIZE_IMAGE);
#endif

/* Release input frame */
if (vStream_VideoIn->ReleaseBlock() != VSTREAM_OK) {
SDS_PRINTF("Failed to release video input frame\n");
Expand Down
84 changes: 84 additions & 0 deletions RockPaperScissors/AppKit-E8_USB/scripts/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# ML Out SDS Generator

`generate_ml_out_sds_from_header.py` reads model class information from a C header file and generates an ML OUT SDS YAML file.

## Requirements

- Python 3.9 or later
- PyYAML

Install the dependency with:

```bash
python3 -m pip install PyYAML
```

## Input file

The input is a C header file, typically named `model_config.h` in RockPaperScissors\AppKit-E8_USB\algorithm\ML Directory.

## Usage

Example using all defaults:

```bash
python3 generate_ml_out_sds_from_header.py model_config.h
```

Example with a custom frequency:

```bash
python3 generate_ml_out_sds_from_header.py model_config.h --frequency 10.0
```

Example with a custom output path:

```bash
python3 generate_ml_out_sds_from_header.py model_config.h --output generated/ML_Out.sds.yml
```

Example using short option names:

```bash
python3 generate_ml_out_sds_from_header.py model_config.h -f 10.0 -o generated/ML_Out.sds.yml
```

## Command-line arguments

| Argument | Required | Default | Description |
|---|---:|---:|---|
| `model_config` | Yes | None | Path to the input `model_config.h` file. |
| `-o`, `--output` | No | `./ML_Out.sds.yml` | Path of the generated YAML file. Parent directories are created automatically. |
| `-f`, `--frequency` | No | `8.33` | Frequency written to `sds.frequency`. The value must be greater than zero. |
| `-h`, `--help` | No | N/A | Displays command-line help. |

To display help:

```bash
python3 generate_ml_out_sds_from_header.py --help
```

## Output file

By default, the script generates:

```text
ML_Out.sds.yml
```

For the example input above, the generated file will look like:

```yaml
sds:
name: ML output
description: ML classification results
frequency: 8.33
content:
- value: CAT
type: float
- value: DOG
type: float
- value: BIRD
type: float
```
**Note:** All labels are automatically converted to uppercase in the generated `ML_Out.sds.yml` file.
Loading