Unity Integration Guide: Multiple Agent Simulation & Debug Visualization

This guide outlines the steps to integrate the C++ core of AgentSimMiddleware with a Unity project to demonstrate a multiple agent simulation with in-editor debug visualization.

Prerequisites

A Unity project (version 2021.3 or newer recommended).
CMake (version 3.15 or newer).
A C++ compiler (e.g., Visual Studio on Windows, GCC/Clang on Linux/macOS).
The C++ AgentSimMiddleware project compiled into a dynamic link library (DLL for Windows, .so for Linux, .dylib for macOS).

Integration Steps

1. Build the C++ DLL

The C++ core needs to be compiled into a shared library (DLL on Windows) that Unity can load.

Navigate to the project root: Open your terminal or command prompt and go to the AgentSimMiddleware project root directory.
```
cd path/to/AgentSimMiddleware
```
Clean previous build (optional but recommended):
```
Remove-Item -Recurse -Force build
```
Create build files using CMake:
```
cmake -B build -S .
```
This command creates a build directory and generates the necessary build files (e.g., Visual Studio solution on Windows, Makefiles on Linux/macOS).
Build the project:
```
cmake --build build --config Release
```
This command compiles the C++ code. The AgentSimMiddleware.dll (or equivalent for your OS) will be generated.
- On Windows: The DLL will typically be found at build/Debug/AgentSimMiddleware.dll or build/Release/AgentSimMiddleware.dll depending on the --config specified. For Unity, you will generally use the Release version.
- On Linux/macOS: The shared library will be named libAgentSimMiddleware.so or libAgentSimMiddleware.dylib respectively.

2. Copy C# Files to Unity Project

Copy the following C# files from AgentSimMiddleware/src/csharp/ into your Unity project's Assets folder (e.g., Assets/AgentSimMiddleware/Scripts/):

DataStructures.cs
AgentCommand.cs
WorldSnapshot.cs
UnityAdapter.cs
WorldSnapshotMarshaler.cs
AgentBehaviour.cs
SimulationManager.cs
SimulationSettings.cs
AgentDebugVisualizer.cs
DebugSettings.cs
DebugUI.cs

3. Copy C++ DLL to Unity Project

Copy the compiled C++ DLL (AgentSimMiddleware.dll, libAgentSimMiddleware.so, or libAgentSimMiddleware.dylib) into your Unity project's Assets/Plugins/ folder. Unity automatically handles platform-specific DLL loading.

4. Expose C++ Functions to C# (P/Invoke)

The UnityAdapter.cs file uses Platform Invoke (P/Invoke) to call native C++ functions. Ensure the corresponding C++ functions are properly exposed from the AgentSimMiddleware.dll using the AGENTSIM_API macro.

The functions exposed by AgentSimApi.h that are consumed by UnityAdapter.cs include:

// Functions from AgentSimApi.h
[DllImport(DLL_NAME)]
private static extern IntPtr CreateAgentContainer(int maxAgents, string goapJsonContent);

[DllImport(DLL_NAME)]
private static extern void DestroyAgentContainer(IntPtr container);

[DllImport(DLL_NAME)]
private static extern void SetLogCallback(LogCallbackFn callback);

[DllImport(DLL_NAME)]
private static extern void SetLODThresholds(IntPtr agentContainerHandle, float fullSimDistance, float liteSimDistance);

[DllImport(DLL_NAME)]
private static extern int GetGoalIdByName(IntPtr container, string goalName);

[DllImport(DLL_NAME)]
private static extern void SetVerboseMode(bool enabled);

[DllImport(DLL_NAME)]
private static extern int AddAgent(IntPtr container, Vector3 initialPos, Quaternion initialRot, int initialGoal, int initialState);

[DllImport(DLL_NAME)]
private static extern void Update(IntPtr container, float deltaTime, [In] ref Vector3 cameraPosition,
                                         [In] ref MarshaledWorldSnapshot marshaledSnapshot,
                                         [Out] AgentCommand[] agentCommands, ref int commandCount, int maxCommandCapacity);

[DllImport(DLL_NAME)]
private static extern void StepSimulationIncremental(IntPtr container, float deltaTime, [In] ref IncrementalWorldSnapshot incrementalSnapshot,
                                                             [Out] AgentCommand[] agentCommands, ref int commandCount, int maxCommandCapacity,
                                                             [In] ref Vector3 cameraPosition);

[DllImport(DLL_NAME)]
private static extern void GetLastSimulationTimings(IntPtr container, out SimulationStageTimings outTimings);

// Other functions from AgentSimApi.h, potentially used indirectly or by debug tools
// [DllImport(DLL_NAME)]
// public static extern void RemoveAgent(IntPtr agentContainerHandle, int agentId);
// [DllImport(DLL_NAME)]
// public static extern int GetAgentCount(IntPtr agentContainerHandle);
// [DllImport(DLL_NAME)]
// public static extern void GetAgentData(IntPtr agentContainerHandle, [Out] AgentDebugData[] agentData, ref int count);

5. Create and Configure Unity Simulation Settings ScriptableObject

The simulation's core parameters are managed through a SimulationSettings ScriptableObject.

In your Unity project, navigate to Assets/AgentSimMiddleware/Scripts/SimulationSettings.cs.
In the Unity Editor, right-click in your Project window -> Create -> AgentSim -> Simulation Settings. Name it MySimulationSettings.
Configure the settings in the Inspector:
- Full Simulation Distance: Agents within this distance of the camera will be fully simulated.
- Lite Simulation Distance: Agents within this distance (but outside Full Simulation Distance) will be lite simulated.
- GOAP Json File Path: Path to your GOAP JSON file (e.g., actions_goals).
- Max Agents: Maximum number of agents the simulation can handle.
- Agent Prefab: Assign a simple cube or sphere prefab (see Step 7).
- Enable Debug Visualizations: Toggle debug rendering in the editor.

6. Create and Configure Unity Simulation Manager Script

Create an empty GameObject in your scene (e.g., named "SimulationManager") and attach the SimulationManager.cs script (copied in Step 2) to it.

Assign your MySimulationSettings ScriptableObject to the Settings field of the SimulationManager component in the Inspector.
Add a UIDocument component to the "SimulationManager" GameObject and assign the DebugUI.uxml file (found in Assets/AgentSimMiddleware/UI/) to its Source Asset field. This will display the debug toggles.

The SimulationManager script will handle initialization, agent creation, and updating the C++ simulation core based on the settings you've provided.

using UnityEngine;
using UnityEngine.UIElements;
using AgentSimMiddleware;
using System.Collections.Generic;

public class SimulationManager : MonoBehaviour
{
    public SimulationSettings settings; // Assign your SimulationSettings ScriptableObject in the Inspector
    private Dictionary<int, GameObject> agentGameObjects = new Dictionary<int, GameObject>();
    private AgentDebugVisualizer agentDebugVisualizer; // Handles visual aspects of debugging

    void Start()
    {
        if (settings == null)
        {
            Debug.LogError("SimulationSettings ScriptableObject is not assigned to the SimulationManager. Please assign it in the Inspector.");
            enabled = false; // Disable the component if settings are missing
            return;
        }

        // Initialize the C++ simulation core via UnityAdapter
        bool initSuccess = UnityAdapter.Init(settings.MaxAgents, settings.GOAPJsonFilePath);
        if (initSuccess)
        {
            Debug.Log("C++ Simulation Initialized Successfully.");

            // Apply LOD thresholds from settings
            UnityAdapter.SetLODThresholds(settings.FullSimulationDistance, settings.LiteSimulationDistance);

            // Initialize debug visualizer if enabled
            if (settings.EnableDebugVisualizations)
            {
                agentDebugVisualizer = new AgentDebugVisualizer();
                agentDebugVisualizer.Initialize(settings.MaxAgents);
            }

            // Add agents based on MaxAgents setting
            for (int i = 0; i < settings.MaxAgents; i++)
            {
                // Example initial positions and states
                Vector3 initialPos = new Vector3((i % 20) * 1.5f, 0.5f, (i / 20) * 1.5f);
                Quaternion initialRot = Quaternion.identity;
                int initialGoal = 0; // Default Goal ID (e.g., GATHER_WOOD)
                int initialState = (int)AgentState.IDLE; // Default FSM State

                int agentId = UnityAdapter.AddAgent(initialPos, initialRot, initialGoal, initialState);
                if (agentId >= 0)
                {
                    CreateAgentGameObject(agentId, initialPos);
                }
                else
                {
                    Debug.LogError($"Failed to add agent {i}.");
                }
            }
        }
        else
        {
            Debug.LogError("Failed to initialize C++ Simulation.");
            enabled = false;
        }
    }

    void Update()
    {
        // Simulate camera position (important for LOD calculations in C++)
        Vector3 cameraPosition = Camera.main != null ? Camera.main.transform.position : Vector3.zero;

        // Step the C++ simulation using the wrapper which handles incremental snapshots and command processing
        UnityAdapter.StepSimulationWrapper(Time.deltaTime, cameraPosition);

        // Update debug visuals
        if (settings.EnableDebugVisualizations && agentDebugVisualizer != null)
        {
            UnityAdapter.AgentDisplayData[] agentData = UnityAdapter.GetAgentDataWrapper();
            agentDebugVisualizer.UpdateAgentVisuals(agentData);
        }
    }

    void OnDestroy()
    {
        // Shutdown the C++ simulation core when the GameObject is destroyed
        UnityAdapter.Shutdown();
        Debug.Log("C++ Simulation Shutdown.");
    }

    void CreateAgentGameObject(int agentId, Vector3 initialPos)
    {
        if (settings.AgentPrefab != null)
        {
            GameObject newAgentGo = Instantiate(settings.AgentPrefab, initialPos, Quaternion.identity);
            newAgentGo.name = $"Agent_{agentId}";
            agentGameObjects.Add(agentId, newAgentGo);
            // Attach AgentBehaviour script and assign ID
            AgentBehaviour agentBehaviour = newAgentGo.AddComponent<AgentBehaviour>();
            agentBehaviour.AgentId = agentId;
            // Register GameObject with UnityAdapter for world object lookup if needed
            UnityAdapter.RegisterGameObject(newAgentGo.GetInstanceID(), newAgentGo);
        }
        else
        {
            Debug.LogError("Agent Prefab is not assigned in Simulation Settings!");
        }
    }
}

7. Implement AgentBehaviour Script (with Debug Visualization)

The AgentBehaviour.cs script (copied in Step 2) is automatically attached to each agent's GameObject by the SimulationManager. It is responsible for reacting to AgentCommands and providing visual feedback.

using UnityEngine;
using AgentSimMiddleware;
using UnityEditor; // Required for Handles.Label in OnDrawGizmosSelected

public class AgentBehaviour : MonoBehaviour
{
    public int AgentId = -1; // Public property for the agent's ID, assign in Unity Inspector

    // Called when the script instance is being loaded or enabled
    private void OnEnable()
    {
        if (AgentId != -1)
        {
            UnityAdapter.RegisterAgentBehaviour(AgentId, this);
        }
    }

    // Called when the behaviour becomes disabled or inactive
    private void OnDisable()
    {
        if (AgentId != -1)
        {
            UnityAdapter.UnregisterAgentBehaviour(AgentId);
        }
    }

    public void ApplyAgentCommand(AgentCommand command)
    {
        switch (command.commandType)
        {
            case AgentCommandType.MOVE_TO_POSITION:
                transform.position = new Vector3(command.targetPosition.x, command.targetPosition.y, command.targetPosition.z);
                break;
            case AgentCommandType.ARRIVED_AT_POSITION:
                // Optional: Trigger an event or animation when agent arrives
                break;
            case AgentCommandType.INTERACT_WITH_OBJECT:
                GameObject targetObject = UnityAdapter.GetGameObjectById(command.targetObjectId);
                if (targetObject != null) {
                    Destroy(targetObject); // Destroy the Unity object
                    UnityAdapter.UnregisterGameObject(command.targetObjectId); // Unregister from adapter
                }
                break;
            case AgentCommandType.PLAY_ANIMATION:
                // Assuming the agent GameObject has an Animator component
                Animator animator = GetComponent<Animator>();
                if (animator != null)
                {
                    animator.Play(command.AnimationName);
                }
                break;
            case AgentCommandType.CHANGE_STATE:
                // Example: change material color based on FSM state
                MeshRenderer renderer = GetComponent<MeshRenderer>();
                if (renderer != null)
                {
                    switch ((AgentState)command.newState)
                    {
                        case AgentState.IDLE: renderer.material.color = Color.gray; break;
                        case AgentState.WANDERING: renderer.material.color = Color.blue; break;
                        case AgentState.PURSUING_GOAL: renderer.material.color = Color.green; break;
                        // ... handle other states
                        default: renderer.material.color = Color.white; break;
                    }
                }
                break;
            default:
                Debug.LogWarning($"Agent {AgentId} received unknown command type: {command.commandType}");
                break;
        }
    }

    private void OnDrawGizmosSelected()
    {
        if (!DebugSettings.ShowAgentGizmos) return;

        if (AgentId == -1) return;

        // Fetch current agent data from the adapter
        UnityAdapter.AgentDisplayData[] allAgentData = UnityAdapter.GetAgentDataWrapper();
        UnityAdapter.AgentDisplayData? myAgentData = null;
        foreach (var data in allAgentData)
        {
            if (data.agentId == AgentId)
            {
                myAgentData = data;
                break;
            }
        }

        if (myAgentData.HasValue)
        {
            UnityAdapter.AgentDisplayData data = myAgentData.Value;

            // Gizmo styling
            GUIStyle style = new GUIStyle();
            style.normal.textColor = Color.white;
            style.fontSize = 14;

            // Display Agent ID
#if UNITY_EDITOR
            UnityEditor.Handles.Label(transform.position + Vector3.up * 1.0f, $"ID: {AgentId}", style);
#endif
            // Display LOD
            Color lodColor = Color.white;
            string lodString = "";
            switch (data.lodTier) // Using int directly as LODTier enum is internal to C++
            {
                case 0: lodColor = Color.green; lodString = "FullSim"; break;
                case 1: lodColor = Color.yellow; lodString = "LiteSim"; break;
                case 2: lodColor = Color.red; lodString = "None"; break; // Assuming 2 is 'None' for now
                default: lodColor = Color.gray; lodString = "Unknown"; break;
            }
            Gizmos.color = lodColor;
            Gizmos.DrawWireSphere(transform.position, 0.6f);
#if UNITY_EDITOR
            UnityEditor.Handles.Label(transform.position + Vector3.up * 0.8f, $"LOD: {lodString}", style);
#endif
            // Display State
#if UNITY_EDITOR
            UnityEditor.Handles.Label(transform.position + Vector3.up * 0.6f, $"State: {(AgentState)data.fsmState}", style);
#endif
            // Display Goal
#if UNITY_EDITOR
            UnityEditor.Handles.Label(transform.position + Vector3.up * 0.4f, $"Goal: {(Goal)data.currentGoal}", style);
#endif
            // Display Action
#if UNITY_EDITOR
            UnityEditor.Handles.Label(transform.position + Vector3.up * 0.2f, $"Action: {data.currentActionName}", style);
#endif
            
            // Visualize target position if enabled in DebugSettings
            if (DebugSettings.ShowAgentTargetPositions && 
                ((AgentState)data.fsmState == AgentState.PURSUING_GOAL || (AgentState)data.fsmState == AgentState.MOVE_TO))
            {
                Gizmos.color = Color.blue;
                Vector3 targetWorldPosition = data.targetPosition;
                Gizmos.DrawLine(transform.position, targetWorldPosition);
                Gizmos.DrawSphere(targetWorldPosition, 0.2f);
#if UNITY_EDITOR
                UnityEditor.Handles.Label(targetWorldPosition + Vector3.up * 0.3f, "Target", style);
#endif
            }

            // Visualize perception radius if enabled in DebugSettings
            if (DebugSettings.ShowAgentPerceptionRadius)
            {
                Gizmos.color = new Color(1f, 0.5f, 0f, 0.3f); // Orange, semi-transparent
                // Assuming a default perception radius for now, or fetch from C++ if exposed
                Gizmos.DrawWireSphere(transform.position, 5.0f); // Example: 5 unit radius
            }
        }
    }
}

8. Add GOAP Data File to Unity Project

The GOAP planner loads its actions and goals from a JSON file.

Copy AgentSimMiddleware/data/actions_goals.json into your Unity project's Assets/Resources/ folder (you may need to create this folder). Unity will then be able to load this file as a TextAsset.
Ensure the GOAPJsonFilePath in your MySimulationSettings ScriptableObject (Step 5) matches the name of this file (e.g., actions_goals).

9. Create an Agent Prefab

Create a simple 3D object (e.g., a Cube or Sphere) in Unity, drag it into your Assets folder to create a Prefab, and assign this Prefab to the Agent Prefab field on your MySimulationSettings ScriptableObject in the Inspector.

10. Run the Scene

Run your Unity scene. You should see debug messages confirming the C++ simulation initialization and agent addition. The agent's GameObject should then move and behave according to the loaded GOAP data, and debug visualizations will appear in the Scene view if enabled.

This completes the updated setup for basic C# to C++ interop and multiple agent simulation with data-driven GOAP and debug visualization.

FilesExpand file tree

unity-integration-guide.md

Latest commit

History