PlotCore Studio

AI-Powered Animated Storytelling Studio
IBM Hackathon Submission · Full-Stack Next.js Web Application

PlotCore Studio is an end-to-end AI animation production workspace that brings story ideation, character and voice development, world building, scripting, scene planning, storyboarding, quality review, rendering, export, publishing, and analytics into one unified interface.

At the heart of the platform is PABU — PlotCore's intelligent Creative Director and Production Assistant — who oversees every stage of every project from first idea to final published animation.

Demo Video

See how users can use PlotCore Studio in the interactive product demo.

Meet PABU — Your AI Creative Director

PABU (PlotCore AI Brand-Unified Director) is the intelligent creative director and production assistant at the heart of PlotCore Studio. It collaborates with creators throughout the entire production lifecycle — from the first spark of an idea to the final published animation.

Rather than simply responding to prompts, PABU thinks like an experienced creative director. It asks insightful questions, challenges weak ideas, suggests stronger creative choices, and coordinates every stage of production to help transform concepts into compelling visual stories.

What PABU Does

Role	How PABU helps
Story Development	Generates story concepts, refines premises, identifies plot holes, and ensures narrative coherence across all scenes
Character Direction	Builds character personalities, enforces visual and voice consistency, and maintains relationship dynamics scene-to-scene
Scene & Cinematography	Suggests camera angles, lighting moods, transitions, and shot composition matched to each scene's emotional beat
Production Management	Runs the full quality review pipeline — visual consistency, lip sync, voice adherence, environment assignment, plot continuity
Animation Generation	Coordinates the render pipeline, dispatches generation jobs, and flags issues before a single frame is produced
Creative Refinement	Reviews the completed project, scores it 0–100, categorises errors and warnings, and auto-fixes common issues with one click

How PABU Thinks

PABU does not wait to be asked. It proactively:

Challenges story choices that weaken the narrative
Surfaces continuity errors before they reach the render queue
Proposes creative alternatives when a scene feels underdeveloped
Remembers every character, world, and scene decision made in the project
Adapts its guidance to the creator's experience level and production goals

Who PABU Serves

Whether you are an individual creator, a studio team, an educator, or a marketing team, PABU functions as your dedicated creative partner — ensuring every project maintains quality, consistency, and artistic vision from concept to completion.

PABU in the Interface

PABU is always present as a global floating assistant (bottom-right of every studio page). Click the PABU button to open a terminal-style chat panel with:

Route-aware quick actions (different suggestions on each studio page)
Full project context injected automatically with every message
Capability pills for Scenes, Characters, Script, Generate, and Analytics
Opt-in PABU Live — voice conversation with real-time speech recognition and optional screen context

Screenshots

Dashboard

Story Ideas Lab

Voices

Analytics & Audience

Plan & Billing

Profile Settings

Notification Settings

Problem Statement

The current AI animation landscape is deeply fragmented. Writers use ChatGPT for scripts, artists use Midjourney or Flux for character art, producers use ElevenLabs for voices, animators use Runway or Kling for motion, and editors use CapCut or Premiere for final assembly. This fragmentation causes three compounding problems:

Inconsistency — Characters look and sound different across tools because no shared memory exists between them.
Inefficiency — Constant context switching between 5–6 separate tools kills creative flow and doubles production time.
Inaccessibility — The full stack of professional AI tools costs hundreds of dollars per month, pricing out independent creators, students, and educators.
No unified quality gate — There is no automated process to catch continuity errors, lip-sync drift, or plot holes before a project is rendered.

Solution Description

PlotCore Studio unifies the entire animation production pipeline into a single guided 7-step workspace:

Characters → Voices → Worlds → Script → Scenes → Review → Generate

Each step feeds into the next through a shared Story Memory Engine that tracks character appearance fingerprints, voice embeddings, personality profiles, wardrobe records, and relationship graphs. This ensures a character looks and sounds identical from Scene 1 to Scene 30 — something no combination of standalone tools can guarantee today.

Key capabilities:

Module	What it does
Character Studio	Create characters with AI-generated portraits, consistency IDs, and personality matrices
Voice Studio	Assign from 50+ AI voices or transform uploaded audio (pitch, age, personality)
World Builder	Define environments, lighting, and camera styles reused across scenes
Script Editor	AI-assisted dialogue generation, scene description, and plot-hole detection
Scene Planner	Compose scenes by assigning characters, worlds, camera angles, and dialogue
PABU AI Director	Automated quality review with a 0–100 score, error categorisation, and one-click auto-fix
Render & Export	Generate the final animation, export at up to 4K, and publish or share
Analytics	Track views, engagement, and audience retention per project

AI Approach and Architecture

Story Memory Engine

Every character, voice, and world created in PlotCore is stored as structured data with AI-generated embeddings. When a scene is generated, the Story Memory Engine validates:

Does the character's appearance match its stored visual fingerprint?
Is the outfit consistent with prior scenes?
Does the dialogue match the character's personality profile?
Are relationship dynamics between characters correct?

Inconsistencies are flagged automatically for review.

PABU AI Director

PABU (PlotCore AI Brand-Unified Director) is the automated quality review engine. It runs six parallel analysis passes before any render job is dispatched:

Project Data
     │
     ├── Visual Consistency Checker   — character appearance across scenes
     ├── Voice Consistency Checker    — voice embedding adherence per scene
     ├── Lip Sync Analyser            — dialogue timing vs. frame timing
     ├── Environment Validator        — all scenes have an assigned world
     ├── Transition Analyser          — detects abrupt or missing transitions
     └── Plot Coherence Engine        — story logic and continuity

Quality score: (passedChecks / totalChecks) × 100

PABU AI Engine — Python Backend

A standalone Python 3.11 / FastAPI service (backend/pabu-ai/) handles all real AI inference. The Next.js frontend calls it at NEXT_PUBLIC_PABU_API (default http://localhost:8000).

PabuLive (Next.js)
       │  POST /chat  |  POST /chat/image  |  POST /review
       ▼
FastAPI Server  (api/server.py  :8000)
       │
       ▼
pabu_engine.chat()
  ├── system_prompt.txt     (PABU persona + director rules)
  ├── context injection     (current route, active project)
  └── LLM routing
        ├── OpenAI GPT-4o       (default)
        ├── Anthropic Claude    (fallback / long-context)
        ├── Google Gemini       (multimodal image/video)
        └── Ollama (local)      (offline / fine-tuned model)

PABU can be fine-tuned on custom animation project data using LoRA (Low-Rank Adaptation) via HuggingFace peft, fitting on a single consumer GPU.

Production AI Pipeline

Scene description
      │
      ▼
Environment AI (background from World settings)
      │
      ▼
Character Placement AI (pose + position from action description)
      │
      ▼
Motion Synthesis AI (AnimateDiff / Kling)
      │
      ▼
Lip Sync Module (phoneme → viseme mapping via Wav2Lip)
      │
      ▼
Final animated scene

Selected Challenge Theme

PlotCore Studio addresses the Creative AI and Content Generation challenge theme — specifically, democratising professional-grade animation production through AI.

The platform targets a broad audience of underserved creators: independent filmmakers, educators producing classroom content, students entering film competitions, and parents creating personalised stories for children. By collapsing a five-tool workflow into one guided workspace and automating the quality review stage that previously required an experienced producer, PlotCore makes animated storytelling accessible regardless of technical skill or budget.

How IBM Bob Was Used

IBM Bob was used as the primary engineering collaborator throughout the development of PlotCore Studio. Specific contributions include:

Architecture and scaffolding Bob designed the overall Next.js 16 App Router structure, laid out the 20-page studio workflow, and scaffolded the component library (AppShell, Sidebar, PabuFloater, PabuLive, FeaturePage, ProjectCard, and all shared UI primitives).

PABU AI engine Bob authored the full Python backend: pabu_engine.py (core orchestrator), api/server.py (FastAPI service), config.py (environment and model config), conversation_memory.py (per-session memory), voice_handler.py (transcription and TTS adapters), and training/trainer.py (LoRA fine-tuning loop). Bob also wrote the PABU system prompt in prompts/system_prompt.txt.

PabuFloater and PabuLive components Bob built the global floating chat assistant (fixed FAB with scan-line rings, slide-in terminal-style chat panel, route-aware suggestions, capability pills, and minimise support) and the opt-in voice assistant with browser speech recognition, speech synthesis, and optional periodic screen-frame analysis via POST /chat/image.

Context and state management Bob wired PabuContext for global open/close state, ThemeContext for dark/light switching, and SidebarContext for collapse state — all integrated into AppShell.

UI/UX refinements Bob resolved Turbopack stale-cache issues, fixed the sidebar collapse button visibility, raised the dashboard two-column breakpoint, implemented the landing page dual-panel commercial section, and enforced a canonical system font stack.

Documentation Bob authored the full documentation suite under documentation/ (PROJECT_OVERVIEW.md, ARCHITECTURE.md, AI_FEATURES.md, FEATURES.md, COMPONENTS.md, DESIGN_SYSTEM.md, API_REFERENCE.md, DEPLOYMENT.md, ROADMAP.md) and backend/pabu-ai/README.md.

Pricing system Bob implemented a single-source-of-truth pricing module (src/lib/plans.ts) consumed by all pages so prices are never hardcoded.

Quick Start

npm install
npm run dev

Open http://localhost:3000.

First-time setup — install PABU's Python dependencies:

cd backend/pabu-ai
python -m venv .venv
.venv\Scripts\Activate.ps1  # PowerShell
pip install -r requirements.txt

After that, npm run dev starts both PlotCore and PABU automatically. For production, npm start also starts both services.

To run the PABU engine standalone:

cd backend/pabu-ai
uvicorn api.server:app --reload --port 8000

Set NEXT_PUBLIC_PABU_API=http://localhost:8000 in .env.local to use a different engine URL.

Commands

npm run dev      # Development server (Turbopack)
npm run build    # Production build
npm start        # Production server
npm run lint     # ESLint

Stack

Layer	Technology
Framework	Next.js 16.2.10 (App Router + Turbopack)
UI	React 19.2.4, Radix UI, Framer Motion
Language	TypeScript 5+
Styling	Tailwind CSS 4, CSS custom properties
Charts	Recharts
AI Engine	Python 3.11, FastAPI, OpenAI GPT-4o / Anthropic Claude / Google Gemini / Ollama
IBM AI	IBM watsonx.ai — Granite foundation models (NLU, script analysis, story generation)
IBM Tools	IBM Bob (primary engineering AI collaborator throughout development)
IBM Watson	Watson NLU · Watson Text-to-Speech · Watson Speech-to-Text · Watson Tone Analyser
IBM Cloud	Cloud Object Storage (character portraits, scene assets, voice exports)

Documentation

Full documentation lives in documentation/README.md. The PABU service is documented separately in backend/pabu-ai/README.md.

Name		Name	Last commit message	Last commit date
.vscode		.vscode
backend		backend
documentation		documentation
public		public
scripts		scripts
src		src
.gitignore		.gitignore
AGENTS.md		AGENTS.md
CLAUDE.md		CLAUDE.md
README.md		README.md
eslint.config.mjs		eslint.config.mjs
next.config.ts		next.config.ts
package-lock.json		package-lock.json
package.json		package.json
postcss.config.mjs		postcss.config.mjs
tsconfig.json		tsconfig.json

Navigation Menu

Repository files navigation

PlotCore Studio

Demo Video

Meet PABU — Your AI Creative Director

What PABU Does

How PABU Thinks

Who PABU Serves

PABU in the Interface

Screenshots

Dashboard

Story Ideas Lab

Voices

Analytics & Audience

Plan & Billing

Profile Settings

Notification Settings

Problem Statement

Solution Description

AI Approach and Architecture

Story Memory Engine

PABU AI Director

PABU AI Engine — Python Backend

Production AI Pipeline

Selected Challenge Theme

How IBM Bob Was Used

Quick Start

Commands

Stack

Documentation

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

PlotCore Studio

Demo Video

Meet PABU — Your AI Creative Director

What PABU Does

How PABU Thinks

Who PABU Serves

PABU in the Interface

Screenshots

Dashboard

Story Ideas Lab

Voices

Analytics & Audience

Plan & Billing

Profile Settings

Notification Settings

Problem Statement

Solution Description

AI Approach and Architecture

Story Memory Engine

PABU AI Director

PABU AI Engine — Python Backend

Production AI Pipeline

Selected Challenge Theme

How IBM Bob Was Used

Quick Start

Commands

Stack

Documentation

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages