Skip to content

Optimize Retrieval Pipeline and Parallelize Data Loading#210

Open
d-oit wants to merge 2 commits into
mainfrom
perf/search-pipeline-optimization-5569261335882667502
Open

Optimize Retrieval Pipeline and Parallelize Data Loading#210
d-oit wants to merge 2 commits into
mainfrom
perf/search-pipeline-optimization-5569261335882667502

Conversation

@d-oit
Copy link
Copy Markdown
Owner

@d-oit d-oit commented May 27, 2026

This patch implements multiple performance optimizations in the core data and search pipelines:

  1. Parallel Data Loading: App.tsx now fetches entities and links concurrently during refreshes, reducing I/O wait time.
  2. Orama Search Limits: All search functions now support and enforce limits at the engine level, preventing over-fetching and reducing main-thread blocking.
  3. Zero-Copy UI: Consolidating RankedResult and SearchResult interfaces allows the SearchPanel to consume results directly without re-allocating the entire result set on every progressive stage.
  4. Interface Unification: Standardized on 'title' and 'content' fields across database, search, and UI layers for better consistency.

PR created automatically by Jules for task 5569261335882667502 started by @d-oit

- Parallelize entity and link loading in App.tsx using Promise.all
- Add limit support to searchKnowledge, semanticSearch, and progressiveSearch in search.ts (default: 20)
- Unify RankedResult and SearchResult interfaces with shared title/content properties
- Eliminate redundant result mapping in SearchPanel.tsx and Chat.tsx
- Update unit tests to match unified interface properties

Co-authored-by: d-oit <6849456+d-oit@users.noreply.github.com>
@google-labs-jules
Copy link
Copy Markdown
Contributor

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

@github-actions github-actions Bot added config tests Related to automated/manual tests labels May 27, 2026
@codacy-production
Copy link
Copy Markdown
Contributor

codacy-production Bot commented May 27, 2026

Not up to standards ⛔

🔴 Issues 1 high

Alerts:
⚠ 1 issue (≤ 0 issues of at least minor severity)

Results:
1 new issue

Category Results
ErrorProne 1 high

View in Codacy

🟢 Metrics 8 complexity · 0 duplication

Metric Results
Complexity 8
Duplication 0

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

config tests Related to automated/manual tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant