Features 1-5 + Multiset VPS spatial integration (FieldMatrix.Ai demo)#53
Open
850Media wants to merge 1 commit intoIntent-Lab:mainfrom
Open
Features 1-5 + Multiset VPS spatial integration (FieldMatrix.Ai demo)#53850Media wants to merge 1 commit intoIntent-Lab:mainfrom
850Media wants to merge 1 commit intoIntent-Lab:mainfrom
Conversation
Features 1-5 (contextual memory, voice-driven reports, knowledge lookup, multi-worker collaboration, predictive safety alerts): - SessionContext model: worker/job/site/GPS, notes, photos - Gemini tool declarations: save_photo, save_note, generate_report, knowledge_lookup, start/stop_inspection, start/stop_safety_monitor - ToolCallRouter with local handlers for each tool - LocationService (CoreLocation + reverse geocoding) - ReportGenerator (UIGraphicsPDFRenderer) with embedded photos and categorized notes (hazard/action/observation/measurement/reference) - WebSearchService (DuckDuckGo Instant Answer API fallback) - WebRTC collaboration: simultaneous Gemini+WebRTC mode, transcript broadcasting via signaling relay, CollaborativeBadge - Inspection + Safety timers with badges and auto-start settings - FieldMatrix.Ai branding on NonStream + Stream overlays, new app icon Multiset VPS spatial integration: - SpatialPose + SpatialLocalizationService with provider pattern (Multiset → Google Geospatial → GPS fallback) - #if canImport(MultiSetSDK) guards so build works today with GPS-only; Multiset activates automatically once xcframework is dropped in - Settings section for Client ID/Secret/Map Code with secure storage - Pose attached to every saved note and photo; included in PDF reports with accuracy label (±5cm VPS / ±1m GEO / ±5-10m GPS) - SpatialBadge on streaming overlay shows active source - Secrets.swift scaffold (gitignored) holds Multiset credentials Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This branch builds VisionClaw out from the base DAT SDK sample into a full AI field-worker assistant (FieldMatrix.Ai demo), then adds scaffolded spatial positioning via Multiset VPS.
All 5 roadmap features shipped
SessionContext,LocationService, Field Worker settings (worker/job/site/GPS injected into Gemini system prompt at session start)ReportGenerator(PDFKit),generate_reporttool, embedded photos, categorized notes (hazard/action/observation/measurement/reference/general), iOS share sheetWebSearchService(DuckDuckGo Instant Answer API),knowledge_lookuptool with OpenClaw fallbackCollaborativeBadgestart_safety_monitor/stop_safety_monitortools with periodic[SAFETY CHECK]prompt injection, auto-start setting,SafetyBadgePlus supporting work:
save_photo,save_note,start_inspection/stop_inspection,InspectionBadge,BrandingBanner(FieldMatrix.Ai branding), new app icon.Multiset VPS spatial integration (scaffolded)
SpatialPosemodel +SpatialLocalizationServicewith provider pattern: Multiset → Google Geospatial (future) → GPS fallback#if canImport(MultiSetSDK)guards — build works today with GPS-only; Multiset activates automatically when xcframework is dropped into XcodeSecureFieldstorage±5 cm VPS/±1 m GEO/±5-10 m GPS) for each noteSpatialBadgeon streaming overlay dynamically shows active providerWhat's not in this PR
Secrets.swift(gitignored) with real credentials stays on local machinesMultiSetSDK.xcframework— must be downloaded from developer.multiset.ai and drag-dropped into Xcode per Multiset's install instructions (no SPM support)Files Changed
20 files, +904 / -29
Utilities/SessionContext.swift,Utilities/LocationService.swift,Utilities/ReportGenerator.swift,Utilities/WebSearchService.swift,Utilities/SpatialPose.swift,Utilities/SpatialLocalizationService.swift, newAppIcon.pngimagine_a_film_camera_in_the_style.jpegicon with new FieldMatrix iconTest plan
MultiSetSDK.xcframeworkpresent (verifies guards work)🤖 Generated with Claude Code