Skip to content

GH-35460: Introduce simdjson and migrate ObjectParser#50469

Draft
Reranko05 wants to merge 3 commits into
apache:mainfrom
Reranko05:gh-35460-simdjson
Draft

GH-35460: Introduce simdjson and migrate ObjectParser#50469
Reranko05 wants to merge 3 commits into
apache:mainfrom
Reranko05:gh-35460-simdjson

Conversation

@Reranko05

@Reranko05 Reranko05 commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Rationale for this change

This PR is the first in a series of PRs for GH-35460 to migrate Arrow's JSON implementation from RapidJSON to simdjson.

This PR introduces simdjson into the build system and migrates the self-contained arrow::json::internal::ObjectParser as an initial step.

What changes are included in this PR?

This PR:

  • Introduces simdjson as a third-party dependency.
  • Adds CMake support for discovering and building simdjson.
  • Links simdjson into the Arrow JSON library.
  • Migrates arrow::json::internal::ObjectParser from RapidJSON to the simdjson DOM API.
  • Adds unit tests covering:
    • String retrieval
    • Boolean retrieval
    • String map extraction
    • Invalid JSON
    • Missing keys
    • Incorrect types
    • Non-object root documents
    • Empty objects

This is intended to be the first PR in a series that will incrementally migrate the remaining JSON components.

Are these changes tested?

Yes.

Added dedicated unit tests for ObjectParser and verified that the JSON test suite passes:

Related #35460

@github-actions github-actions Bot added the awaiting review Awaiting review label Jul 10, 2026
@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #35460 has been automatically assigned in GitHub to PR creator.

@Reranko05

Copy link
Copy Markdown
Contributor Author

@kou As we discussed, I've started working on the RapidJSON → simdjson migration and will try to split it into small, reviewable PRs. This PR is the first step, introducing the dependency and migrating ObjectParser.

I'm currently investigating the remaining CI failures. The implementation builds locally, the JSON test suite passes, and several CI workflows also pass. The remaining failures all seem to fail during CMake configure because simdjson::simdjson isn't available in some configurations, while other workflows successfully fall back to building simdjson from source.

I've narrowed it down to the dependency integration (FindsimdjsonAlt.cmake / CMake target creation). If there's an existing Arrow pattern you'd recommend for handling this kind of third-party dependency integration, I'd really appreciate your guidance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant