[Bug]: two divergent is_binary detectors in forge_fs — fs_search/fs_write vs read_range disagree on same file

[chirag127]

Problem

forge_fs ships two different is_binary implementations that are invoked from different code paths, so the same file can be classified as binary by one path and text by the other.

Cite

• crates/forge_fs/src/binary_detection.rs:32 — pub is_binary(path): BOM + zero-byte heuristic over first 512 bytes.
• crates/forge_fs/src/is_binary.rs:23 — ForgeFS::is_binary(&mut File): infer crate MIME sniff over first 8192 bytes; infer::get() == None returns is_text=true ("Assume text if type can't be determined").
• crates/forge_fs/src/lib.rs:20 re-exports only the BOM variant as forge_fs::is_binary.
• crates/forge_fs/src/meta.rs:12 ForgeFS::is_binary_file → BOM detector.
• crates/forge_infra/src/fs_meta.rs:14 FileInfoInfra::is_binary → is_binary_file → BOM detector. Used by fs_search.rs:218,257,456, fs_write.rs, plan_create.rs, image_read.rs.
• crates/forge_fs/src/read_range.rs:51 read_range_utf8 → ForgeFS::is_binary (infer variant).

Steps

1. Create a file with a UTF-8 text prefix and a stray 0x00 in the first 512 bytes (e.g. printf 'KEY=value\0trailing text' > /tmp/mixed.txt).
2. fs_search / any FileInfoInfra::is_binary caller: BOM detector sees the zero byte, returns true → file is skipped as binary.
3. read_range_utf8 on same file: infer::get() returns None for unknown bytes, is_text=true, read proceeds and returns lossy UTF-8.

Expected

One canonical detector. All call sites route through the same function so classification is consistent.

Actual

Same file is skipped-as-binary by fs_search but readable via read_range_utf8; user sees content in read responses that search claims doesn't exist.

Environment

• Repo: tailcallhq/forgecode@main (HEAD 2026-07-03)
• Cargo.toml: rust-version = "1.94", edition = "2024", infer = "0.19.0"

Thanks for maintaining tailcallhq/forgecode!

[hiSandog]

Consolidating the detectors would benefit from a shared corpus of ambiguous inputs, including UTF-16 text with NUL bytes, empty files, short binary headers, and mostly textual files containing an isolated NUL. The public operations could then assert the same classification for every fixture, preventing read_range and fs_search from diverging again.