[Bug]: Codex Step B3 can overwrite in-memory semantic results with an empty disk merge

[dwchoo]

What happened?

The Codex skill has incompatible result-collection instructions between Step B2 and the shared Step B3. This affects host-agent semantic extraction for documents, papers, and images.

Step B2 instructs the parent to collect returned JSON in memory and write graphify-out/.graphify_semantic_new.json. It explicitly says there are no per-chunk files and that Step B3's disk-based success checks do not apply.

However, Step B3 still:

1. Requires .graphify_chunk_NN.json files as the success signal and recommends a Claude-specific general-purpose agent when they are missing.
2. Unconditionally runs a disk-based merge over .graphify_chunk_*.json, then overwrites .graphify_semantic_new.json.

Consequently, following B3's checks can incorrectly classify successful Codex results as failures. Skipping only those checks, as B2 instructs, is insufficient: running the subsequent merge with no chunk files overwrites the collected results with empty arrays.

Expected: Codex follows the same file-based semantic extraction pipeline as Claude. Each write-capable subagent writes its extraction JSON directly to the assigned absolute CHUNK_PATH (.graphify_chunk_NN.json), and the parent uses the shared Step B3 to validate, collect, cache, and merge those files without losing results. B2 and B3 must agree on this contract.

Source references

Confirmed against current v8 commit 4c735618f3d56fd622c2049771584621c31ba9ff:

• Codex B2 generator fragment
• Shared B3 generator fragment
• Generated Codex skill
• Existing Codex regression test checks the memory-collection wording and B2 dispatch, but does not verify compatibility with B3.

Steps to reproduce

From a checkout of the commit above, run the following standard-library-only script. It seeds a minimal B2 output and executes the actual B3 merge snippet from the generated skill in a temporary directory. It does not run an LLM or modify the repository.

python - <<'PY'
import json
import os
import tempfile
from pathlib import Path

skill = Path('graphify/skill-codex.md').read_text()
b3 = skill.split('**Step B3 - Collect, cache, and merge**', 1)[1]
merge_code = b3.split('import json, glob\n', 1)[1].split('\n"\n```', 1)[0]
merge_code = 'import json, glob\n' + merge_code.replace('\\"', '"')
with tempfile.TemporaryDirectory(prefix='graphify-codex-repro-') as root:
    os.chdir(root)
    Path('graphify-out').mkdir()
    target = Path('graphify-out/.graphify_semantic_new.json')
    target.write_text(json.dumps({
        'nodes': [{'id': 'doc_example', 'label': 'Example'}],
        'edges': [], 'hyperedges': [],
        'input_tokens': 0, 'output_tokens': 0,
    }))
    print('Before B3:', len(json.loads(target.read_text())['nodes']), 'node(s)')
    exec(compile(merge_code, '<actual Step B3 merge snippet>', 'exec'))
    after = json.loads(target.read_text())
    print('After B3:', len(after['nodes']), 'node(s)')
    assert after['nodes'] == []
PY

Observed output

Before B3: 1 node(s)
Merged 0 chunks: 0 in / 0 out tokens
After B3: 0 node(s)

This is a deterministic reproduction of the merge-snippet data loss, not a claim that every Codex run follows the conflicting instructions in the same way. An agent may compensate for the contradiction, but the documented pipeline should not require that inference.

Version and environment

• Graphify: 0.9.67, current v8 commit linked above.
• Host: Codex; reproduction executed on Linux with Python, using only the standard library.
• Scope: host-agent semantic extraction. The reproduction does not exercise code-only AST extraction or the direct LLM backend path.

Related work and proposed fix

• fix(skills): remove invalid OpenCode/Codex agent guidance (#825) #804 is related prior work: its description mentions normalizing returned Codex JSON into chunk files. It was closed without a GitHub merge; the conflicting instructions are still present in current v8.
• skill-codex.md: There is no wait tool in the available tool definitions, instead there is wait_agent. #273 concerns the wait / wait_agent tool name, a separate issue.
• fix(skills): use disk extraction prompts for write-capable hosts #2746 changes extraction prompts for Claw/Kiro/Pi while retaining Codex's inline-return contract.
• Fix token accounting observability for skill-based extraction #3681 concerns token-usage observability; any change here should avoid unnecessarily overlapping that work.

Updated proposal: align Codex with Claude's existing file-based pipeline rather than introduce a separate in-memory collection path.

• Update Codex Step B2 to dispatch write-capable subagents with an absolute CHUNK_PATH for each chunk. Each subagent writes its JSON directly to that file; the parent waits for completion and proceeds to the shared Step B3.
• Use the same extraction prompt source as Claude, including the extraction rules, JSON schema, and CHUNK_PATH output contract.
• Keep the shared B3 collection/cache/merge code and Part C AST/semantic merge. Adapt only platform-specific tool invocation and recovery wording to the tools available in each host; do not create a Codex-only collection or merge implementation.
• Update the generator sources and regenerate the affected skill artifacts and snapshots.
• Replace regression assertions that require Codex's in-memory/compact behavior with checks for shared extraction instructions, direct chunk-file output, and identical collection/cache/merge behavior for the same chunk fixtures. Include a small live Codex subagent check that confirms the assigned chunk files are created and consumed by the shared pipeline.

This supersedes the original proposal to preserve in-memory collection. The reported B2/B3 inconsistency and reproduction remain applicable to the linked affected revision. The reproduction demonstrates the old contract mismatch; the fix should prevent it by making B2 produce the chunk files B3 expects. Token-usage observability remains separate work tracked by #3681.