LLM POST-TRAINING · TOOL USE
Training on Broken Contexts: Source Loss and Repair in Tool-Augmented LLM Post-Training
Does a supervised answer still have access to the tool output that produced it?
Auditing final tokenized sequences reveals targets that survive in the loss after their supporting tool observations disappear from context. Source-aware reconstruction keeps targets with the evidence they use.
A valid stored conversation can become an unsupported training example after templating, masking, truncation, or packing. Counting loss tokens alone cannot reveal this failure.
Approach
Trace rendered tokens to their conversations and verify source visibility within each target’s attention group. Reconstruct complete target–source groups and retain only the tool schema content needed by the supervised call.
Main result
At 512 tokens, BFD-split separated 38.66% of retained matched ToolACE targets from their recorded sources. Grouped reconstruction retained 875/918 matched targets (95.32%) and eliminated 1,244 duplicated source tokens relative to one-example-per-target reconstruction. Implementation checks also identified a multi-batch token-stream defect in TRL 1.9.2 wrapped packing.
Retaining targets with fewer input tokens
