Trajectory Supervision Enhances Continual Tool-Use Learning in LLMs
A new research paper submitted to arXiv investigates the impact of retaining tool-use trajectories during the continual learning process of Large Language Models (LLMs). The study addresses a common limitation in training data, which typically presents final artifacts rather than the intermediate processes used to create them. Researchers fine-tuned the Llama 3.1 8B Instruct model using QLoRA on the API-Bank dataset, processing four sequential domain blocks. They compared two conditions: one where previous API request and response lines were stripped from the prompt, and another where the full trajectory context was preserved. Results from a single-seed pilot evaluation indicated that preserving trajectory context significantly improved performance. The model achieved 56.9% exact full-call accuracy with trajectory supervision, compared to 39.2% without it, alongside a 7.7-point increase in API-name accuracy. However, this improvement comes with a 25.1% increase in training token usage. The findings suggest that maintaining intermediate steps aids in next-call prediction tasks, though further research is needed to assess full dialogue success and robustness across multiple seeds.
Wire timeline
Trajectory Supervision Enhances Continual Tool-Use Learning in LLMs
A new research paper submitted to arXiv investigates the impact of retaining tool-use trajectories during the continual learning process of Large Language Models (LLMs). The study addresses a common limitation in training data, which typically presents final artifacts rather than the intermediate processes used to create them. Researchers fine-tuned the Llama 3.1 8B Instruct model using QLoRA on the API-Bank dataset, processing four sequential domain blocks. They compared two conditions: one where previous API request and response lines were stripped from the prompt, and another where the full trajectory context was preserved. Results from a single-seed pilot evaluation indicated that preserving trajectory context significantly improved performance. The model achieved 56.9% exact full-call accuracy with trajectory supervision, compared to 39.2% without it, alongside a 7.7-point increase in API-name accuracy. However, this improvement comes with a 25.1% increase in training token usage. The findings suggest that maintaining intermediate steps aids in next-call prediction tasks, though further research is needed to assess full dialogue success and robustness across multiple seeds.
cs.AI updates on arXiv.org