New method enhances LLM performance by targeting single-sample decoding failures

VERA Brief

AI-generated. Grounded in the article and its cited sources.

Researchers have developed GapFT, a fine-tuning technique to improve large language models' single-sample decoding performance. The method targets problems the model fails to solve initially but can resolve with multiple samples, leading to increased accuracy on specific datasets.

Key facts

• GapFT is a fine-tuning technique designed to enhance large language models' single-sample decoding capabilities.
• The method addresses issues where models fail to solve problems on the first attempt but can succeed within a set number of samples.
• GapFT focuses on selecting training evidence based on the model's single-sample outcome and fine-tuning on the difference between Pass@K and Pass@1 scores.
• Improvements were demonstrated on LogiQA 2.0 and ReClor datasets using Llama-3.1-8B.
• The research aims to improve AI systems' accuracy in providing correct outputs on the first attempt for reliable deployment.