Automatic Hint Generation and Evaluation for Reasoning Questions

<title>Abstract</title> <p>We study fine-tuning large language models for automatic hint generation in reasoning tasks. Starting from five diverse question–answer datasets, we generate hints using four prompt-ing strategies: answer-aware, answer-agnostic, decomposed answer-aware, and decomposed answer-agnostic. This process is applied to three models: LLaMA-3.1-70B, Gemma-3n-E4B, and DeepSeek-V3.1. We evaluate hint quality using HintEval metrics capturing con-vergence, answer leakage, familiarity, and rel-evance, and propose a metric-guided dataset merging strategy for fine-tuning. Experimental results show that fine-tuning yields modest but consistent improvements in convergence and familiarity while maintaining high relevance and low answer leakage.</p>