Abstract
<jats:p>Human feedback is a key mechanism for aligning large language models (LLMs), yet it appears in complex forms that vary in expressive-ness, granularity, and cost. While recent work has explored richer feedback beyond binary preferences, the literature remains fragmented, and the relationships between feedback structure , optimization methods, and evaluation protocols are often unclear. This survey provides a feedback-centric overview of LLM alignment. We propose a taxonomy that organizes complex feedback by supervision form and orthogonal dimensions, use it to categorize alignment methods into reward-model-free and reward-model-based approaches, and survey evaluation practices, distinguishing direct verification from proxy measures derived from human feedback. By synthesizing results across feedback modalities, optimization strategies, and evaluation designs, we clarify their interdependen-cies, identify recurring mismatches in current pipelines, and highlight underexplored settings for future feedback-driven alignment research.</jats:p>