Boundary-Gate Collaborative Enhancement GeoVis-GNN for Joint Segmentation and Label Recognition of Video Human-Object Interactions

<title>Abstract</title> <p>Joint temporal segmentation and label recognition in video human--object interaction (VHOI) aims to partition entity interactions along the video timeline and identify the interaction label of each segment, serving as an important task in dynamic interaction understanding. GeoVis-GNN integrates geometric--visual features and interaction-entity graph modeling with a learnable main gate that organizes frame-level representations into temporal segments. However, weak boundaries, gradual transitions, and short-termdisturbances may yield insufficient, shifted, or spurious gate responses. Because the discretized gate directly controls segment-level state updates, these errors can cause boundary shifts, erroneous short segments, and local label discontinuities, degrading segment-structure stability.To address this problem, we propose Boundary-Gate Collaborative Enhancement GeoVis-GNN (BGCE-GeoVis-GNN). The Auxiliary Boundary Enhancement and Temporal Smoothing module (ABETS) suppresses short-term fluctuations and learns multi-scale auxiliary boundary confidence. The Boundary-Gate Collaborative Constraint mechanism (BGCC) imposes two training-time constraints: one aligns the main-gate response with auxiliary confidence before discretization, while the other promotes intra-segment compactness and separation between adjacent segments. This boundary guidance strengthens the original gate-controlledprocess without replacing the main gate. During inference, ground-truth boundaries and collaborative losses are not used, and predictions follow the original main path.On MPHOI-72, BGCE-GeoVis-GNN improves F1@50 by 3.9 percentage points over the reproduced baseline. On CAD-120, it improves four of six metrics, including object-affordance gains of 0.9 and 1.3 points at F1@10 and F1@50. These results demonstrate improved segment-structure stability and strict temporal matchingacross VHOI settings.</p>