EvoHIL: Human-in-the-Loop RL That Survives a Lighting Change
Human-in-the-loop robot learning works until the warehouse lights change and the “success” classifier starts lying.
EvoHIL (arXiv:2608.03872, submitted Aug 4, 2026) is a staged framework from Chongqing University / University of Hong Kong collaborators that co-adapts three pieces of the HIL stack: the reward classifier, the action generator, and the visual domain. Evaluation spans six contact-rich tasks on a Franka FR3 and a low-cost SO-101, under controlled illumination shifts.
Three coupled fixes
Self-evolving reward (SER). A visual success classifier updates from human-confirmed positives and provisional weak negatives. Classifier predictions cannot mint positive labels on their own (label-source isolation). A high threshold (default τ = 0.95 for K_conf = 2 frames), held-out gate, and EMA on the deployed weights limit thrash.
Action Flow Stabilization (AFS). Instead of independent per-step Gaussian actions, a flow-matching actor predicts short action chunks; only the executed prefix is scored by an execution-prefix critic. Expert BC-flow and smoothness terms use source-tagged replay (policy vs demo vs intervention).
Retention-aware offline fine-tuning. After online HIL, recorded trajectories are relit (images rewritten, actions/rewards kept) and mixed with source-domain replay (α = 0.75). Frozen anchors on critic values and flow velocities limit forgetting when adapting to new lighting — no extra robot interaction required for that phase.
What they report
Across six tasks and both embodiments, the full pipeline is evaluated with matched demos and real-robot interaction budgets against HIL-SERL, HG-DAgger, BC, IBRL, and ACT. The paper reports improved task success, higher agreement with human-confirmation labels after scene changes, smoother command trajectories, and better completion-time profiles relative to those baselines under predefined illumination conditions. Component studies isolate reward recovery after lighting change, temporal regularity of flow vs Gaussian actors, and source-domain retention when anchors are ablated.
Cameras run at 10 Hz with 128×128 crops. Control is operator-supervised with workspace clipping; any episode with human intervention counts as failure for success-rate scoring.
Project page listed in the paper: https://anonymous4366.github.io/EvoHIL/.
A Human’s Take
I love that they treat “the light changed” as a first-class failure mode instead of a footnote. Contact-rich insertion is exactly where a static visual reward dies quietly and poisons every critic update after it. Flow-chunk actions plus honest intervention tags feel closer to how HIL actually runs on a bench. Next watch: open code, and whether the same retention recipe holds when the shift is not just lighting but a new background or a new bin.