In 5c boundary_reached is sticky, while finished environments immediately reset and continue. When the final environment reaches its boundary, others may already be mid-game. The scheduler then loads new opponent weights without resetting environments or RNN state.
Games can therefore start against one checkpoint and finish against another, corrupting self-play trajectories and win-rate metrics.
Problematic code: swap logic
Expected: reset affected environments and RNN state when swapping opponents.
In
5cboundary_reachedis sticky, while finished environments immediately reset and continue. When the final environment reaches its boundary, others may already be mid-game. The scheduler then loads new opponent weights without resetting environments or RNN state.Games can therefore start against one checkpoint and finish against another, corrupting self-play trajectories and win-rate metrics.
Problematic code: swap logic
Expected: reset affected environments and RNN state when swapping opponents.