Prevent velocity re-direct when kicked for being AFK

I have been trying to work out why this is happening (or how I could prevent it). I have three Fabric servers running with Velocity. AFKPlus on the Fabric server does a kick after 30min of inactivity. However, if you're not on the lobby server you are kicked to the lobby server. If you are connected, you're kicked from the network. How can I prevent kicks from just moving you to the main server?

Velocity 3.4
Fabric 1.21.1
AFKPlus 1.7.8