Eldritch Empowerment Recharge Interruption

I previously had both Eldritch Empowerment and Event Horizon allocated, and my energy shield recharge was interrupted by casting spells, as expected from the ES loss from Event Horizon. I then removed Event Horizon and kept Eldritch Empowerment. Now the first cast of a spell does not seem to interrupt ES recharge, but subsequent casts do. However, recharge can still occur while casting, and isn't interrupted by casting until after the ES reaches full.

Steps to see effect: repeatedly cast any spell in Hideout, watch ES drop, then recharge, all while spamming cast.

OP here, forgot to add that removing Eldritch Empowerment removes all ES loss, so there are no other effects on my character that cause ES loss besides Eldritch Empowerment.