The Prisoner fight just feels wrong

He has like 3 seconds to use his abilities in the last phase, before you can hit him with a spear again.

For once, I spaced out and didn't hit him with the spear. Turned into a real fight with mechanics. Surely this is a bug? He's such a pushover.