Gem Enthusiast movement speed bonus is bugged

I have 10 green supprt gems socketed in one weapon set and I have the movement speed bonus when I log into the game in my hideout. Removing green gems to have less than 10 stopps the bonus, putting them back in enables the buff. Switching weapon sets doesn’t change anything. Works as intended. But when I go into a map, there is no bonus anymore and when I return to my hideout the bonus remains missing.