Allocating Key Stones to different weapon sets

I know that in an interview there was an explanation of why they dont let players allocate Key Stones to different weapon sets, but it really takes the fun out of some builds.

I get it, there are Key Stones that were completely broken when you can turn them on and off by switching weapon. But there is also a downside to not allowing this.

If you play a different weapon set for mapping and for bosses, sometimes you want different Key Stones. Right now that means: every time i want to do a boss i have to go back to town, refund my key stone passive, fight the boss and then reallocate it after the fight so i can go mapping again.

I mean, there has to be some sort of solution for this without breaking the game...