Same thing with companions. you have to swap them out and back again to gain their buffs.