If it’s what I’ve been having, try moving your mouse around. Seems to happen especially after I’ve clicked through an NPC dialogue. I can run around and instant skills still work but nothing else does. Best guess, some mouse interaction glitches the transition back to controller and retransitioning fixes it. Cursor position may also play a role.
I’m having exactly same issue mate. I’m on Windows 10 and Xbox controller too. Skills randomly not working I can see skill is clicked in the UI but character doesn’t perform it. I can’t use a skills on keyboard too when this happens. Sometimes reloading location helps but not always and I have to restart the game.