Getting kicked for inactivity LE-100 while allocating skill points

I’m getting constantly kicked for inactivity while I’m rearranging my stash, allocating my skill points, and rerolling for my prophecies.

I don’t see what’s wrong when I’m actually playing the game and clicking my mouse/pressing buttons from my controller and I get kicked for “inactivity”.

Last time I got kicked looking for exalted items and forging them to slam into my LP uniques after I killed the boss and got kicked, losing my chance to merge.

I hope this is not intended somehow. Please fix it soon.