What has math to do with it?
It is a design choice.
Respeccing already was made easier way back in 0.7.X with skill catchup exp and minimum respec levels, but the devs wanted to keep the core system of losing some skill levels when switching skills.