Masteries are about making each class feel and play differently. If you could use absolutely every skill from every mastery, then no matter which mastery you picked, that class would play the same. Then every class would basically narrow down to the best five-skill combination it has. Not only would you know for a fact that’s how that class should be played, but you’d know you’re hamstringing yourself for not playing it. So would other people, who might choose not to play with you based on that.
To put it differently, skill restrictions by mastery don’t exist to limit build variety, but to create a larger total number of optimal builds across the same number of skills. Instead of having 1 or 2 per class, you have a minimum of 3 per class. This means there’s at least that much more to do that is considered valid, as long as each mastery is about as useful as the others for that class.
Masteries also exist to create a distinct gameplay experience that is different from one you would have with a different mastery. As long as each mastery gets to specialize in doing something a little different than the others, then it can be considered a unique way to play the game. Rather than many more builds with all the skills that become irrelevant, each one can be tweaked so that it gets something only it can do. And you don’t have to worry about balancing the specific things that mastery can do against every other skill you don’t have access to in the other masteries. This means each mastery can get something that is ridiculous and interesting without it breaking or invalidating what other masteries get that are exclusive to them.
Imagine in the extreme that you could use every skill of every class on the same character. Well then, every single character regardless of class would fundamentally play the same and do the same things. Not only would it be impossible to balance all the skills against each other such that changing one or making one strong without making several others pointless or less useful would be impossible, but it would also be a nightmare for players to keep up with unless they stuck to what the community identified as the meta for each patch. Moreover, the incentives to avoid that complexity and stick to the meta would increase exponentially, because for all you know, you’re going to pick a bunch of skills that are useless and waste your time. Might as well get the good ones, with some small variation at most.
In reality, you’ll find that complexity does actually emerge from restrictions in the exact same way. If you just think in terms of running, if you can run anywhere at any time for no reason, you probably will never just set off running. But if you set a finish line, then you have a race. This same principle applies to every game: I know this is a philosophical point, but if you think about the rules of other games, you’ll realize that the rules themselves are restrictions. But those restrictions create more interesting ways to act in the world, because they create goals, as well as a pattern for acting out the game across multiple iterations.
All of that to say that if the developers didn’t have the option to make some abilities exclusive to certain specs, they actually probably couldn’t make the game as complex and interesting as it is.