When you “just” use it, maybe.
But when this interaction would be possible, you can make it compeltely broken.
Having 2 skills with good singletarget hitting at once, while you only need to scale one of their attack rate stats (in this case attack speed), will lead to some ridiculous singletarget damage if you build around it.
You can put any elemental melee damage on your weapon and scale tons of this elemental damage (most likely fire or lighting, depending on what you want your smite damage to be).
So again, if you don’t build around it, having a negative effect on such a node would not affect you, when you don’t build around it anyway, but it would reduce the top end potential for some edge cases.