If you don’t pick up the shard in the last phase of “Saving Last Refuge”, where you have to kill the boss, you can run past it and continue on. This included entering the Temple of Eterra as well. The issue with entering the temple is that, if you do it without having the shard, you can’t kill the guardian outside it to complete the main quest phase for that, as once you have entered the temple, it can’t aggro you.
Completing side quests in the areas after The Lower District, without picking up the shard, also makes the quests complete without giving you the rewards. I finished a quest that was supposed to give me 2 passive points, however after turning in the quest, the NPC played the dialogue for completing it, and it shows up completed in the Quest Journal, however i didn’t recieve the XP/Gold or the passive points.
An easy fix for this is to make it so that you can’t leave the area without picking up the shard first.