When you use the town portal (T) to return to the Outcast Town in Imperial Era: 1005 AG (Area level 23) it places you on the far east side of town just south of the forge (just east of Ansella behind the breakable pots). It should place you on the town portal pad in the center of town.
This is repeatable every time you use a town portal (T) to return to The Outcast Town. I have used the town portal (T) from each map location around the Outcast Camp. Every time it returns me to the east side of town.
If you use the map to return to The Outcast Town it places you on the Town Portal pad.