There is a band of thieves who bully the tunnels near the factory.
The biggest spawn is here.
There are three major Glooth bandit spawns, west, south and east. Both west and east are accessible as soon as you reach the 300 points needed to enter the Glooth factory. The south spawn requires the mission "Glooth Bandits" from the Oramond Quest in order to reach the full spawn.