The bay is seriously not that bad as a whole, The 'bay rats' or whatever are not ever a threat to anyone unless you're alone in the middle of nelson bay CBD at 1am (i.e not where you'd usually hang)
If you're not 18 and don't have many people comming here with you, i wouldn't recommend it...