You won't need to be a mathematician to do psychology but it helps if you're at least somewhat scientifically oriented. With the Bachelor of Psychology at UNSW, you don't have to do any explicit mathematics course (i.e. no courses from the School of Mathematics), but as Iruka said, there will be statistics taught in psychology courses. Maybe the stats won't turn out like the mathematics you dread? It's quite different from the 'pure' mathematics.