There are a number of things to consider. As Namu said, your rank, difficulty of exams at your school, your overall undersrtanding of the 2 unit maths prelim course, how much work you did etc. If u have a reasonable grasp of areas such as trig and calculus etc, then u should be OK to stick with 2 unit. If u put in the work, then u can get a decent mark in 2 unit. In terms of scaling, you're probably better off in 2 unit than general.
Anyway i dont know u at all so its best u make your own decision. Talk to your teacher and other people in your class to find out how they think your going etc.
Oh and another thing to think about is uni- some courses have 2 unit maths as a prerequisite so dropping to general can limit your options.