it wont "scale you down", there is no such thing. a bad result is off course gonna lead to a lower scaled mark for these 2 units. its true for any subject. but if you are asking about that, then just look at the scaling figures for 2006.
math ext 2 has very, very sexy scaling.
if you just get in the top 75% of this subject, your final scaled mark for these 2 units is already pretty high, like 82 if my memory serves me. its around the same as getting top 10% is business studies, and like top 25% in physics or economics.