With commitment and sustained efforts, I am sure OP can still perform well in Physics with only general maths.
1. The amount mathematical/calculation type questions in Physics papers are slowly declining.
2. I've personally seen some 2U students outperform 3U and 4U students in Physics. Even without mathematical understanding, if you're committed to memorising the syllabus, you've already guaranteed yourself quite a good mark. Plus, in my opinion, HSC Physics tests your understanding of the 'physics' behind a physical phenomena rather than the mathematics behind it.
3. Although the 'mathematics' behind Physics was never too troublesome for me, I was never really any good at understanding the 'Physics' behind things compared to many people in my school but I was committed to knowing the syllabus and from that, I have achieved consistent and respectable results all year for Physics - improving from Year 11 (with a mark of 81/100 for trials and an overall mark of 83 for Year 12 internal assessments --> definitely top 20 out of around 97 students.). Just shows that commitment to the syllabus can still help you achieve a good result. Funny thing is that in Year 11, my maths was better and my Physics was worse. Then in Year 12, my maths results/ranks declined yet my Physics results/ranks improved.
But, I'm not the best Physics student out there nor am I as experienced as Fizzy_Cyst or Drongoski so yeah my opinion might not be the best.
I do agree that there is somewhat of a correlation between mathematical ability and physics since the kids in the VERY top end of Physics are really dominant at 4U as well.
But I'm sure 2U and General Maths people have a chance to score a low band 6 in Physics as well.