One of the most crucial skills required in Extension 2 Maths which separates the best from the rest, is prediction.
Basically you can mentally trial and error a method in your head and see if it works. If you did integration by parts, what would you end up with? If you tried a substitution what would you end up with? etc
For example, if you were to integrate x.ln x using integration by parts. It should intuitively occur to you that the expression you have to choose to integrate is x, because if you chose ln x instead, you would know that you can't integrate ln x without getting some complicated formula.
In terms of reduction formulae, say we had to integrate cos<sup>n</sup>x.
So lets say we jump straight to integration by parts. So if we think about doing that we have to integrate dx and differentiate cos<sup>n</sup>x.
BUT if we do that, we'll get x times a power of sin x and cos x, which makes it more complicated, so we discount that.
So now, we need to inegrate something that doesn't give us another x, but will give us a trig function so everything remains as a trig function which we can simplify later.
Therefore the only simple way we can integrate something and give ourselves a trig function is if we take cos x to integrate. This will simply give us sin x, and when we differentiate cos<sup>n - 1</sup>x, we'll also get a sin x somewhere. When we multiply the two, we'll get something with a sin<sup>2</sup>x, which we can easily convert in terms of cos x expressions and when you actually do it, everything simplifies. Hence this method is the way to go.
So it should be a matter of prediction and often intuition...