I'm not one of those people who looks down upon other subjects, although many pure mathematicians DO look down on other sciences a bit. This is often on the basis that what constitutes proof in the sciences pales in comparison with the rigour of mathematical proof. A physicist/chemist/etc never *knows* his theory is correct, in some sense each advance in theory is simply a closer approximation to reality.
Which you prefer largely depends on your personality/mind, as I prefer the clean pure abstraction of mathematics to the sciences, yet I also prefer sex to masturbation
. Feynmann's quote was just showing us his slant, (in a not entirely respectful way I feel, but I am generally a fan of the man.)