On the way down, the velocity is negative (as you define upwards as positive). In this case, an 'resistive force' of 0.1v is still negative i.e. in the same direction as the velocity. So your resistive force is pushing the mass down when it should be pushing it up. You should have a = -g - 0.1v...