If it reacts with a hydronium ion, it indeed then acts as a base, but that's not the only thing that's occurring. You thus cannot say that it's acting as a base only.
Plus, the most probable candidate for the cause of acidic taste is hydronium ions.
It's known that bicarbonate ions do not...