Can you pay any type of bill, and not just a utility bill, in person?
The only way you would be able to pay a certain bill at the bank is if that bill states that you may do so. Check the disclosures on the back of the invoice for additional information.