Quick AnswerIf you pay quarterly estimated taxes late, the IRS may charge an underpayment penalty based on the amount you should have paid, when it was due, and how long it stayed unpaid. Paying late is still usually better than waiting until you file, but it...