Are there any penalties for delivering accounts late?

Yes, and they vary depending how late your accounts are – from £100 for less than 3 months to £1000 for more than 12 months. A word of warning, they are not negotiable!