Public borrowing rose to £10bn in April, a record for the month, according to data released by the Office for National Statistics (ONS).
The figure - up from last year's £8.8bn - has underlined the size of the task facing the new government and Chancellor George Osborne.
The ONS figures also revealed that the amount borrowed by the government last year had been revised down from £163bn to £156bn.
The new government are planning to make savings of £6bn over the next year and further deficit reduction plans will be announced in June's emergency Budget.