So a backup maintenance job, Ola Hallengrens Maintenance Solution, see ola.hallengren.com, was having problems removing old backups, the removal was done with Microsofts own xp_delete_file, the thing is that the instance held over 2000 databases, and after a week or two the backup times went from 10 minutes to 10 hours. Found that the part taking time was deleting old files, so I contacted MS which were to no help really, usually they are, then Ola Hallengren stepped up to the plate, got their attention and finally they released a fix! :-)
See http://support.microsoft.com/kb/2926028 for more info, it seems there was some kind of memory leak they fixed, now my backups are running fine again.
Happy customer = happy DBA consultant!