If I were to guess, I would say that changing your / bin (in this case, the change that was removed) launches IIS to recycle your application. app_offline.htm stops this due to changes and deletions of content, but does not delete the bin / bin directories. I would suggest that you see this behavior if you rename the directory rather than delete it.