in the latest build, 2176, there's a bug with regard to holding onto file descriptors of open files on Windows (Windows 7 in my case).
1) Create a directory, say C:\Test
2) Create a document in said directory, say C:\Test\Test.txt
3) Open it in SublimeText 2
4) Close the file with SublimeText 2 and keep SublimeText 2 open
5) Delete the file
6) Go up to C:\
7) Delete the directory and get greeted with a message that there's still a program using this directory.
8) Close SublimeText 2 and delete the directory, which now works
So it seems SublimeText 2 is holding some file descriptor to either the file or directory around.