C++/Win32: How to wait for a pending delete to complete

Question

Solved:

Workable solution: sbi's answer
Explanation for what really happens: Hans's answer
Explanation for why OpenFile doesn't pass through "DELETE PENDING": Benjamin's answer

The Problem:

Our software is in large part an interpreter engine for a proprietary scripting language. That scripting language has the ability to create a file, process it, and then delete the file. These are all separate operations, and no file handles are kept open in between these operations.

(I.e. during the file creation, a handle is created, used for writing, then closed. During the file processing portion, a separate file handle opens the file, reads from it, and is closed at EOF. And finally, delete uses ::DeleteFile which only has use of a filename, not a file handle at all).

Recently we've come to realize that a particular macro (script) fails sometimes to be able to create the file at some random subsequent time (i.e. it succeeds during the first hundred iterations of "create, process, delete", but when it comes back to creating it a hundred and first time, Windows replies "Access Denied").

Looking deeper into the issue, I have written a very simple program that loops over something like this:

while (true) {
    HANDLE hFile = CreateFileA(pszFilename, FILE_ALL_ACCESS, FILE_SHARE_READ,
                               NULL, CREATE_NEW, FILE_ATTRIBUTE_NORMAL, NULL);
    if (hFile == INVALID_HANDLE_VALUE)
        return OpenFailed;

    const DWORD dwWrite = strlen(pszFilename);
    DWORD dwWritten;

    if (!WriteFile(hFile, pszFilename, dwWrite, &dwWritten, NULL) || dwWritten != dwWrite)
        return WriteFailed;

    if (!CloseHandle(hFile))
        return CloseFailed;

    if (!DeleteFileA(pszFilename))
        return DeleteFailed;
}

As you can see, this is direct to the Win32 API and is pretty darn simple. I create a file, write to it, close the handle, delete it, rinse, repeat...

But somewhere along the line, I'll get an Access Denied (5) error during the CreateFile() call. Looking at sysinternal's ProcessMonitor, I can see that the underlying issue is that there is a pending delete on the file while I'm trying to create it again.

Questions:

Is there a way to wait for the delete to complete?
Is there a way to detect that a file is pending deletion?

We have tried the first option, by simply WaitForSingleObject() on the HFILE. But the HFILE is always closed before the WaitForSingleObject executes, and so WaitForSingleObject always returns WAIT_FAILED. Clearly, trying to wait for the closed handle doesn't work.

I could wait on a change notification for the folder that the file exists in. However, that seems like an extremely overhead-intensive kludge to what is a problem only occasionally (to wit: in my tests on my Windows 7 x64 E6600 PC it typically fails on iteration 12000+ -- on other machines, it can happen as soon as iteration 7 or 15 or 56 or never).

I have been unable to discern any CreateFile() arguments that would explicitly allow for this ether. No matter what arguments CreateFile has, it really is not okay with opening a file for any access when the file is pending deletion.

And since I can see this behavior on both an Windows XP box and on an x64 Windows 7 box, I am quite certain that this is core NTFS behavior "as intended" by Microsoft. So I need a solution that allows the OS to complete the delete before I attempt to proceed, preferably without tying up CPU cycles needlessly, and without the extreme overhead of watching the folder that this file is in (if possible).

1 Yes, this loop returns on a failure to write or a failure to close which leaks, but since this is a simple console test application, the application itself exits, and Windows guarantees that all handles are closed by the OS when a process completes. So no leaks exist here.

bool DeleteFileNowA(const char * pszFilename)
{
    // Determine the path in which to store the temp filename
    char szPath[MAX_PATH];
    strcpy(szPath, pszFilename);
    PathRemoveFileSpecA(szPath);

    // Generate a guaranteed to be unique temporary filename to house the pending delete
    char szTempName[MAX_PATH];
    if (!GetTempFileNameA(szPath, ".xX", 0, szTempName))
        return false;

    // Move the real file to the dummy filename
    if (!MoveFileExA(pszFilename, szTempName, MOVEFILE_REPLACE_EXISTING))
        return false;

    // Queue the deletion (the OS will delete it when all handles (ours or other processes) close)
    if (!DeleteFileA(szTempName))
        return false;

    return true;
}

You sure all handles are closed? Because what you wrote is exactly described on MSDN: "If you call CreateFile on a file that is pending deletion as a result of a previous call to DeleteFile, the function fails. The operating system delays file deletion until all handles to the file are closed. GetLastError returns ERROR_ACCESS_DENIED." — Kra
I have created a console app that does the above code in a loop. It fails eventually... usually around iteration 12000-15000. So unless you see some way that the above 8 lines of code leaks a handle, I think it's quite impossible (at least for this test applet). — Mordachai
I had this issue caused by the Windows Search service, it would noticed I made a directory and lock it to index it, at the same time I attempted to delete it and got error 5. Its easy to reproduce this by creating and deleting dirs/files in a loop with the indexer enabled. — paulm
It was MsMpEng in my case, which blocked the file. I have discovered MS confirmation that index/antivirus SW can block the file. Now, I therefore accustomed to delete in retry loop or better create new temp file instead of reusing single name, with a hope that single delete will be effective ultimately. Now, I have got a similar issue. Gcc p2 fails in gcc p.c > p.exe && p && gcc p2.c > p.exe && p ... because just-terminated p.exe is blocked and nothing in filemon but p process accesses the file. — Val

Hans Passant Hans Passant · Accepted Answer · 2010-09-21T20:43:10

There are other processes in Windows that want a piece of that file. The search indexer is an obvious candidate. Or a virus scanner. They'll open the file for full sharing, including FILE_SHARE_DELETE, so that other processes aren't heavily affected by them opening the file.

That usually works out well, unless you create/write/delete at a high rate. The delete will succeed but the file cannot disappear from the file system until the last handle to it got closed. The handle held by, say, the search indexer. Any program that tries to open that pending-delete file will be slapped by error 5.

This is otherwise a generic problem on a multitasking operating system, you cannot know what other process might want to mess with your files. Your usage pattern seems unusual, review that first. A workaround would be to catch the error, sleep and try again. Or moving the file into the recycle bin with SHFileOperation().

C++/Win32: How to wait for a pending delete to complete

13 Answers