Here's what it is.
Or remote desktop connection gets messed up and I'm no longer able to get into the server:
Or Chrome starts to randomly crash:
None of these errors happen when the Windows session is fresh, so to work around these issues we implemented a Windows session cycler that simply logs out the user that runs tests, and logs it back in again every 6 hours. The implementation is pretty straight forward and just uses the standard Windows command line tools for managing users, such as users.exe and logoff.exe, as well as rdesktop.exe.
The user cycler is about a 100 line program but here's how it basically works:
user_id = find user id of testling user using users.exe call <logout.exe user_id> wait until user has been logged off login testling user again using rdesktop.exe
The testling user's startup script initializes the testing environment and tests continue running after a short interruption.
A better solution would be to boot a clean Windows instance for every test but we're not there yet. Until next time.