Speaking of Heisenbugs

Sean caught his. Here's one I ran into. Well, it's not really a heisenbug, but it was annoying.

I had to modify the install program for the software I write to allow the user of a client installation to check that the database connection works. Easy enough. I'm told what program to modify and I do so. It will now try to connect to the database and display a messagebox with a yea or nay. It goes to testing. It gets rejected because it doesn't show up during a client install, only during a server install.
They told me the wrong program. The program I modified only displays during a server install to tell the user what to enter during a client install.

Great.

I revert the code and now have to figure out how to add a custom dialog to Installshield. I don't like Installshield, it feels clumsy and overcomplicated. I much prefer Inno Setup, but we already had these setups built in Installshield before I got here.
So I add the custom dialog and the code backing it up. Now I have to make a DLL to test the connection. That takes about an hour including the app to test it with. I add the DLL to the install and start testing. It works.

Almost.

As long as you already the database client software already installed. Did I mention that the install installs the database client software? I modify the install to copy the database client DLL to the machine if it is not there beforehand. It works.

Almost.

If I type a correct database path, all is well. If I test for failure, the install locks up when trying to unload the DLL. I spent about a day trying variations trying to get it to work. Is it the database client DLL copy? No. Then I noticed that there was another thread created when there was an exception, but not when it connected successfully.
After digging into database component code, I disabled a monitor flag that records database events and status. It was creating a thread to log some information on an exception, but the thread hung around, stopped rather than freed. This would cause it to wait forever for the thread to finish. In an application rather than a DLL this works fine. It works.

Almost.

Now the standalone install is failing. I didn't change any of that code! More time is spent. Then I try building the setup from the main build script instead of from inside Installshield. It works.

Really? Yes, really.

I had modified the blank database and checked it into source control. This marked it read only. That prevented it from working correctly in a standalone install. The build script marks those files writable for exactly this reason.

Ah. Working software.