Here are some general steps that will get you on your way:
First, you must change your compiler's settings so that it creates PDB files, even for release builds. Later versions of the Visual C++ compiler do this by default, but in many versions of Visual C++ you must do this yourself. Create program database files, and then keep an archive of those files along with each build of your application. It is critical that every build of your applications has its own set of PDBs. You can't just reuse the same ones you made with build 10 to examining the dumps generated by build 15, for example. Over the life of your project, you will end up with a ton of PDBs, so be prepared for that.
Next, you need to be able to identify the exact version of your application which generated the dump file. If you are creating your own MiniDumps (by calling MiniDumpWriteDump() for example), probably the easiest way to do this is to simply make part of the filename of the MiniDump the complete version number of your application. You'll need to have a reasonable version numbering scheme in place for this to work. In my shop, we increment the build number across all branches by one every time the autobuilder creates a build.
Now that you have received the dump file from the customer, you know the precise version of the application that created the dump, and you have found the PDB files for this build.
Now you need to go through your source control's history and find the source code for this exact version of the software. The best way to do this is to apply 'labels' to your branches every time you make a build. Set the value of the label to the exact version number, and it becomes easy to find in the history.
You're almost ready to fire up WinDbg/Visual C++:
c:\app_build_1.0.100for application version 1.0 build #100.
Now you have two options for viewing the dump file. You can use Visual Studio or WinDbg. Using Visual Studio is easier, but WinDbg is much more powerful. Most of the time the functionality in Visual Studio will suffice.
To use Visual Studio, all you have to do is open the dump file like it is a project. Once opened, "run" the dump file (F5 by default) and if all the paths are set correctly it will take you right to the code that crashed, give you a call stack, etc.
To use WinDbg, you have to jump through a couple of hoops:
.symfix. This may take a few moments as it will pull a ton of stuff down from the Internet.
.sympath+ c:\pdblocation, substituting wherever you put the PDB files for the pathname. Make sure you get the plus sign in there with no whitespace between
+sign or else you'll screw up step 3.
.srcpath c:\app_build_1.0.100substituting the path where you got code from source control for this version of the software.
After a few moments, if everything is configured correctly, WinDbg will take you right to the location of your crash. At this point you have a million options for digging deep into your application's memory space, the state of critical sections, windows, etc. But that is way beyond the scope of this post.
(see the "Dump" sections below)
Understanding how Workspaces work...
A "cmdtree" allows you to define a "menu" of debugger commands for easy access to frequently used commands without having to remember the terse command names.
You don't have to put all the command definitions into the same cmdtree text file....you can keep them separate and load multiple ones if you wish (they then get their own window).
You can use the -c option on the command line to automatically run a WinDBG script when you start WinDBG.
Gives opportunity to turn on DML (Debugger markup language) mode, load particular extensions, set .NET exception breakpoints, set kernel flags (e.g. when kernel debugging you might need to change the DbgPrint mask so you see tracing information....ed nt!Kd_DEFAULT_Mask 0xffffffff), load cmdtrees, etc.
An example script:
$$ Include a directory to search for extensions $$ (point to a source controlled or UNC common directory so that all developers get access) .extpath+"c:\svn\DevTools\WinDBG\Extensions" $$ When debugging a driver written with the Windows Driver Framework/KMDF $$ load this extension that comes from the WinDDK. !load C:\WinDDK\7600.16385.1\bin\x86\wdfkd.dll !wdftmffile C:\WinDDK\7600.16385.1\tools\tracing\i386\wdf01009.tmf $$ load some extensions .load msec.dll .load byakugan.dll .load odbgext.dll .load sosex .load psscor4 $$ Make commands that support DML (Debugger Markup Language) use it .prefer_dml 1 .dml_start $$ Show NTSTATUS codes in hex by default .enable_long_status 1 $$ Set default extension .setdll psscor4 $$ Show all loaded extensions .chain /D $$ Load some command trees .cmdtree c:\svn\DevTools\WinDBG\cmdtree\cmdtree1.txt .cmdtree c:\svn\DevTools\WinDBG\cmdtree\cmdtree2.txt $$ Show some help for the extensions !wdfkd.help !psscor4.help .help /D
"Extensions" allow you to extend the range of commands/features supported inside WinDBG.
Some blogs (mixture of native and managed code debugging).