Thursday, September 12, 2013

Access Violation in CPropertySheet::DoModal

One "feature" about MFC that has always annoyed me is when you create a wizard or property sheets, calling CPropertySheet::DoModal results in an access violation.  This problem is discussed in detail in Q158552.  In short, MFC tries to modify the dialog template, but the dialog template is stored in a read-only page.  This results in an exception.  The operating system catches this exception, changes the page to read/write access, and tries again.  So really this is just a nuisance if you're debugging your application because it breaks on the exception every time.

Well, turns out it can be more than a nuisance.  If you're testing with Microsoft Application Verifier enabled, you cannot debug your application further.  AppVerify is designed to catch buffer overruns by using read-only memory pages.  So when CPropertySheet::DoModal attempts to write to a read-only page AppVerify steps in and puts a stop to the show with a "VERIFIER STOP 0000000000000002" error.

So how can you get by this?  You could disable AppVerify, but testing with AppVerify enabled is a good thing.  It helps you catch other bugs that would normally go undetected.  But I found a simple technique that allows you to keep AppVerify enabled.  Use Visual Studio's resource editor to change the font of the wizard pages.  The default font is "MS Shell Dlg."  You can change this to any font, but I recommend "Tahoma" as this is the font the Visual Studio resource editor uses when the dialog is set to "MS Shell Dlg."  This means your dialog units will be the same which makes arranging controls easier.  Other good font options are "MS Sans Serif," "Microsoft Sans Serif," or "Arial."  These are also generic fonts with the same or similar dialog units.

The reason this works is because of what MFC does.  It checks the font of the wizard/property page and if it doesn't match the font of the wizard/property sheet, it makes a copy of the dialog template and changes the font back to the system default.  This new dialog template is read/write so when processed inside CPropertySheet::DoModal it won't cause a problem.  And if you're following along, changing the font of the wizard/property pages will not change your product at all.  Since MFC changes the font back to the default it doesn't matter what font you change it to, the look and feel of your product will not change.

2 comments:

  1. I'll be dipped. Just spent a few hours tracing down the offending exception, new it had to be something with the dialog resource, and finally came across this post.

    Tried changing the dialog font, and bingo no more access violation exception.

    Thanks!

    ReplyDelete
  2. Awesome, thanks for the details!

    ReplyDelete