I had exactly the same with the same software.
I found the only way to get it to work was to set the resolution of the cameras down to SQIF 320 X 240.
The only problem I've got is when dropping to this resolution, all the cameras appear fine while on the main page with all 16 cameras on but as soon as I double click a camera I just get a full screen in pink.
I have 2.01 version, not sure which you have.
I also want to run in the higher resolution but have found if I do, the camera screens start to jump and the motion detection doesn't work(detects all the time).
I have tried all updated GeForce drivers, replaced Geforce 8500GT with the latest 1Gig ATI Radeon Graphics card, upgraded to 2 gig of Ram and upgraded to a Duo Core Processor. All to no avail. I am hoping there is a software upgrade somewhere, but hard to find the authour of the software.
Hope this helps, or someone can help me, Marlon