In virtualdub, select Video, Compression, and select a codec. Preferrably DivX or XviD. Set it up for a respectable bitrate (higher is better).
Now try your capture. You should be able to cap at least 2 hours, depending on hard drive space.
If the source is interlaced, use the deinterlace filter.
To work with your VCD, get ISOBuster and extract the .dat to mpeg.
You can now re-encode, or whatever you want with it.
Tutorial on .dat to .mpg here:
http://www.videohelp.com/guides.php?link=95