What format is the audio file you're using? If it's compressed (like MP3), you might get better results using an uncompressed audio file - try converting it to an uncompressed WAV file, for example.
Thanks guys, it was a quicktime issue from what I gather. I was using an quicktime alternative and everything is working fine now that I've gone to the free QT7.