You can try a few things.
1) reduce your G.I. and sampling settings, because you are going to render it to 640 x 480.
2) bake textures to scene objects. Which means the computer will render the lightings/shadows to your texture. (like many games they do it this way) Once that is done, you can disable G.I calculations of those objects which allows the render time to speed up. Do not bake texture on objects that will move in the scene though.
This method requires quite a bit of planning, and the time for render to texture could take just as long...but you only need to do it once.
I only know how to do it with default scanline render and mental ray only. I am not sure if other renderer support this at all. You may need to find tutorials for Brazil or Vray.