How and where and why have you opened ports?
You havn't let us know what model router you are using, please do that.
When the lag spikes occur it may be worth running a traceroute to try and identify where the hold up is.
Have a look at the link for info, when you have a lag, run traceroute on something like
Google, and also on your gaming server if you can.
[Tutorial] Traceroute on Windows XP - Surpass Web Hosting Forums
Post the results.