What happens if you type CTRL-V followed by a comma in insert mode? Do you get a comma?
Also, since you only have this problem in gvim, not vim, try running xev and typing into it to see if the events look odd. (you may need apt-get install x11-utils for xev) The events I get look like this:
KeyPress event, serial 30, synthetic NO, window 0x4000001, root 0x236, subw 0x0, time 788140933, (138,120), root:(144,139), state 0x10, keycode 59 (keysym 0x2c, comma), same_screen YES, XLookupString gives 1 bytes: (2c) "," XmbLookupString gives 1 bytes: (2c) "," XFilterEvent returns: False KeyRelease event, serial 30, synthetic NO, window 0x4000001, root 0x236, subw 0x0, time 788141013, (138,120), root:(144,139), state 0x10, keycode 59 (keysym 0x2c, comma), same_screen YES, XLookupString gives 1 bytes: (2c) "," XFilterEvent returns: False