Put one of those small headphone covers on your N64 Joystick. It doesn't hurt as much, is nice and soft, protects the joystick a bit, and your joystick doesn't get all sweaty after several hours of yanking it around.