Phone chargers draw almost nothing, usually only 1 or 2 watts. If the GPS is being charged by a normal phone-style USB charger then it's just as small. If the GPS is plugged into a cradle with a built-in charger then it might draw more but it should still be only a few watts. If you leave the bike parked in the garage for a few weeks and forget to disconnect the GPS then it might be an issue but for normal riding you should have no problems at all.
I am running a full car audio head unit with 4 speakers and charging my cell but I run my gps out of my cell as well. I also have a hooked up cigareete lighter charger for my wifes phone in the saddle bag.
If my bike sits for a weak or so I can't run the radio without jacking up the firing of the bike for awhile until the battery charges completely if I leave it off my battery tender.