I might be missing it but I don't see how long it took you to lose the 12 pounds? If it was a short amount of time, just getting rid of grains could result in weight loss from water weight, glycogen stores and waste. If you're worried seeing a doc is your best bet.
Keep in mind that regardless of whether gluten is good or bad for you, grains do contain a lot of calories. So if you have cut them, you may be low on calories, or at least significantly lower than you were. You may need to make up those calories somewhere else (this of course depends on how many calories you need vs how many you're taking in). Just something to think about.
Also, just reading your post...whether you have an official gluten intolerance / sensitivity it seems you have your answer on whether grains are affecting you negatively.