I had the same experience as PegLeg84 when it comes to food intolerances.
I do not tolerate grains well, but for whatever reason, if I have corn specifically, it increases my anxiety levels unbelievably. I will literally wake up anxious for no reason the day following corn consumption, and feel anxious for the entire day, sometime two if it was a lot of corn. And then it just disappears if I stay off the corn. Very strange, and I have no explanation for it. But I have met others who have mentioned similar experiences, with food consumption of some kind tying in to increases in anger or anxiety typically. The most common I've heard were grains, dairy, or soy, at least among my acquaintances. Haven't heard anyone able to explain it well, though, so it's all anecdotal and speculative.
Nutritional deficiencies might be really worth looking at. If you get vitamin levels tested, you might want to get a copy of the test results. My PCP and I have had some discussions about vitamins, and he happened to mention that in med school, he was taught that vitamin levels caused no problems unless they were significantly deficient. But in practice, he had a few patients with borderline low vitamin levels report back to him that their symptoms/problems had improved significantly when they took supplements of these same vitamins, so he no longer believes this is true. He tends to recommend trying a supplement for borderline vitamin levels now for some of his patients who have symptoms that match those of low vitamin levels. So, it might be worth looking at that, perhaps.