I don't think Milk is the culprit, although I did try cutting it out for a while (Went on an exclusion diet). I get a runny nose pretty much immediately (within minutes) of eating certain foods, but can drink vast amounts of milk without issue. I also believe whatever my food allergy is is aggravating some neurological issues I have (Tourettes syndrome, OCD, and crippling anxiety). These have significantly subsided now that I'm 'Gluten Avoidant' (i.e. mostly avoid gluten, although not anally strict about it).

Wheat is, most of the time, pretty much guaranteed to set my nose off, but I can eat (for example) higher quality breads generally without issue (Cheap bread will make me congested for hours), and I ate a danish the other day without problems.

The overwhelming factor is that things with wheat usually set me off (not always), and foods without wheat very rarely do (but sometimes can). Restaurant or Takeout food is almost guaranteed to set me off regardless of what it is, whereas home cooked food is much safer. (I think this is probably largely due to cross contamination).

I'm thinking it might be a preservative, or a specific wheat protein other than gluten that doesn't always make it to the final product. Alternatively it could be multiple things, which complicates the process considerably.