One question, though. Do you have a lot of fat in your diet? I'm like you, and stay think pretty much no matter what. I'm also dairy free. When I'm breastfeeding, especially those first few months, I crave fatty things and I figured it was to supply fat to my milk. I would literally eat spoonfuls of coconut oil each day. And I'd use a LOT of olive oilin my cooking.
Getting your supply up (if your body will let you) will require almost around the clock nursing. (iI've kept track, and my all time record was 9.5 hours with a baby latched on to me in a 24 hour period... Intense stuff... But they seem to be worth it )