It is a common misnomer that fiber is the solution to constipation or the lack thereof causes constipation. Do you think wolves or polar bears are constipated from eating meat all day? Or, if you eat watermelon exclusively, which is 0.2% fiber, you will get constipated?
It's complicated, but it is caused by drugs. In most cases, it is caused by casein and gluten, which are decomposed into morphine peptides. Casein is the bigger culprit in bodybuilding land as virtually all protein supplements include it. Truth is, it is a low quality protein and it exists in milk mostly to facilitate the mother/child bonding experience (this is why mu opioid agonists are addictive and calming in mammals, but not other species classes).
Go gluten and casein free, which should be easy to do if you are seriously training and making your own food. But, it may be hard depending on your schedule. It can even be hard for me!
If all else fails, head for the herbs. Dr. Schultze is the best brand by far.
Eliminate
Also - don't eat too many nuts or dried foods. Nuts can be healthy, but either soak them or use a high power blender like a nutribullet.