Sounds like your Doc would rather manage your E2 with your testosterone dosage. E follows T, so if he lowers your testosterone dosage then your estrogen should follow. This is without introducing another medication (AI) . It’s not a bad logic as long as you don’t have low testosterone symptoms at the new lower dose.
It’s an entirely different story if he’s making the change without labwork or doesn’t have a follow up appointment with labs included. If he’s blindly making changes and doesn’t plan on monitoring estrogen, then I’d question what is he monitoring. Is he monitoring hematocrit, hemoglobin, RBC’s etc.. if that’s the case, get a new Doc.