You're a low aromatizer. There's no clinical evidence of any long-lasting post-exemestane syndrome. Frankly, 20 pmol/L is not that bad (I've seen much lower), and I'd consider it generally to be a good thing, aside from the concern of lipid regulation. In terms of individual metabolism, you may prefer aromatization to E1 or E3 with certain compounds. Did you happen to have DHEA serum measured?
Just watch your lipids (LDL, HDL), as estradiol is a potent regulator of these; this is really its sole beneficial function in men in my view.
Next time you get blood work see if you can have measured:
Estrone (E1) and estriol (E2) as well as E2.
DHEA (serum).
These results might show something interesting.