I think this link will explain some thing about receptors/signalling/downregulation:
Explain To Me: Receptor Upregulation/Downregulation | meandmymastcells
Also this list may be helpful:
List of human hormones - Wikipedia, the free encyclopedia
"Almost everyone postulates this is due to some downregulating of AR's"
You used right word: "postulates", and in reality we dont know what the exact problem is. Maybe is problem with signaling or somewhere else? Or combination of number of problems. I really dont know.
"are all androgen receptors built for a specific androgen like estrogen and estrogen only, or do some or all take all types etc?"
some hormones can act on different receptors, sometimes testosterone can mimic DHT and vice versa, T3 downregulates TRH receptors in the a/p which leads to decreased responsiveness of HPTA with chronically elevated T3 , progesterone can downregulate some estrogen receptors, prolactin upregulates some receptors in breast/nipples etc...
I think if you continue this path, there will be more improvement but how much- only time will tell. You should not worry too much as it does not help.