IMO any knowledgeable health professional would never tell you to stop taking HRT on just one bad BP reading. It sounds like this doctor is trying to find excuses for not prescribing it, purely based on their own biased opinion, and that sounds like I doctor I wouldn't want to be using.
A lot of people suffer from "white coat syndrome", and I'm one of them. My doctor is lovely and I'm not nervous about seeing her at all, yet my BP would always be high when she takes it. After my mother passed away I kept her BP monitor and I started taking mine on a regular basis and it's perfect. I now just jot it down in a little book and show it to my GP every time I go and she's happy with that.
I'm not saying that you don't have a BP problem, but you really need to be monitoring it yourself at home where you will get much more accurate readings. They're not that expensive to buy. Just get your own and you can use it anytime you want to. You can get them at most pharmacies.