You are not imagining it and you are not being difficult. Document it and take the document in.

Bring a list. Embarrassment makes everyone forget things, including you.

Ask for the longer appointment slot. Ten minutes is not enough for a complex history and pretending otherwise wastes both your time.

Test one thing at a time. It feels slower and it is the only way to learn anything. Currently on tamoxifen myself.

Log dose, time and sleep. Ten seconds a day and it turns the anecdote into something you can look at.