You are not imagining it and you are not being difficult. Document it and take the document in.

Ask for the longer appointment slot. Ten minutes is not enough for a complex history and pretending otherwise wastes both your time.

Written confirmation email after every phone call. Cheapest insurance there is.

Bring a list. Embarrassment makes everyone forget things, including you.