News that the picture - commissioned by Gordon Brown - had been removed was first reported in The Herald last month.
Politicians on the right were furious at the decision.
Former first minister of Northern Ireland and DUP leader Baroness Arlene Foster described it as “vindictive and petty."
Asked why he had relocated the portrait, Sir Keir said: “I use the study for quietly reading most afternoons where I have got to have… where there is a difficult paper that I need to.
“This is not actually about Margaret Thatcher at all. I don’t like images and pictures of people staring down at me.
“I’ve found it all my life. When I was a lawyer I used to have pictures of judges. I don’t like it. I like landscapes.
…
“I might tolerate Thierry Henry on the wall, but that’s about as far as I go.”
Keep the picture but turn it sideways, then everyone’s happy.