I can't change my primary poster

I'm having trouble changing my primary photo. Currently, it's set on the wider photo, but I want the smaller grey photo. Either is fine.

I have a few I'd like to change out. Is this going to be an issue every time and is there some way I can avoid this problem?