In more detail, binaural recordings place the microphones in an ideal location for capturing the sound, often with a dummy head. But the mics never move, so if the 3D effect of being there is to be experienced, you need to keep your head as still as the mics were when the recording was made. This is difficult, but crucially important.
Our brains are extremely adept at locating sound sources around us. Most believe it was a survival of the species thing where those who could hear and locate prey the most quickly survived. Even more critical is hearing and locating larger predators and getting away from them. One of the ways our brain locates the source of a sound, especially in the frequencies above 5kHz or in the 100Hz to 500Hz range, is to pay extremely close attention to how the source sound changes from subtle movements of the head which relocate the ears which are using very sensitive time arrival perceptions to know where the sound is coming from. The way we (or dogs) cock our heads when we want to locate a sound, such as our phone ringing or our children somewhere in the house, is crucial to how we locate things. If we want to trick our brains through binaural sound recording into believing we are at the location where the recording was made, we have to ensure our heads do not move at all. I often call those listening sessions my "recovering from neck surgery" sessions because I pretend I am locked into one of those complex metal head braces which prevents me from moving my head around. It can work, but you have to fight your instinct to cock your head to locate the sounds in the recording. For instance, you may hear a member of the audience slightly behind you yelling something you find terribly intriguing, but it isn't as clear as you'd like and you cannot make out everything. If you were there in person you'd turn your head and try to see that person so you could focus on them and hear their words more clearly. You have to fight that instinct when doing binaural listening of a live event. It gets even worse if you succeed in experiencing the whole damn thing because the realism if the illusion can force you to drop your guard and start moving your head to hear things in the recording which are not clear in the current microphone placement.
This issue is one reason the success of binaural recording has relatively limited. Most people who are into the technology don't really experience it as intended because they are caught up in the culture of fetishizing some cool tech the rest of the world is too stupid to understand. Others are plain ignorant. A portion of the enthusiasts are good at what they do and know exactly what the tech can offer. That makes it hard for the mainstream to really enjoy.
So, that's my little discussion of binaural listening.
