Eight correct answers out of ten sounds convincing. In an ABX comparison, however, a person guessing between two answers can get eight or more right about once in eighteen ten-trial sessions. A score needs its context before it tells us much about MP3 and FLAC.
The ABX question is fairly modest. A and B are known reference files; X is one of them, selected without revealing which. Can you identify X? You are not voting for a favourite format, and the experiment cannot tell you whether you should replace an entire music collection.
The arithmetic behind ten trials
For ten independent trials with a 50% chance of each answer, there are 1,024 equally likely answer sequences. The NIST explanation of the binomial distribution supplies the model; the sums below are our calculations.
| Correct answers out of ten | Probability of at least this many under the guessing model |
|---|---|
| Eight | 56 / 1,024 = 5.47% |
| Nine | 11 / 1,024 = 1.07% |
| Ten | 1 / 1,024 = 0.098% |
For the first row, add the sequences giving eight, nine and ten correct answers: 45 + 10 + 1. Divide 56 by 1,024. That is where 5.47% comes from, rather than a measurement of anybody's hearing.
These figures describe outcomes under the stated model. They are not the probability that a particular listener was guessing. Starting over until you obtain nine correct answers also changes the procedure. Keep those abandoned sessions; reporting only the pleasing one hides information.
Prepare a comparison that answers the codec question
Use a lossless source you may legally play and encode a copy from it. Record the encoder and settings. An MP3 from one release and a FLAC from another could differ in mastering, editing or level before their formats enter the picture.
Converting an old MP3 into FLAC does not restore the information removed during lossy encoding. Our format and storage guide explains why retaining a lossless original and making a portable copy are different jobs.
Choose a short, recognisable passage. You need to return to the same section, not hunt through a long track for something you vaguely remember hearing. There is no need to upload the music to conduct a comparison on your own computer.
Write this down before pressing start
| Record | Your entry |
|---|---|
| Source | Release, track and excerpt |
| Encode | Codec, encoder version and settings |
| Playback | Player, output device and connection |
| Controls | Volume, EQ and normalization |
| Plan | Trial count decided in advance |
The official foobar2000 ABX Comparator page provides a component for double-blind comparisons. Check its compatibility and instructions for your installation. We have not run a listening trial for this article.
Learn the controls, then leave them alone
Listen to A and B first. Use a comfortable level and resolve an obvious loudness mismatch before beginning. A level difference supplies a clue unrelated to recognising encoding artefacts; our ReplayGain explainer covers that distinction.
Keep the playback path fixed during the trial. Changing headphones or EQ midway makes the report harder to interpret. Save the full result and record interruptions separately.
A successful identification applies to the files and conditions you used. An unsuccessful attempt means that session did not establish reliable identification. Neither result decides what everybody can hear, or removes the practical reasons for keeping an archive.