Context | Listening test setup | Results

Comparing the encoders

Now comes the fun part: the listening tests! I've chosen two (and now three, since I rewrote this page) songs and seven encoders from between 1994 and 2017. We're gonna run each song through all the encoders and see what comes out the other end, ranking them by listenability and comparing the "artifacts" (distortion and noises introduced by the encoding process) of each one.

The goal here is not to find out which is "the best" encoder, though some encoders will be consistently ahead of others. It's pretty easy to use LAME these days and just not think about it, but there's no fun in that. The full encodes will be provided on the results page for you to listen at home with. I'm sure you'll be able to tell some difference between at least the worst ones.

Picking songs

Song	Artist	Year	Album	Rationale for inclusion
"You Dodged a Bullet"	c.layne	2013	Loom EP*	A crunchy, slightly wet, well-mastered rock track. The drum track has tons of cymbal noise that can trip up an encoder.
"Nightmare-X"	alaska!	2002	Emotions	Low-key acoustic ballad with two harmonizing singers recorded to fairly noisy tape. Noise is infamously difficult to encode.
"Shiny Leather Boots"	The Dandy Warhols	1996 (released 2004)	The Black Album	Ambient track with vocalizations. This is the new track; I figured the spaciness would make for another good encoder challenge.

* The mix here is an earlier one taken from A Collection of Musical Works: 2004-2013, though the current version of that album doesn't feature the track anymore. The Loom EP version features a new drum track and is much more distorted and compressed for stylistic reasons. The lossless upload on this page may be the only place you can get this mix anymore, unfortunately. I quite like it.

Picking encoders

Encoder	Version	Release date
l3enc	0.99a	March 1994(!)
XingMP3	1.5	January 1999
BladeEnc	0.94.2	May 2001
LAME	3.90	December 2001
Shine	0.1.4	November 2005
Fraunhofer ACM	3.4.0	April 2006
LAME	3.100	October 2017

All files were encoded at 128kbps in the default stereo mode of the encoder. I guess it would've been a more fair test if we were forcing either joint or simple stereo, but stereo modes are another example of improving encoder tech, so as far as this highly unscientific test is concerned, that just adds to the wide variation between all their outputs.

All encoders were run on 64-bit Windows, except for l3enc, which is a DOS program and needed to be run in an XP VM.

Introducing EAQUAL

I thought, since I'm just one person with a flawed set of ears, I'd get a second opinion—a robot's opinion. In browsing ReallyRareWares (which is where I sourced all these encoders from and is a banger website you should check out), I came across a program called EAQUAL, which compares an uncompressed original and an MP3 and gives a repeatable numeric value for how big a difference it can hear between the two. This number is called the Objective Difference Grade, or ODG. The scale goes from -4 (terrible and nothing alike) to 0 (sounds identical), so the closer to 0, the better.

I like the way Alexander Lerch, the guy who wrote it, describes it: "Perhaps you should think of EAQUAL as one listener whose results are not depending on his mood and his concentration, who never gets tired and whose results are repeatable." In other words, treat it like another person doing these with me.

Taming encoder quirks

Before we get to the results and sample clips, I should stress that these encoders can be...idiosyncratic to get going. l3enc and the Fraunhofer encoder both hate you for different reasons. I broke Xing at some point, meaning I had to use the command line to get files to encode. Even EAQUAL needs to be set up with perfectly time-aligned WAV files. Here's the commands I used and what issues I faced getting everything going:

l3enc 0.99a

This is a right pain in the ass to use. It only runs on DOS (or 32-bit Windows OSes), only on a local drive, and your inputs and outputs have to have 8.3 file names, or it'll mock you. When it does get going, expect it to take a few minutes.

Sample command: l3enc_fp.exe night.wav night.bit -br 128000

XingMP3 1.5

Comes with a fairly nice GUI with lots of options, just one I broke at some point (probably when I was trying to move it to Dropbox to live with all my other encoders). Thankfully, Xing is just a frontend for x3enc, which works like any other command-line encoder, even batch encoding WAVs if you give it a folder instead of one singular WAV (nice!). Remember to half your target bitrate when setting the command-line switch, because it'll double it for stereo. -B64 will encode to 128kbps, and -B128 will encode to 256kbps.

Sample command: x3enc.exe nightmarex.wav nightmarex_xing.mp3 -B64

BladeEnc 0.94.2

The most featureful (on the user end, it doesn't even support VBR) of the bunch. It can encode to stdin and stdout, there's drag-and-drop functionality, and support for settings files as opposed to command-line switches.

Sample command: bladeenc.exe -128 nightmarex.wav nightmarex_blade.mp3

LAME 3.90/3.100

Both versions of LAME are pretty straightforward. Newer versions are just less picky about exact syntax.

Sample command: lame.exe -b 128 nightmarex.wav nightmarex_lame.mp3

Fraunhofer MPEG Audio Layer-3 Codec (professional) 3.4.0

This one is weird because you have to access it through the ancient Audio Compression Manager framework in Windows, requiring an ACM manager of some kind. I couldn't get ACM Station to take any of my WAV files, but ACMENC worked fine once I figured out how to specify the codec in a way it likes (and it is case-sensitive).

Sample command: amcenc.exe -c "Fraunhofer IIS MPEG Layer-3 Codec (professional)" -b 128 nightmarex.wav nightmarex_fhg340.mp3

EAQUAL 0.1.3alpha

EAQUAL requires time-aligned WAVs. Once you have your MP3, convert it back to WAV and use an audio editor like Audacity to align the starts of the files perfectly (padding the one that starts later with silence). We're talking sample-perfect. It also takes a bit to analyze, maybe 5-7x real time. This does get a little annoying doing it over and over again, yes.

Sample command: eaqual.exe -fref nightmarex_timecorrected.wav -ftest nightmarex_xing.wav

< Context | Listening test results >