Okay, with a toolchain I could compile more modules for the kernel, namely the loop.o driver. With that, I could install a swap file over NFS on the disc server.
So, starting from base2_2.tgz (unfortunately Debian doesn't seem to provide base tarballs anymore -- shame really, it's the easiest way to install when you have no local storage and no compiler), I chrooted to it through NFS, added a swap file so apt-get wouldn't choke anymore (the box only has 32Mb of RAM, which isn't enough for apt-get apparently), and upgraded to Etch accidently. I meant to stop at Sarge, but 'stable' changed in my back. Hmpf.
Then installing madplay and I can play MP3. First step, done.
Next step: Building Tremor (the fixed-point arithmetic version of ogg-vorbis) and a player so I can play OGG file.