Yes - use the --disable-aso configure option.
I have tried configuring with --enable-fpm=arm/default/64bit and --disable-aso but still the asm errors when compiling with -mcpu=arm7di.
One question, why it there a file called imdct_l_arm.s and not any other called imdct_l_mips/sparc/i386/xxx ? Why does arm get its own file ? I thought this was maybe generated by configure but it is there from a fresh untar so this can't be so.