I get some popping as well, and I think they always occur in the same place. I'm not yet sure what the cause is or if there is a solution. It shouldn't be clipping for me, but I will have to investigate that further.
When you enter in a buffer size do you not hear clicks? And if that's the case, what buffer size are you using? The default is 1024 and from my understanding that's a perfectly good value for almost any situation. The larger the buffer you use, the more latency gets introduced. For instance, I was trying to do real-time visualizations so I dropped the buffer down to 512; however, at that size I was getting lots of really bad pops because my CPU couldn't fill the buffer faster than the audio card was depleting the buffer. So in general I think you want to stick with 1024 or 2048.
As for .wav files of 96/24-bit quality, I don't think they are supported but I haven't tried. As far as I know, Minim only supports 8 and 16 bit files (see the paragraph under "Getting an AudioOutput"). As for the 96kHz aspect, I suspect it may be asking too much to expect a Java library to be able to record 96,000 samples per second, but I am not sure about playback.
I hope this helps somewhat.
Best,
Kevin