for some reason, i cannot start jack with -R, it produces an error, starting without the realtime option works fine though.
what kernel are you using? have you loaded the realtime module, and configured the group setting properly (
http://the.headacher.googlepages.com/jackonvectorlinux)? if not, you can only get realtime privileges if working as root.
If you want some serious realtime capabilities I highly recommend a 2.6.24.x kernel patched with molnar's realtime preemption patch. (You'll either need a little known program called set_rlimits or be good with PAM to get it to work for non-root users, realtime-lsm won't work.) with it I can get 2.9 ms latency with an older budget laptop and a usb soundcard. it's sweet.
some other questions: sometimes i edit a project in ardour and suddenly the main sound (big play button) stops to work (the track is still transported, but i can hear no sound). however, just selecting the single files (ctrl+rightmousebutton, play) works. any hints?
I have no idea, sorry.
and i cannot have xmms and ardour playing a sound when both are opened. is this normal behaviour? why do they block each other, as they both rely on alsa (not oss)?
you should get xmms-jack from the repository. then you can connect xmms to jack as well and play both at the same time.