Here is how I tweaked xorg.conf in VL 5.8 Standard for my logitech marble trackball.
The second line, Identifier, matches the line at the top of xorg.conf.
The changes include "Protocol" "Device" "Buttons" "DragLockButtons" and "EmulateWheelButton". This is set up so that the small right button changes the trackball into a scrollwheel when pressed, and the small left button locks the large one, as when you drag to select large blocks of text, etc.
Option "Name" "Logitech USB Trackball"
Option "Protocol" "auto"
Option "Device" "/dev/mouse"
Option "DragLockButtons" "8 1"
Option "EmulateWheel" "true"
Option "EmulateWheelButton" "9"
I haven't used any other mouse tools, such as imwheel, on this install.
Naturally, other options will result in different behaviour ... There is really no one right way for a trackball to work: it depends on what you want.
I've used similar configurations in other distros; in VL 5.1.1, I had the DragLockButtons set for 4 and 1, and EmulateWheelButton set for 5. It seems illogical that the current config, which uses buttons 8 and 9 and yet has the option for buttons set to 5, would work, but it does.
Tools like xev and xmodmap may be helpful. Sometimes the buttons are reported (numbered) differently, and you have to try different numbers in the options.
Some useful information can be found at xorg:http://ftp.x.org/pub/X11R7.0/doc/html/mouse5.htmlhttp://ftp.x.org/pub/X11R7.0/doc/html/mouse6.htmlhttp://ftp.x.org/pub/X11R7.0/doc/html/mouse7.html
Caveat: make a backup of your xorg.conf file (/etc/X11/xorg.conf) before editing it. If you make a mistake, x may not start, leaving you in text mode until you restore your backup.
 I've removed Option "Buttons" "5"
from the config above ... this option doesn't seem necessary in my current setup, and, oddly, xorg always sees more buttons. (If I set it to 5, Xorg.0.log reports seeing 11; if I set it to 11, the log says there are 15 buttons ... but as everything works I haven't explored further.)