Bigpaws beat me to it while I typed a rather wordy post, but I'll add my $0.02 to his answer:
Short answer: yes, you can.
Longer answer: Linux does not impose artificial limits on what you can do to your computer. When you are working as root, it will let you do whatever is technically possible, even if it means breaking the system. If you command it to delete everything on the hard drive, it will. You can also craft your own personal system with exactly the programs that you want.
There are thousands of packages that work together to make an operating system. The dependencies are many and complicated. Getting it all to work takes a lot of trial and error. The resulting release represents much time and effort by the developers. When you start removing and modifying pieces, you venture into the developer/hacker territory. You are invited and encouraged to do so. Have some fun, learn a lot. But, count on breaking your system beyond repair, many times. You will get good at re-installing and re-building. So make sure you start with a machine dedicated to experimenting. Or at least a separate hard drive for that purpose. I believe I speak for most of the oldtimers in this forum when I say "been there, done that".
We have talked about making a minimal install and selectively add to it, as well as starting with a heavy install and subtract from it. Unless you are limited by hard drive capacity, there is a third option: Make a full install and re-configure it. Files on the hard drive do not make a system heavy, it's what's loaded into memory that counts. You can have a machine with all the goodies available, but run a lightweight window manager. That way you get the best of both worlds. Try starting with a full install of VL Standard. Install the icewmvmods package. Then select IceWM from the login screen. Use the system configurator to disable most services. Now you can choose which environment you want. Keep XFCE for when you want a modern, automated environment, switch to IceWM when you want more speed.
