It's not all that hard to OC. Especially the CPU. I OCed mine from 2.66 to 3.2 in about 7 minutes.
The video card is a bit more challenging, just because you gotta make sure that you don't get any artifacts and that the OC is stable. Also, that at full load, your card stays under 80C (I'm not comfortable with anything over 80). What video card/CPU do you have? Any coolers and what kind?
I think that it is well worth it to overclock. Get something like RivaTuner or EVGA precision, and then get ATI Tool and Furmark. With the first two, you can OC your GPU, and with the last two, you can test for artifacts and stability test the OC, respectively.
I think it is a good thing to know about. I believe it was worth doing several years back. The fact is as it stands now, most software cannot even properly take advantage of Dual core procs to get the extra horsepower, forget 4 or 8 core procs. I think if software catches up to hardware as it was back in earl 2000 thru 02-03 maybe it's worth looking into. As for now, you are definatly accomplishing one thing. Degrading your procs lifespan.
My 2 pennies.
I agree with Renderedcook overclocking does seem like a cool idea however it just makes your system components break sooner than they normally would. I actually buy components designed for overclocking but then don't overclock them and my computers will work for years (I've actually never had a whole system totally die on me). Also his statements about software being behind hardware are true and I don't think software has ever been able to fully exploit hardware. Just don't worry about it and keep you system how it is.