Drawing a line or a mess of lines in any CAD program is a simple matter of knowing where the buttons are. Getting that mess of lines to represent something that someone else can understand is a whole new ball game... producing a drawing that is to a set of standards is even harder. Producing a set of drawings, all to standards, that are consistent in both content and representation, is the suffering of many a cad jockey.
I know Microstation, AutoCAD, Solidworks fluently, with a side dish of Revit, and a handful of other programs I've dabbled in over the years... Knowing one basically allows you to pick up any drafting program and learn it to a functioning level very quickly (i.e. you end up searching for the line button for 5 minutes, then forget where it is... a few times. In the end, all of them get the same product, just walk a different path to get there). Being fluent in a program can take a long time and alot of practice.
I tell anyone who asks that you could train a monkey to use autocad. But, to get anything you could understand out of the program requires skill, training, and an understanding of the subject. To get something out of CAD that is good enough for an engineer to stamp their name on, will take some credentials.
Beyond the simple text, line, polyline, circle commands, there are tools to make your life easier like external referencing, blocking, written blocking, dimension editing, attributed text, layer controls, block libraries, tool palets, macros... the list is endless. Knowing when to use what and when not to, is where the real paycheck is earned.
If you're looking at a career, it's not very stressful unless you make it stressful, but it is hard on the eyes and can be hard on the back if you don't get a good workstation setup. Patience, communication, and attention to detail are what will move you up in a company.
"How hard is it to learn" is more a question of what you want to accomplish with it. If you want to make some sketches or something for a school project, that's easy. If you're considering making a buck freelance or trying to get hired onto a firm, that's not so easy.