I can say that I can vouch for the difficulty of programming in 2D. I KNOW I should be able to understand SDL and Allegro and all that. I just... don't. OpenGL is easy, though. Odd.
Command-line programs, though, which are all we've done in my C++ class (well, SOME graphics stuff, but nothing that'll get you far; they're vector-based and with a limited palette, too), are really freakin' easy. Oh sure, every now and then some odd assignment comes where you have to do something the teacher hasn't quite explained how to do, and that can be a challenge, but only about a day or two's work. And by a "day", I mean 45 minutes. Most of that's just debugging, since every now and then an odd glitch will occur that you can't understand. My latest program, for instance, was displaying a hex number for some reason; it took me a while, but it turned out I'd coded "cout<<cout<<[insert crap here];". Minor things like that.
I just want to get to memory pointers already, and why they're so invaluable. Other than allowing functions to directly manipulate variable data and maybe the odd array, I've not figured it out. But it'll be a while, we need to deal with vectors first. Not that I need it, I know how arrays work thanks.
Until then, all game programming sticks to The Games Factory. Seriously, about the most advanced game I can program right now is Pong...