Learning About Software Engineering-Where to Start?

slick_rick asks: "I'm a junior in the CS program at a very horrible university. I just completed my junior level "algorithms" class where the teacher covered linked lists and binary searches. I would really like to learn something before I graduate that I didn't already know...like how to structure a significant piece of C++ code. I have looked at a few books on software engineering, and poked around the Web a bit, but I was hoping for some guidance from the master coders in the Slashdot community on who to believe."

Extremely good advice...

[Anonymous Coward]

You should try eXtreme Programming: http://www.xprogramming.com/ [xprogramming.com]

TAOCP

[Kaufmann]

The one book you must read (well, actually it's a series) is Prof. Donald E. Knuth's The Art of Computer Programming. It's widely considered to be the seminal book on algorithms, software design, low-level computer architecture and concrete mathematics (*) in general. TAOCP is language-agnostic (high-level algorithms in English, low-level in assembly language for a fictional processor called MIX), which, IMAO, is a Good Thing: it frees you to work with the language which best suits you.

Apart from that, I'd say that the only way to learn to program well is - well, d'uh - by programming. Ditch C++; get acquainted with a high-level language environment - Squeak or DrScheme are good - where you can program, test and debug interactively. Start with small stuff - individual classes and functions - and then pick a project or two to put together, something which is useful but which doesn't require a lot of black magic and programming voodoo. Then release its sources on a site like like Sourceforge, so you'll learn from other people who may be more familiar with the subject than you.

So, good luck!

(*) I'm using the term "concrete mathematics" to refer to the field of research that most people call "computer science". Why? Because it's neither really a science, nor really about computers. If you have a better name for it, drop me a line.

SE is a very broad discipline

[RobertEdwards]

Coding distinguishes Software Enginneering from other engineering fields. However, it is not the entire field by any means. Project management, Requirements and Analysis, Architecture, and testing are also vitally important. Steve McConnell has written a book "After the Gold Rush" based on his IEEE essays on Software Engineering. I found it interesting, but not very deep - perhaps it would be a good start for a student. His home page [construx.com] has biblographies and links that I've found very useful.

Re:TAOCP

[Mignon]

It's funny you mention Knuth's TAOCP and then have a footnote explaining your use of the term "concrete mathematics", since that's the title of another Knuth book, co-authored with Ronald Graham and Oren Patashnik. Its preface states that its purpose is to teach the mathematics that are necessary for reading TAOCP.

The Hard Way

[Alik]

From what I've seen (which is primarily academic, not industry), in order to understand good software design you must first do poor software design. Our CS department had a required freshman software engineering class; our team of 7 put together a text editor in a few weeks. We tried to use modular design, but that didn't teach any of us how to build a well-designed system on our own. (Modularity comes a lot more naturally when doing team coding, since you have to break things up into tasks.)

For another year, I still wrote lousy code by cut-and-paste. Slowly, I began to recognize the places where I could use objects and abstraction to clean up the code and make maintenance/revision easier. By the time I got to doing my thesis, I was able to make a reasonably well-built program. (It'd have been even better if the STL worked on the machine I was using.)

Reading the key books will certainly help. However, your code also needs to mesh well with your thought patterns, so that you can work with it easily. Therefore, the majority of becoming a good software engineer is sitting down and integrating the theory with your own style and getting used to thinking for a while before you type. By making mistakes and suffering for them, you will eventually develop an instinctual understanding of good style.

Theory

[jeroenb]

Bad universities probably exist, but how do you know yours is one if you've only been there for a fairly short time? (considering the fact that you are still a junior.)

I'm not quite sure what you expected from your university, but if you thought Software Engineering or Computer Science to be about neat C++ structuring or writing a Quake3 engine, then think again. Sure these are all part of it - but just a tiny bit.

In the end, it's all about understanding Information Technology - what it is capable of and how to apply it. A Quake3 engine is not much different from Oracle 8i in that they both use complex datastructures to maximize the efficiency in throwing data around and presenting it as information to the outside world.

The same is true for C++, COBOL and Basic. They are just tools, nothing more. It doesn't matter whether you give a construction engineer a bulldozer or just a simple hammer to take down a house - it is the knowledge he has about the way houses are built and how they are structured that decides how he approaches the problem. Some tools are better in some places though - the bulldozer is probably better for tearing down a house, the hammer is most likely a better choice for putting a nail in the wall.

The same is true for Software Engineering: it's all theory, but once you master it, you'll be able to do anything. So have some patience with your university - or find another one.

Check out Steve McConnell's "Code Complete"

[cpeterso]

Don't be turned off just because it's a Microsoft Press book. This book taught me more about "real programming" than my college classes did. It's very readable and well researched. This book is my #1 recommendation for all programmers!

Check out the reviews at Amazon for Code Complete : A Practical Handbook of Software Construction [amazon.com].

Admittedly opinionated answer...

[rjh]

1. YOU DON'T KNOW AS MUCH AS YOU THINK YOU DO.

Nobody does. This is the first and most important rule of software engineering; most badly engineered projects I've seen got that way because people thought they knew more than they did, whether on the engineering, managerial or executive level.

Mark Twain said that what people don't know rarely gets them in trouble--what gets them in trouble is what they know that ain't so.

2. GOOD DESIGN CANNOT BE TAUGHT.

Good design is kind of like skydiving--it can't be taught. You don't really know beans about skydiving until you're headed toward the ground at terminal velocity, fumbling around for the D-ring (or lollipop, or throw chute, or what-have-you for your particular parasail). At that point, it all becomes crystal clear in the way that only being confronted with imminent mortality can make things.

Software design is much the same way. You don't know beans about software design until you're actually designing. See a project through from conception to fruition, and then take a look back on it. If your first experiences with real software design are anything like mine were, you'll be embarassed that your name is on it.

3. GOOD SOFTWARE IS GREAT, BUT BAD SOFTWARE TEACHES YOU MORE.

It's always nice to make software that doesn't suck, but you don't learn very much from doing it right. You learn from screwing up, then assessing your screw-ups and making different choices in the future.

4. THE BEST WAY TO LEARN SOFTWARE DESIGN...

... is by doing it. Find your itch and scratch it. Sit down with a notebook and sketch out how you think it should work, then sit down and code the monster. With luck, your design will suck. Print out your code, compare it against your design, and see where reality fell short of your goals.

Then re-code the monster. Make different choices this time; see how much more (or less) elegant your code appears.

Repeat this process ad infinitum until you get some code that you're moderately proud of. At that point, move on to a different project and start over again.

Before too long, you'll find your design skills improving by leaps and bounds. Remember: your goal here is not to make perfect software, but to learn about software. Don't be afraid to experiment, to fiddle around, to intentionally break your software just to see how it falls apart.

And finally, the most important rule:

5. HAVE FUN!

One more bit...

[trims]

I heartily agree with the preceeding posters' recommendations. They all have very valid points, and I suggest you listen to them.

A couple more on my own:

• As was stated before, being a true Software Engineer is less about coding, and more about project management, interpersonal relationships, human management, and other fuzzy details. For you, I wouldn't worry; the best way to pick up on these skills is by working in the industry, and observing your superiors. Some will be good, some horrible, and a few amazing. I've gotten lucky and have worked with more than my share of amazing.
• What your question really is asking has to do with software design. I agree that this is critically important - a huge amount of what the industry deals with today is shoddy design. Besides doing small projects yourself (as a previous poster suggested), I would look at some other people's code. I suggest the GNU code, in particular. Several packages have been long stable (such as bison), and are really solidly designed. I would look at some of the pre-1.0 Gnome code (it's still floating around) to see some messy stuff (I like Gnome, but hey, it was really early code). Look at how they put the stuff together, and try to improve it (not functionality-wise, but style-wise). Learn by doing.
• My final recommendation are several books I've found really helpful:
  • Structure and Interpretation of Computer Programs (Abelson and Sussman) - this is the intro CS book at MIT. While it concentrates alot on programming methodology (and uses Scheme), it's really a great refresher on stuff.
  • Abstraction and Specification in Program Development (Liskov and Guttag) - this is the core Software Engineering book at MIT. It's great - covers all the random things you should be thinking of when starting a project. It uses CLU as a language, but the concepts translate directly to other languages. Unfortunately, it's out of print, but you can still pick up copies all around. The replacement for it (Program Development in Java: Abstraction, Specification, and Object-Oriented Design (liskov)) won't be out for another couple of months.
  • Finally, read Introduction to Algorithms (cormen, leiserson, rivest) - this is both a great book on all sorts of algorithmic methods, but a good software design book as well. It's used as the core CS algorithms book at a large number of colleges (like, maybe in 40% of all CS people I know used it in college).

The books above can be found at the MITPress [mitpress.com], including how to get out-of-print stuff.

Handouts and other stuff for classes at MIT that used to above books (6.001, 6.170, and 6.046J) can be found from the MIT Course Catalog here [mit.edu] - you might explore a couple of the other 6.0xx series (ie, look at the web pages associated with them) as they have useful stuff there too.

Best of luck!

-Erik (an MIT dude, if you missed that...)

The Impossible...

[Carnage4Life]

I would really like to learn something before I graduate that I didn't already know...like how to structure a significant piece of C++ code.

What you have asked for has yet to be created. There are books that try to explain software engineering and many that explain the mysterious workings of C++ but only experience can teach what you want. Here are a few books that will advance your knowledge of both C++ and software engineering to help you along your way.

1. Mythical Man Month [amazon.com] by Frederick Brooks. This book is about 25 years old but still rings true today. It is a classic in computer science history and dispelled several myths that existed at the time (e.g. he showed that adding developers to a late project can only make it later).

1. Code Complete [amazon.com] by Steve mcConnel - he may be a Microsoftie but he knows how to write a good book. He covers everything from analysis and design to proper coding and testing practices. A must-have.

1. The Practice of Programming [amazon.com] by Kernigan (K from K and R) and Pike. A handy little book that contains several truths that seem like common sense to the experienced programmer but are precious jewels of knowledge. It contains dozens of tips in C, C++, Java and Perl for help in testing, debugging, programming style and design.

1. Effective C++ CD: 85 Ways to Improve your C++ [amazon.com] by Scott Meyers - This is a combination of his bestselling books and articles (by himself and otheres) all on one CD. This book is the best C++ resource I've seen and is owned by most of the C++ gurus I have ever met. The knowledge on this CD is invaluable.

1. Design Patterns: Elements of Reusable Object Oriented Software [amazon.com] - by the gang of four. This contains 23 of the most often used designs that occur in Object Oriented Programming in industry. The book is a classic and has saved many a developer from reinventing the wheel several times over.

I can't wait for the book that will show developers how to create well engineered software without being bogged down in the tediousness of the text or the infeasibility of the suggestions as is displayed by most of the current crop of software engineering books today.

Some hints

[randombit]

Design Patterns is a good book for OO design. Given you situation, I recommend you also read Kernighan and Pike's The Practice of Programming (both very good books). Another is to simply code a lot of things, in several different languages. I've found that new languages could give me insight into design (even shell). Not just 'how to design in this language', but 'why was this languages invented? what can it do so much better that something else couldn't be used?' kind of thing. Dunno, maybe I'm just weird, and that won't help you at all. :) Definitely read TPOP though.

Apprenticing...

[Orville]

The way I have learned many good software engineering techniques is simply by learning from others. The part I have always struggled with is "where to start" and "what's *really* important". (Especially with Java Enterprise technology...) Having experienced people around to ask questions and bounce ideas off of is the best experience you could have.

Unfortunatly, as a student there typically aren't that many experience software designers. (They usually don't teach as a primary profession...) Get a part time programming job for a local company if you can, or look for project teams formed in other university departments. (Especially teaching hospitals. They often need software comparable to anything in private industry...)

Learn from experienced people, and gain experience. The best way to learn.