Tools For Understanding Code? 383
ewhac writes "Having just recently taken a new job, I find myself confronted with an enormous pile of existing, unfamiliar code written for a (somewhat) unfamiliar platform — and an implicit expectation that I'll grok it all Real Soon Now. Simply firing up an editor and reading through it has proven unequal to the task. I'm familiar with cscope, but it doesn't really seem to analyze program structure; it's just a very fancy 'grep' package with a rudimentary understanding of C syntax. A new-ish tool called ncc looks promising, as it appears to be based on an actual C/C++ parser, but the UI is clunky, and there doesn't appear to be any facility for integrating/communicating with an editor. What sorts of tools do you use for effectively analyzing and understanding a large code base?"
Doxygen (Score:5, Informative)
For C++ code, Doxygen [stack.nl] can be useful, as it shows the class inheritance. As requested, it uses a (rudimentary) parser. It works with several other languages too, although I can't vouch for its utility for them.
doxygen (Score:3, Informative)
Understand C++ (Score:5, Informative)
RR & EA (Score:3, Informative)
lxr (Score:1, Informative)
http://lxr.linux.no/ [linux.no]
What I do (Score:5, Informative)
ETrace : Run-time tracing http://freshmeat.net/projects/etrace/ [freshmeat.net]
This book is worth a read http://www.spinellis.gr/codereading/ [spinellis.gr]
Draw some static graphs of functions of interest using CodeViz http://freshmeat.net/projects/codeviz/ [freshmeat.net]
Write lots of notes, preferably on paper with a pen rather than electronically.
Re:Doxygen, and Extracting Software Architectures (Score:5, Informative)
If the system you need to understand has a really big undocumented architecture, then this presentation [uwaterloo.ca] might be useful to you (there is a research paper, but it's not free yet). In it, the authors present a systematic method of extracting the underlying architecture of the Linux kernel.
GNU Global (Score:4, Informative)
Re:Doxygen (Score:3, Informative)
Understand the design first, then the code (Score:5, Informative)
I'm afraid you've set yourself an almost impossible task. IME, there are no shortcuts here, and it it's going to take anywhere from a few months to a couple of years for a new developer to really get their head around a large, unfamiliar code base.
That said, I recommend against just diving in to some random bit of code. You'll probably never need most of it. Heck, I've never read the majority of the code of the project I work on, and that's after several years, with approx 1M lines to consider.
You need to get the big picture instead. Identify the entry point(s), and look for the major functions they call, and so on down until you start to get a feel for how the work is broken down. Look for the major data structures and code operating on them as well, because if you can establish the important data flows in the program you'll be well on your way. Hopefully the design is fairly modular, and if you're in OO world or you're working in a language with packages, looking at how the modules fit together can help a lot too. Any good IDE will have some basic tools to plot things like call graphs and inheritance/containment diagrams, if not there are tools like Doxygen that can do some of it independently.
If you're working on a large code base without a decent overall design that you can grok within a few days, then I'm afraid you're doomed and no amount of tools or documentation or reading files full of code will help you. Projects in that state invariably die, usually slowly and painfully, IME.
Look at doxygen/umbrello (Score:3, Informative)
http://www.stack.nl/~dimitri/doxygen/ [stack.nl]
and:
http://uml.sourceforge.net/index.php [sourceforge.net]
These tools allow you to 'visualize' a codebase in several very helpful ways.
One important way is to generate connection graphs of all functions.
These images can look like a mess, or a huge rail yard with hundreds of connections.
The modules, libraries, or source files that are a real jumble of crossconnected lines are a clear indication of where to start clean up activities.
Good luck!
Re:What I do (Score:1, Informative)
I have used for code bases more than 3000 C/C++ files and yet the IDE behaved well -- jusk like Eclipse for java platform, and consumes very less memory
Re:Stepping Through (Score:3, Informative)
Where be dragons? (Score:2, Informative)
Browse-by-Query (Score:3, Informative)
Browse-by-Query [sourceforge.net]-- it won't help with C/C++(sorry for the original questioner), but it will handle Java or C#.
It dumps the code into a database and lets you query it to find the relationships.
I'm biased, of course, but I've found it's just the thing to understand how a particular piece of functionality in an unfamiliar code base fits into the big picture.
Re:Mod parent up (Score:5, Informative)
Good advice -- breaking randomly. However, it works best in CPU-intensive applications. If the app is mostly idle and event-driven, you're best off searching the code and looking for a place to set breakpoints.
Also, when I use the debugger to help understand some new code, often I'll open a text file and build a "trace" as I go. As I explore things in the debugger and find new call stacks, I add more detail to the trace, in a hierarchical (indented) style. Then I save the traces in case I forget something later.
As for the original question, I would recommend staying focused. Don't go all over the program trying to understand every system at once. Pick a specific part you really need to understand (say, based on a task you have to do) and focus on understanding that.
Unfortunately, the best tool for understanding code is experience. Not theory and not some fancy visualization program. Once you've seen a lot of different code, you come to recognize what each person was thinking when they wrote it. Once that kind of thing comes easily, you no longer find it necessary to bitch about each different programmer's coding style (as some do). So in a way, the guy who posts this question is lucky to have such a big pile of code in front of him.
Use UML, and focus on the interfaces (Score:3, Informative)
I recommend focusing on all interface classes first. This can give you a remarkably sane picture of a system, and will help you divide up the code into more conceptually meaningful chunks.
The tool I use is Enterprise Architect [sparxsystems.com], which does quite a lot of heavy lifting yet is still inexpensive enough for me to own a personal copy.
Re:Doxygen (Score:5, Informative)
Doxygen is more than a javadoc replacement.
I like Doxygen + Graphviz. Just set Doxygen to document all (instead of just the code with tags) and set it to generate class diagrams, call trees, and dependency graphs and allow it to generate a cross reference document that you can read using your web browser. Set the html generator to frame based, and your browsing of code will be easier. I would also set Doxygen to inline the code within the documentation.
I've use Doxygen to reverse engineer very large programs and had good luck with it. I will say Doxygen is not going to do all your work for you, but it will make your job easier. Especially if you add comments to the code as you figure each section out.
Now if you like to see the logical flow of each method then try JGrasp (jgrasp.org). It has a neat feature called CSD that allow you to follow the logic of the code a little better. It's a java based IDE so that may be a turn off for you. I do whole heartedly recommend the Doxygen (w/ Graphviz).
Good luck.
Re:Absolute tosh ! (Score:3, Informative)
The question he's trying to answer is what does the code "do"? why does it exist? what problem does it solve? When you inherit some homegrown ERP system for example, it's easy to find a bug in a routine... not so easy is why input from program A is displayed wrong in program E that is processed by B, C, & D then stored for a week. He's looking for a quick picture of what it all looks like.. in 90% of cases nobody has that info for the CURRENT version of their homegrown system...they might have made the flowcharts, data dictionaries, and code books years ago, but nobody keeps them current.. and DOCUMENTED. How do you get enough info in a short amount of time?
Re:Doxygen (Score:4, Informative)
More than tools (Score:5, Informative)
Feathers, Michael. Working Effectively with Legacy Code [amazon.com], Chapter 16 especially.
Spinellis, Diomidis. Code Reading: The Open Source Perspective [amazon.com], Chapter 10 lists some tools for you.
My own thoughts now. First, don't trust the comments, they are probably outdated. Second, if it's a big code base, forget the debugger. Write some little unit test cases that exercise the sections of code you need to understand, and assert what you think the code is supposed to do.
Finally, unless you are cursed with a codebase which is not kept in version control (in which case, ugh, time to start the jobhunt up again maybe), then take a look at the revision history. See what changes have been made to the area you are working on. With luck, someone will have put in a revision message that points you towards greater understanding of why a change was made, which will in turn nudge you towards knowing the purpose of the section of code that was change.
I had a pile of C++ dropped in my lap 2 years ago. (Score:3, Informative)
I've since added both cscope [sourceforge.net] and freescope [sourceforge.net], as well as the old Red Hat Source Navigator [sourceforge.net] for good measure.
Re:Stepping Through (Score:3, Informative)
Clearly you don't write (or at least read source for) applications of any substance as that would be mildly described as tedious if not impossible.
One of the best ways to understand code is to do so visually with the software equivalent of blueprints. UML is generally considered a very capable way of modeling/communicating both static structures and dynamic behavior of software. There exist any number of tools that are capable of reverse-engineering existing source into UML. Two tools that I consider to be more capable than others are IBM's Rational Rose [ibm.com], and No Magic's MagicDraw [magicdraw.com]. If commercial products aren't a possibility there are likely a number of open-source/free tools--though likely of lesser ability--available. A Google search on "reverse engineering UML" should point you at some.
Re:Stepping Through (Score:2, Informative)
As for using "lines of code", I don't, they do. It seems the biggest issue they have with rewriting code (or refactoring if you prefer) has something to do with the way it's budgeted and accounted for. Apparently adding new code/features to a project comes out of a different budget, than replacing or repairing already existing code does. Don't ask me why that is, I just know whenever we've tried to push to replace some horrendous piece of code they would tell us it wasn't in the budget, and as long as the code ran we weren't allowed to change it. We had to work our way around the bean counters eventually by carefully picking features to implement that touched on code we wanted to replace, then as part of implementing the feature we would rip out and rewrite the code we wanted to.
Source Insight (Score:3, Informative)
Re:Wait for cenqua's solution (Score:1, Informative)
Re:Stepping Through (Score:5, Informative)
Well, the learning curve is certainly important in the real world, although I expect a professional to know his or her tools before they arrive on the job. But there are a metric crapload of things I like better about Visual Studio that make it a much more effective debugger than gdb, in my opinion. (Note that I am not a big gdb user, so I may be cutting it a bit short in the feature set here. My apologies in advance if I do so.)
Things I've found I prefer include many tool windows simultaneously showing the states of registers, memory, the call stack, an object or seven (expanded to show a few properties), and automatic resolution of virtually every symbol and name, including the operating system (although you have to download the symbol files for your OS version from Microsoft.) And you still have full navigation through the source.
Simply hovering the mouse over a symbol will bring up a tool-tip to display the contents. If you highlight an entire expression such as pFoo->pBar->Blah.count+7 and hover, the tooltip will display the calculated result.
You can set a temporary breakpoint by setting the cursor on a line of code and clicking "run to cursor." You can run, single step, run to the current cursor, or run till function return. That last one is great for re-entering a function multiple times to test different conditions.
The variables window contains the current call stack as a dropdown list -- changing the stack lets you see the newly-local variables. Watch windows can display data as hex or decimal, just right click and select. Watch entries can even be used as calculators (enter a literal value, such as 0xf0 + 12, and it will display the results.)
In the watch windows, you can also call arbitrary functions (good for testing without driving your code to that point) or other functions in your memory space, such as the C runtime memory checkers. If you're trying to track an errant pointer, create a debug build, start running and break, type _CrtCheckMemory() into a watch window, and every time the watch window is refreshed, it will check all your fenceposts. You might get lucky and spot your corruption as it happens. The /GZ compiler option will perform a similar task at the function level, but this would let you do it at a line level.
There are also dozens of possible formats it can display your watch variables in -- suffix a pointer with ,s and it'll display the contents as an ASCII string. Only see one byte because of Unicode? Suffix the pointer with ,su and you'll see the unicode string. A ,wm suffix displays window messages by name. ,hr suffix displays HRESULTs by name.
The memory windows will highlight in another color any data that's changed since the last time it was refreshed, whether it be a single step or a previous breakpoint. You can have memory displayed as bytes, shorts, or longs. And with the newer visual studios, you can have multiple memory windows, so you can keep track of two, three or four arrays simultaneously. You simply drag and drop them wherever they're convenient, then step through the code and watch for colored variables indicating change.
Again, all these windows are automatically updated every time the debugger drops from the program to your control. I've got two 17" monitors, and I can fill them both. The problem with debugging is that sometimes you are really starting blind, and the faster you can get more information, the less time you waste debugging.
There's a cute "magic trick" I like to show people with the memory window and the disassembly window. Let's say you've had a crash, and attached the debugger to the running program. You're looking at a corrupt stack in the call stack window -- just one line of garbage data. What to do? Where did it break? Enter @ESP in the memory window. Change the view to 'long' and it displays the memory as 8-digit numbers. If y
Re:Stepping Through (Score:3, Informative)
I used to teach a course in debugging with Visual Studio, and I basically trawled through my syllabus looking for the cool tricks. Using the stack-crash demo to drop into the source code of the crashing module is a real attention-grabber.
I found debugging in gdb to be a lot like debugging in WinDBG. You have to learn a lot of esoteric commands that you don't use very often, so it takes a lot of practice to learn them. And if you aren't constantly searching for the side effects of each step, you can miss a valuable clue. Seeing the color change on watched values that have changed is a great way to pick up on otherwise subtle corruptions. Seeing an entire object's value hierarchy go red because you munged its pointer really stands out, at least to my eye.
Here is the bibliography and recommended references from my syllabus. It's pretty out of date these days (I especially miss the C/C++ Journal,) but the references are still good if you can find them.