Slashdot Log In
Use of Math Languages and Packages in Research?
Posted by
Cliff
on Wed Feb 26, 2003 02:32 PM
from the using-the-right-tools-for-the-job dept.
from the using-the-right-tools-for-the-job dept.
CEHT asks: "As a research programmer at the university, I have encountered numerous times when I need to choose which language(s) or package(s) to use for different projects. Tradeoffs and performance issues have to be considered: results from one package may be more compatible with the data from other researchers, another package may find the solution faster and use less resources, and so forth. Maple,
Matlab, Magma, and Mathematica
are among the most well-known packages. Libraries such as IMSL is also popular. Of course, there are smaller (and mostly free) packages that tend to target specific types of problem, such as LiDIA, Singular, and LAPACK.
The question is, how useful are these [and other] math packages? Do researchers use only one or two packages for most of their projects? Or do people like to mix things a little by pulling the strength of different packages together to solve a math problem? If not, do researchers write C/C++ programs and use GMP or Matpack to solve math problems?"
This discussion has been archived.
No new comments can be posted.
Use of Math Languages and Packages in Research?
|
Log In/Create an Account
| Top
| 454 comments
(Spill at 50!) | Index Only
| Search Discussion
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
(1)
|
2
(1)
|
2
Octave (Score:4, Insightful)
(http://tanpiover2.livejournal.com/)
Depends (Score:5, Informative)
(http://caseysoftware.com/blog/ | Last Journal: Wednesday March 31 2004, @05:31PM)
CHALKBOARD is great for addition and the other basic operations, but if you want to do symbolic algebra, Maple or MathCad are your best bets.
If you want to do some sort of signal processing and/or crazy matrix applications, the Matlab is probably the answer.
If you want to do something with statistics, Matlab or Minitab are the way to go.
Re:Octave (Score:5, Insightful)
(Last Journal: Wednesday December 13 2006, @06:43PM)
Re:Octave (Score:5, Informative)
Re:Octave (Score:4, Insightful)
First, Redhat (I'm sure other distros as well) INCLUDE OCTAVE. That's right, you don't NEED to compile it because it comes with your distro. You can't get much easier to install than selecting the package at OS install time.
Second, Octave doesn't require a user to jump through 8 hoops to run it on your system. No FlexLM, no licence files, no keys, no giving Mathworks your hostID when you change computers, etc.
Finally, most people don't compile from source. And those who do are usually experienced enough to figure out how to get things compiled.
This might be useful ... (Score:4, Informative)
http://www.gentoo.org/dyn/pkgs/app-sci/octave.xml [gentoo.org]
Re:Octave (Score:5, Interesting)
Octave is a nice MATLAB clone, developed from chemical engineers in the beginning, but now used extensively in virtually any area that math is usefull.
Many packages have their open source counterparts: Octave [octave.org] for MATLAB, R-system [r-project.org] for SPLUS (statistics algebra system), and so forth. But IMHO you raise another issue: you can use each of these packages to do whatever calculations you want, since all of them are extended in the C/Fortran end, i.e. they can use programs written in these languages. Custom code is readily integrated. And above all, the GNU Scientific Library [redhat.com]. If you don't like or you don't trust the numerical solvers integrated in MATLAB, you can investigate the source in the GSL.
And yes, you can use all of these together. So, what is the question again?
Re:Octave (Score:4, Informative)
(http://www.davidgrant.ca/)
MathCad? (Score:3, Insightful)
(http://www.IgniteSoft.com | Last Journal: Wednesday March 16 2005, @01:44PM)
Re:MathCad? (Score:4, Insightful)
I want to learn how to use Matlab more effectively as it's (apparently) the most effective for physical modelling, but we don't get taught it over here (Mathcad, Maple and Mathematica are all these scuzzers will teach us); anyone know a good intro to it on the web?
-Mark
Re:MathCad? (Score:5, Informative)
(http://francisbarnhart.com/)
User friendly? Are you talking about the program that I use on a daily basis? Surely not. MathCAD is without a doubt the prettiest of all the options but it is among the worst in user interface.
For those of you who are not familiar with MathCAD, it works like this:
Anything and everything that you want to input into MathCAD is in it's own little box. Be it a text or an equation box.
The horrid part is trying to organize all these boxes on the page. Putting everthing in a box means that it operates completely contrary to what most people are used to with MS Word. Say you enter some equations and then decide you want to add a few more in the middle. You can't just hit the up arrow and start typing with maybe an enter. Instead, you'll often have to select the later equations and drag them down to make room for the new. Then, if you have a lot of equations you likely didn't move all of them down. So, you have to select the equations that now overlap and select 'Separate Regions' from a menu. This gets to be very tedious.
Furthermore, is it too much to expect MathCAD to figure out that I don't want have my equation on page one and the rest on page two? Why should I have to go and select "Reimpaginate' from a menu before I print?
Entering equations is no joy either. I'm constantly frustrated when I try and do something as simple as add antoher term to an equation, like changing x^2 - 3 to x^2 + x - 3. I find myself starting over and at times typing 1 + 1 - 1 and then replacing the ones. I mean, come on, I've seen many math typeing solutions that are far better, in MathType, and LyX for example.
Sure you might have a nice looking document but was it really worth the pain? Furthermore, I find MathCAD to be seriously lacking in function compared to Maple et al.
Of course, Maple et al. all have their problems with user interface. Why should I have to end with a semi-colon? And you have to realize that it's never going to look the way you want it to. So you have to suck it up and do the math without worrying about the beauty of the output.
Not to sell MathCAD short, there are some things that it does do well:
Units, the best unit management system I've had the joy to use. Very nice.
The output is beatiful.
Simple math that doesn't require big complicated equations and lots of loops.
Personally, I can do the easy math by hand. For more advanced stuff check out SciPy.org [scipy.org]. They provide a python interface to established numerical algorithms in C and Fortran. But it's much quicker and 'funner' to use. Unfortunately they are only at alpha right now. But, you can't be the price and for the most part, I've found the optimization sections to be quite stable. Combine it with pychart and your've got a good science package for free.
Otherwise, the only package that I've actually heard people rave about is Matlab.
Heh ... (Score:5, Funny)
(http://www.beryllium.ca/)
Fortran (Score:5, Interesting)
(Last Journal: Thursday May 01 2003, @05:04PM)
age-old answer: it depends (Score:5, Interesting)
matlab for design prototypes of numerical algorithms and for visualizing data.
mathematica for doing messy algebra/calculus/differential equations.
my own c/c++ code, with a lapack backend, for doing large-scale computations (matlab and mathematica are too slow for big computations).
So, the answer is e) all of the above!
Perl Data Language (Score:4, Interesting)
(http://sleepingsquirrel.org/ | Last Journal: Friday July 15 2005, @05:07PM)
Re:Perl Data Language (Score:5, Funny)
(http://www.fruitsofinsanity.com/)
Inexact floating point calculations... (Score:4, Interesting)
(http://slashdot.org/~$$$$$exyGal/journal/#naked | Last Journal: Tuesday January 04 2005, @04:23AM)
Do any of the listed tools/languages take care of this problem for me? I understand the nature of the problem, but it is still very frustrating. What do the "pure" math programming languages do with this issue?
--sex [slashdot.org]
Re:Inexact floating point calculations... (Score:4, Insightful)
It's been a while since my Senior Independent project, but it was on products like these. I did a lot of the symbolic maths on the packages to describe and document what they were doing. As part of that, I had to look into how they handled floating point errors. I don't remember the package I was specifically working with; and my copy of the SIP is at home.
With that as an introduction, for the "pure computational" packages,the problem you point out is real. Floating point errors when ignored will slowly move further and further into the significant digits of the FP calculations. For a package to be even reasonable, it must be able to describe in mathematical expressions and textual dialog how it will manage FP errors to keep them in the least significant digits of the number. If you review a package, look and ask specifically how the package does that.
Some may ask why is this important, don't modern languages handle all this according to the FP Specs? Well, basically the specs are not good enough for large computation tasks. When you start multiplying several matrices together, you end up doing so many FP operations, that without carefully written and mathematically backed code the errors will pratically zoom "to the right." This is compounded by the fact that not all chips comply with the specs in exactly the same way, most of these packages will have a lot of conditional code to handle each chip set's specific particularities. Something else to look for: if a package claims one size fits all, and doesn't talk about OS and HW specific compiles, take extra care checking the FP issues out. They may be taking a worst possible processing approach, which will work but at the expense of speed, or they may be taking a more "mean" processing approach, which may end up with different results on different OS and HW combinations.
Now, after all that, the reality is that most of these packages, at least if they have been around a while, have the mathematical grounding and programming "backgrounds" to handle FP operations pretty darn well. After all, this was a fairly well known and documented issue back in 1983 when I wrote my SIP.
The "symbolic" packages seem to side step this by first taking the equations and modifying them in "symbol" form before performing their calculations. Thus, the differential of X^2 is changed to 2x by the program before any FP operations happen. But this does not mean that FP operations do not occur. If your equations still deal with matrices, then a lot of FP operations will have to be done to come up with a numeric answer, no matter what.
a couple other packages (Score:3, Informative)
Other trade offs (Score:5, Interesting)
(http://briancnorton.info/)
I am not able to articulate this well, but the type of research you are doing is MUCH more important of a consideration than computation speed or resource consumption. If you need supercomputer time, then you had better ask the admin what you need to use. I know a bunch of people that do environmental modelling, and I have never seen or heard of anybody writing their own C++ to do it. Researchers GENERALLY have better things to do than re-invent wheels.
MatLab, Mathematica (Score:5, Informative)
(Last Journal: Tuesday February 25 2003, @03:02PM)
For most computer vision code, Matlab is a must for prototyping. It's useful in other areas, and, if you know how to use it, reasonably fast. If you're doing particularly involved matrix manipulations, it takes a lot of work to come up with C/C++ code that will work faster then well-written matlab code.
Personally, I also use Mathematica for doing real math work. If I need to derive something that's particularly complex, then Mathematica's notebook style is really nice to work with, and it makes possible extremely clear and concise mathematical arguments while limiting stupid human errors when doing drudgery like taking derivatives and the like.
I hear Maple and MathCad are both good, too, but I've never used them.
FORTRAN (Score:3, Informative)
(http://roboadam.com/ | Last Journal: Wednesday February 26 2003, @12:23PM)
Mathematics past (Score:3, Interesting)
(http://slashdot.org/~Shadow%20Wrought/journal | Last Journal: Saturday November 17, @12:05PM)
Matlab, C, VB, local scripting (Score:5, Insightful)
(Last Journal: Monday June 09 2003, @06:24PM)
The most commonly-used analytical platform is probably Excel (or some similar tool like Statistica), but the more serious researchers, who are also the more mathematically-aware, nearly all use Matlab in my experience.
When efficiency is an issue, nearly everyone I've worked with turns either to IDL (a Matlab competitor that has more arcane syntax, but much higher processing speed) or writes a C/C++ program by taking algorithms from "Numerical Recipes in C".
Recently, I've also seen a rising use of Visual Basic, especially to do experimental control (although some Matlab hooks do exist for such), and, of course, LabView. Some diehards use LabView for data analysis as well, but their results are suspect just because the tool is so poorly fitted to the task.
And, of course, many data collection hardware manufacturers (CED, National Instruments, TDT, etc.) supply scripting languages to control their hardware and perform rudimentary and sometimes not-so-rudimentary calculations.
The best researchers select the most appropriate tool for the job, but, again in my experience, it seems the selection is normally based on previous experience and inertia. Those who know a particular tool well (eg, Excel, Matlab, SPSS, Mathematica) tend to keep using that tool, even if it is not well-suited. This means you get abberations like Matlab programs that control real-time experiments and LabView programs that do higher-order mathematics.
Why?
Because the largest fraction of a scientists' time should be spent on data collection, not experimental implementation, and the amount of time (for nearly all fields except those with astronomical amounts of data) spent executing code is dwarfed by the time developing it. Clearly this breaks down for certain applications, but most of the science currently being done (read: molecular biology, and no, not bioinformatics) is not algorithm-bound.
Since data analysis is such a huge, broad field, I expect to see radically different answers from other posters!
Maxima! (Score:4, Informative)
Re:Absolutely, Maxima is very, very useful (Score:4, Informative)
A problem I've struggled with ... (Score:3, Insightful)
Are there any viable open-source solutions to either Mathematica or IDL?
Which package... (Score:3, Informative)
(http://www.digitalhermit.com/)
I can use Mathematica for almost all of my dabbling. Sometimes I play with MuPAD, R, GnuPLOT, Octave or Mathematica to show a particular problem. Since these are also free (beer or speech, depending on package) I can be reasonably sure that everyone can get a hold of it.
For example, Octave is suitable for matrix manipulation. It does everything that I need it to do and can replace Mathematica for me. It's also fast enough (the longest calculation has taken just over a minute but it was a huge manipulation of some graphic data).
I've dabbled with some of the libraries but only for fun.
I guess what it comes down to is how comfortable are you with the package. By the time I try to write something in C using a dedicated library I can most likely do the same thing in Mathematica in a tenth of the time. Even if the execution speed was 100 times slower, the "real" time may not about to much.
depends, or, if you have to ask slashdot... (Score:5, Interesting)
You're talking about two different classes of software: "numerical linear algebra packages" and "computer algebra systems". Maple and Mathematica are the latter, Matlab is the former. I don't know about Magma.
Hardcore numerical programmers use LINPACK/LAPACK with platform-optimized BLAS (this latter is often commercial, or at least proprietary to the platform vendor) directly from Fortran. They usually use modern commercial Fortran 90 or Fortran 95 compilers, too.
On numerical linear algebra stuff where you aren't going to recruit and pay a Fortran programmer with a PhD in applied mathematics, most sane people use Matlab or GNU Octave or one of the many other Matlab clones. A lot of people like Numerical Python, if I had a big new project to do, I'd seriously consider it.
Yes, crazy "researchers" who don't want to learn Fortran and think Matlab is too slow or too expensive will write numerical code in C++. Some of them do fine work, too.
Excel and other spreadsheets are fine for small bits of numerical analysis, too. Don't turn up your nose at 'em, you can email your boss your whole analysis and he doesn't have to learn Matlab to do anything with it. Excel is also slowly replacing Qbasic as the computing lingua franca of the Amateur Radio/hobbyist-electronics community.
The class of people who just doodle out the singular integral equations for the airfoil design they're brainstorming seem to like Mathematica a lot. I wish I were more like that. Maxima is seeing a renaissance now that its licensing and distribution issues are cleared up (it's GPL now). I should check it out. There's also GNU (Emacs) Calc, which I use regularly as an RPN desktop calculator. It is actually much more powerful than that and will do all kinds of HP-calculator-style graphing and computer algebra with a liberal sprinkling of Mathematica-style syntax, but I don't use those features much, because they're wicked slow.
MATLAB: Costly but extemely effective (Score:5, Informative)
MATLAB offers student versions for about $99 a pop, which is dirt cheap considering its $1000 price tag for the retail version. Many universities of course have dramatic discounts, but then, you have to have be affiliated with a univeristy. Even the student version requires you to attest that youre using it for course work or student-level research and not commercial gain.
MATLAB has a number of drawbacks. Price is the largest. To enforce its license, MATLAB requires you to run the onerous and clumsy FlexLM license manager. FlexLM is brought to you by GLOBEtrotter....a division of that bastion of consumer rights, Macrovision. That should speak volumes. The license manager makes doing a lot of simple things stupidly difficult, especially if you're (like me) mobile and have to authenticate with a central server running the license manager. I can get into details if people have questions.
On top of that, MATLAB requires a yearly "maintenance" fee. It's more or less software as a service. Apparently, if you let the maintenance contract lapse, you can still use MATLAB, but you get no more support and cannot apply any new updates. That may be, but the particular license my university employs will cause my copy to simply stop working after April 1 if I don't renew. (April 1 being the beginning of the Mathworks license year. I don't think they see the irony in choosing that date).
The maintenance contract does not apply, AFAIK, to the student version.
On top of THAT, the student version or the $1000 base retail installation just gets you the MATLAB core. Which, granted, is extremely powerful. But the Mathworks also has a couple dozen or so Toolboxes, each with a range of specialized functions and tools (i.e. Signal Processing, Image Processing, MATLAB-to-C Compiler, Symbolic Math, etc. etc.). Each of these comes for an additional price, and its own maintenance fees. IIRC, these are like $500-$700 more each.
Did I mention all these prices are for licenses on a per seat basis? Any institution or company thinking about MATLAB is going to shell out serious bucks for the privelage.
On the other hand...MATLAB is a serious, extensible, highly flexible platform for technical and mathematical computing. I find that I can prototype programs for solving scientific problems in MATLAB far faster than I can in any other language. And its visualization features are truly impressive...even if the Handle Graphics system it uses is SO DAMN KLUDGY to program. You can customize visualizations just about however you can imagine...ALTHOUGH, some simple customizations are going to be UNNECESSARILY tedious to program.
Another drawback to programming in MATLAB is speed. MATLAB ("Matrix Laboratory") is exceptionally optimized for handling calculations of very large matrices. However, because it's interpreted, if you have any loops, it's going to be very slow going. There often many tricks to "vectorize" operations you'd normally do iteratively in other languages, but often the only solution is the ol' for-next or while loop. These are slow. Very very slow. Yes, there's a compiler, but in my experience the compiler isn't that great at optimizing code...and, did I mention it costs extra?
Anyway, MATLAB is amazing in its breadth and depth of power. I haven't even touched on its capabilities for engineers, like the SimuLink system design simulator, and hardware interface toolboxes. I can't imagine a problem needing to use a "mix" of math packages (as the original poster asked) if you're using MATLAB. But the purchase and ownership costs are very steep.
Scilab (Score:5, Informative)
Anyway the best package for you in part depends on what you are using it for. Matlab, scilab and octave are great for doing linear algebra things -- manipulating matrices and arrays etc. Some people complain about how slow matlab is. I find matlab is pretty fast as long as you use it for what it was designed for. You should use their built in functions as much as possible and use as few loops as possible. If you find yourself using a lot of loops try writing a mex function in C or FORTRAN.
Maple and Mathmatica are great for Calculus differential equations etc. If you are doing a lot of matrix mulitiplies in Maple, you should be using matlab.
Mathcad is user friendly but it is SLOW. Even old people who have been doing insane integrals in their heads since the 50's and refuse to even look at a computer can see a Mathcad print out and tell exactly what the program is doing.
Hope this helps. Personally I like to use Octave and Scilab since they are GPL. Scilab is prettier IMHO but Octave is closer to Matlab (which I am already used to.)
Phew! (Score:5, Insightful)
(http://foo.ewu.edu/ | Last Journal: Monday June 18, @12:43PM)
First a global kind of classification.....
octave/matlab... are mostly vector/array oriented languages and are useful for doing work in problems that are suited for such - you can experiment easily, then recode in C,fortran... if needed. apl and j are also in this group and should not be ignored - though they're used a bit less frequently.
Macsyma/mathematica/maple/maxima/derive are symbolic math languages and can solve interestingly sized problems and get symbolic answers (that is, things like sqrt(pi/2)) as well as numeric approximations. This can be a very useful tool to have - depending on what I'm doing I use such things a couple times a week (nice to check results done by hand, or to handle all the crufty part of a solution). Most will emit fortran code which can also be useful.
vtk, opendx/khoros(?) are visualization tools - most of the other packages have some visualization tools packaged in them, but vtk and opendx both offer quite a bit more power.
Now the incredibly non specific recommendation
My suggestion is to pick one of each of these and learn it - do enough in it so you know the language/system well. Otherwise you'll be struggling with the language as well as with the problem - and finding bugs will be close to impossible.
If I put on my "computer science professor" hat (probably a wise thing if I'm to keep the top of my head from turning bright red with sunburn), I usually try to recommend that all CS students learn a smattering of these things as well. When you need one of these tools, knowing its there and how to use it can save large and wonderful quantities of time.
And now some more specific comments
On the whole my choices would be as follows - note the caveats - some of them are pretty cave-rnous (sic). I don't have piles of money to spend, so tend to prefer the open source programs just on that basis.
For array/matrix manipulation I much prefer APL or one of its derivatives (check out aplus on sourceforge). Languages in the APL family are also fun to program once you learn how. However the terseness of the syntax (and with APL itself the odd character set) tends to make these a bit forbidding, so a more popular choice would be octave (open source) or matlab. I've had good luck with octave - it seems to handle most matlab programs well enough. If you've got piles of money, go for matlab.
For symbolic math, maxima (sourceforge) is good. Its commercial cousin Macsyma has usually ranked as about the best symbolic math packages for accuracy and power and seems less expensive than the others. Actually writing programs in either of these requires learning quite a bit about the innards of the system though. My second choice for symbolic math would be Mathematica - its programming language is well integrated with the system as a whole and and for general goodness and niceness of the interface it can't be beat. (The other commercial products are building on the best parts of the Mathematica interface - I've not checked recently, but they're getting much better fast.) The visualization capabilities of Mathematica are also very good. Maple is probably the most popular, so using it will probably make it easier to find someone to help you, but on the whole I've just never found Maple as easy to program as Mathematica and I tend to want to program almost everything.
For visualization both vtk and opendx are very nice systems. vtk is more aimed at a programming interface, opendx has a labviewish kind of programming environment. I like both and have both at hand. Both these systems are big enough that you'll want to make sure you understand them before you tackle a project with them.
They don't scale well, but spreadsheets can be very convenient for small models. Careful though, its easy to have errors even in middlin sized models that can be very hard to find.
Odd Zen Endz
As has been noted there are other systems, some smaller, some more specifically focussed on a single domain. Those tend to be harder to match to a problem - unless the problem is right in the center of the domain in question.
There used to be a program AXIOM which had a lot of nice features, but it seems to have gone to that Big Bit Bucket in the sky - but its base language "Aldor" is now available at aldor.org. I have a copy, but haven't looked deeply at it.
Sourceforge is also hosting a new project "lush" - which is a lisp system that has some integration of some of these features. To the extent that I've used it I'm impressed and will probably spend some time working deeper with it in the hopes that it will prove another valuable tool.
A couple of suggestions (Score:3, Informative)
(http://benambra.org/)
If you want to do some statistics, there's also R, a stats analysis package. It's very powerful, but it's designed for experts rather than non-statisticians who occasionally want to crunch some numbers.
Be Careful Though (Score:4, Funny)
recommendations (Score:3, Informative)
Don't underestimate the power of C++: with type checking and overloading, C++ may actually be more convenient to use for many numerical applications that even something like Numerical Python or Octave/Matlab.
Beyond that, yes, obviously, all those libraries are in use by someone if they are maintained. If you have the need to use one of them, you will know. If not, don't worry about it.
Re:Maxima (Score:3, Informative)
We are trying to get the old site to direct people to the new site. Since the old site is not under our direct control, it isn't as easy as one would hope. (I am the Maxima project leader.)