Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Programming IT Technology

Automatic Functional Testing for Mac and Linux? 202

Posted by Cliff from the true-software-testing-suites dept.
testYourGUIS asks: "I work with a company that tests consumer and business software for various venues. One of the programs we rely on heavily is called Rational Visual Test, which is a VB-like compiled programming language that works directly with Windows API to simulate mouse clicks, text entry, menu pulldowns, and so on. We use them to stress-test various applications, collect the results, and then send the vendors a spreadsheet showing what we found. Recently, pressure has started to produce the same results on Mac OS X and possibly Linux applications. I was wondering if anyone knew of such software that works on Mac OS X, and how to go about doing this for Linux?"

"For example, a vendor has a database interface, and wants to know how many times we can access a series of records, what the results show (are the right, blank, time out?), and how long it took for the search to take place during various times of the day, and so on. So we emulate one of their data entry people by logging onto the system, putting in some search parameter from a script, and then record what the text (or error message) was when it came back. Yes, we could connect directly to the SQL database without their front end, but it's the combination of their front end and the database we usually test. We have found many examples where were can connect raw, for instance, but the front end displays a timeout. We also test things like 'What happens if the user enters a colon or asterisk?' or 'What if the text entered in the box is over 256k?' Then we tell the vendor the results, and they hopefully fix them based on the data we gave them."
This discussion has been archived. No new comments can be posted.

Automatic Functional Testing for Mac and Linux? More

Automatic Functional Testing for Mac and Linux?

Comments Filter:

  • AppleScript (Score:3, Informative)

    by krog ( 25663 ) on Thursday July 25, 2002 @04:30PM (#3953682) Homepage
    you can do this with AppleScript, quite possibly the most underrated language of its kind.

    • Re:AppleScript (Score:1, Informative)

      by chris613 ( 237882 )
      "underrated"? Applescript is cumbersome, painful, and WAY too unreliable to be used for verification testing. I've had serious trouble just getting applescript to run and then close certain stock applications. It's a broken mess, and I wouldn't recommend it to anyone, especially not for testing.
      • This is what I thought as well, However creating an AppleScript would probably take forever.

        I think this guys best bet is to develop using cocoa and to give a timed public beta to the mac community.

        However, I couldn't hurt to check out Apple's developer connection page as well. There is a -lot- of good info there. Ask some developers at Apple. Moreover, try to ask some developers from the OmniGroup ( http://www.omnigroup.com ). They do a lot of consulting, they are friendly, and they have been developing with NeXT, Rhapsody, and OS X for years.
      • While certainly not a suitable answer to for the guy who Asked Slashdot, AppleScript is fantastic. How the hell is it cumbersome, painful, and unreliable?

        If you can read and write English, you can become a pretty decent AppleScripter in a few hours. Pick up a decent book on it, and you'll be making your Mac do backflips in a few days.

        I've used AppleScript to write a CGI that duplicates the functionality of Outlook Web Access with Microsoft Entourage.

        I also make extensive use of it in my web-based control system for my home automation rig-- hell, AppleScript is the BACKBONE of my home automation rig, gluing together various apps.

        In about 20 minutes one day, I wrote a script to strip out the stupid, extraneous shit from "CNN Breaking News" e-mails, create a new message with nothing but the actual news item in it, and send it along to my cell phone.

        One of my clients (a huge, multi-national food company whose products are mm mm good) uses two AppleScript CGIs that I wrote for searching job archives they have on their internal web site.

        Sure, sometimes I've had to write a small 'test' script on the side to help myself understand how to extract a particular data item from a record, but it's a relatively painless process.

        Whenever you have trouble with a scriptable app not made by Apple, blame that app's vendor for not doing a good enough job at making it scriptable, DON'T blame AppleScript.

        ~Philly
    • Can applescript handle running 1000 concurrent users from a desktop machine simulating 1000 concurrent users hitting the server?

      What he needs is a load test and functional test product similar to Rational's, but cross-platform. I'd suggest looking into Mercury Interactive's software, though I'm not sure just how cross-platform it is.

    • Re:AppleScript (Score:2, Informative)

      by scrod ( 136965 )
      Actually no, that wouldn't do him any good. He would still have to implement the scripting commands in his program. He wants something to simulate the use of the actual interface, not call the underlying functions.

      • Re:AppleScript (Score:3, Informative)

        by guttentag ( 313541 )
        How about QuicKeys [cesoft.com]? When I couldn't get AppleScript to translate spoken commands into certain keystrokes and mouse clicks in my mapping program, I found that QuicKeys was able to do the job. (I ended up using AppleScript for the Speech Recognition input and QK to manipulate the interface)

        Someone mentioned server load testing earlier... if that's what you're looking for, remember that Mac OS X is basically a pretty window manager on top of a flavor of FreeBSD -- you can have cron automate shell scripts and perl.

    • Re:AppleScript (Score:2, Informative)

      by DLWormwood ( 154934 )
      This assumes that the application being tested is fully scriptable. This requires a factored application design and/or an object model to convert AppleScript commands into their corresponding events. Historically, Mac OS development tools don't automatically create apps with such a structure. And as another poster noted, the means to have AS work with non-scriptable apps requires the use of unreliable "osaxen" that make timing and reliablity metrics meaningless.
      • ...aren't really necessary for functional testing, which is why most of these functional testing tools don't [more like can't] run in real time. IMO, these relatively pricey functional tools from the big venders could hardly be called reliable. I suspect that the tiny niche marketplace for such tools ensures that things will probably stay that way.

        OTOH, Real life timing is really important for scalability testing, i.e. the domain of load testing tools, which mimic front end client behaviour in real time via back-end automation. These are definitely far more reliable, but also exceedingly more expensive.
    • You can do this with AppleScript...

      No, you can't. How does AppleScript "simulate mouse clicks, text entry, menu pulldowns?"

      Answer: it doesn't. It calls the application directly, skipping the UI. Which is not so good if you want to do user interface testing.

  • testing tools (Score:5, Informative)

    by Anonymous Coward on Thursday July 25, 2002 @04:31PM (#3953696)
    try OpenSTA. Its an open source automated testing application, similar to Rational's product.

    www.opensta.org

    There is a commercial company that provides support at www.opensta.com
    • I took a peek at this, and it appears OpenSTA is for Web-based testing, not GUI testing. Did I get that wrong?
    • No. From their site: OpenSTA graphs both virtual user response times and resource utilization information from all Web Servers, Application Servers, Database Servers and Operating Platforms under test, so that precise performance measurements can be gathered during load tests and analysis on these measurements can be performed.

    • Re:testing tools (Score:3, Informative)

      by leandrod ( 17766 )
      For all I could gather OpenSTA is still MSW32-only, with plans to porting to other platforms; and it can't yet do tests other than http yet.

      IANATester.
  • AutoIT [hiddensoft.com] is cheaper

  • Hmm. (Score:5, Informative)

    by dasmegabyte ( 267018 ) <das@OHNOWHATSTHISdasmegabyte.org> on Thursday July 25, 2002 @04:33PM (#3953715) Homepage Journal
    Automatically uses the API to interact with the machine? Yeah, they've got this for Mac...it's called AppleScript and it's been around for roughly 20 years.

    Don't be scared by the name "script"...there are visual tools for ascript and the language itself is so simple my mom could do it.

  • XRunner (Score:5, Informative)

    by micantos ( 252195 ) on Thursday July 25, 2002 @04:33PM (#3953717)
    Check this [mercuryinteractive.com] out for some pretty reasonable X-windows testing. Heard quite a few positive things about it.

    • Re:XRunner (Score:1, Informative)

      by Anonymous Coward
      We currently use Winrunner, The win32 equivalent of XRunner by Mercury Interactive. We've had lots if problems with this product, It's very buggy for what my standard of a automated test suite should be (hmmm a Automated test suite that actually has ben TESTED?). Constant restarts, variables changing values and so on are day to day recurrences. The support from MI is extremely poor, generally taking days to weeks to get any issue resolved. XRunner might be a bit more stable, but given its high price tag and the poor quality of this product I'd definitely advise anyone to stay away from this.
      • If you think that price is high, check out their LoadRunner products (75K-275k, depending on what architecture you want to test).

        But I agree, WinRunner blows. So do many of the competing products, (ahem, QARun) which all seem to lack serious QA prior to release. This is what you get when you've have to fight over a few thousand customers who are used to paying ridiculous amounts of money for broken enterprise software.
  • I've worked with Rational Visual Test(or Irrational Test as we often refer to it.) With both you can call commands from different programs. I have't tried doing similar tasks in PERL, but mostly because I have not immersed myself in PERL.

  • Java... sort of (Score:4, Insightful)

    by d3xt3r ( 527989 ) on Thursday July 25, 2002 @04:34PM (#3953728)
    Are these applications you need to demo like this, Java Swing apps? If so, you can use built-in javax APIs to do this. You could simulate user actions this way, mouse movements, etc as a "demo" mode for the user.

    Obviously this would work on Linux, OS X, and Windows as well.

    Hope this helps.

    • I'm assuming you're talking about java.awt.Robot [sun.com]. This, with BeanShell [beanshell.org] would allow test scripts for GUIs to be built.. A more sophisticated system would be able to capture AWT events, and store them in a file for later playback...

    • Re:Java... sort of (Score:1, Funny)

      by Anonymous Coward
      using java.awt.Robot, it's quite simple.
      to quote the documentation:
      This class is used to
      generate native system input events for the purposes of test automation, self-running demos, and other applications where control of the mouse and keyboard is needed. The primary purpose of Robot is to facilitate automated testing of Java platform implementations.
      I've used it to make remote control apps that work in windows and linux: take a screenshot of desktop A, send it to B. Send mouse events from B to A. Freak out user on machine A. :-)

      Quite simple, and easy to do in Mac OS as well!

    • At work here I suggested as a product that was in beta, on the basis that it looked interesting and "why not give it a shot" and the other recommendations were so expensive. And the guys in testing got a trial copy and have been very happy with it, though I haven't used it myself. It is qftest [www.qfs.de] from Quality Software First. We build our java stuff for Win2K , Linux and Solaris. And this seems to do the job well. And its dirt cheap compared to anything from Rational or Mercury where you've gotta shell out big time.

  • Legos! (Score:5, Funny)

    by djtack ( 545324 ) on Thursday July 25, 2002 @04:37PM (#3953755)
    I've got it... you need Lego Mindstorms. [lego.com] Just build some robotics to push the mouse around, whack random keys, etc. Best of all, such a system would be fully platform-independant. I bet Microsoft is already doing this, only they call it "genetic programming".
    • Let a cat in the room and pretend you are doing some serious work.
    • Lego plural is lego thank you very much!

      one lego
      two lego
      three lego
      one million lego

      • Lego plural is lego thank you very much!

        Actually, LEGO is an adjective, so has no plural.

        one LEGO brick
        two LEGO bricks
        .....

    • The funny thing is, WAAAAYYYY back in the day, mebbe 10 years ago, I remember ZDLabs bragging in PC Magazine about how their testing facility had this jury-rigged arm that pressed keys on a laptop to simulate use for battery life tests.

      Funny thing is, I remember thinking, why not just use a microprocessor hooked up to a keyboard controller, and plug the thing in thru the external keyboard port??? (yes, most laptops back then had 'em) :)

  • We're working on one... (Score:3, Informative)

    by matticus ( 93537 ) on Thursday July 25, 2002 @04:37PM (#3953761) Homepage
    Actually, the company I am currently employed by is in the process of writing a system/gui test environment for this very purpose. One caveat-it's only for Java, but when we complete it (very soon), we will release it to the public. our address is www.atomicobject.com [atomicobject.com].
    The name of the project will be HASTE, look for it in a month or two.

  • XTest and Expect ? (Score:4, Informative)

    by too_bad ( 595984 ) on Thursday July 25, 2002 @04:38PM (#3953764)
    Isnt this what XTest [handhelds.org] and
    Expect [nist.gov] allows us to do on Linux
    (and a vareity of other operating systems) ?

    • Re:XTest and Expect ? (Score:2, Informative)

      by Vee Schade ( 6806 )
      XTest must be compiled into X as an extension and appears only to allow "fake key presses" which will probably not be very useful in this context. Expect allows for "automating interactive processes" (and is VERY good at it - IMHO), where such processes are CLI-driven (e.g. telnet, ftp, etc.), and so is also not useful in this context.

      • Re:XTest and Expect ? (Score:4, Informative)

        by cduffy ( 652 ) <charles+slashdot@dyfis.net> on Thursday July 25, 2002 @06:08PM (#3954519)
        Actually, android [wiki.tcl.tk] lets you script fake keypresses, mouse events and such from a TCL interpreter (which expect happens to be). However, it doesn't let you see and check widget contents as text in the same way that many of the Windows-based test systems do. The commercial products for X suffer from similar failings when not using explicitly supported widget sets; as such, automated testing is just one of those things that isn't so well supported under X Windows. Hopefully Berlin (or some other successor) will help in this.

        (FWIW, my job very recently involved buying or writing an automated test scaffold for graphical apps on Linux that would work across architectures and windowing systems -- it turned out that the latter was necessary, as the available commercial products just weren't/aren't cross-platform capable, and android has some serious faults which I'll expound on if asked. The product partially exists, having been put on hold for other internal development... perhaps when it's finished we'll release it under an open license).
  • You might be able to use Rational Visual Test with a windows based X-server to control Linux (or any Unix) apps.

    • Re:Automatic Functional Testing for Mac and Linux? (Score:1, Informative)

      by Anonymous Coward
      no you won't

      The problem is that tools like VisualTest and WinRunner can find specific objects (buttons, dropdown boxes etc... and treat them as objects. When you use such a tool with an X Server, all the tool can see is a mouse click at a specific location. This makes your automated playback screen size and window location dependent. This is not a reliable way to handle this.
  • I think there's more to that story. A company hired to test program written for Mac OSX and Linux, not just Windows. Anyone else catch what I'm getting at?
  • Why don't these people just do a search on freshmeat and google like the rest of us? You'll undoubtable just hear from the other slashdotters who just got down googling and freshmeating (he he he) for their results.

    Okay there's an offchance that another slashdotter might be working on an esotric project like he is and happens to have a viable solution. Highly unlikely. Or perhpas slashdot just wants to bring this to our attention. Don't know why though? Or mabey IT'S SHAMLESS ADVERTISEMENT PLUGGING yet again =P

    Go ahead and give me your worst repliers.
    • Sure, Google gives you software but Slashdot could give also opinions, experiences and ideas. (Sometimes it actually does.)
    • Suggested format: I just did a google / freshmeat / sourceforge search for whizzbang-mousetraps, and turned up superProjectX... I installed it and... Has anyone else used this? Anyone got any other suggestions?

      Actually, I usually scan these posts, they often spin out some very interesting viewpoints and experiences, especially when the topic is a development or testing tool, I probably encountered ant on /. I would also think that the amount of interest generated for the projects mentioned, which are usually Open is not insignificant.

      My theory is that these are usually people who have not been trolling sourceforge or building linux boxes fgor years, are in fact probably coming out from the dark side. Have some compassion for them!

  • Since you're already a Rational customer, I would check into Rational preVue [rational.com]. Many years ago it had another name - and was a product from a company that Rational bought...

  • xnee (Score:3, Interesting)

    by tompa ( 102471 ) on Thursday July 25, 2002 @04:45PM (#3953840)
    Don't know about OS X, but here is an excellent GPL'ed event recorder/playback app for X Windows: xnee.sourceforge.net [sourceforge.net]

    you could always port it to OS X ... :-)

    /tc

  • Check out Test Quest Pro (Score:3, Informative)

    by Jay ( 1991 ) on Thursday July 25, 2002 @04:49PM (#3953865) Homepage
    It's a PC you can use to drive keyboard / mouse input into another system. It comes with a video capture card to read the output and make decisions about whether the test was successful or not.

    It's fully scriptable using a c-ish syntax, and comes with a nice IDE and runtime environment to get the work done. It seems like it may be a nice match for what you want to do.

    Here [testquest.com] is more info.

  • I've got two words for you... (Score:5, Funny)

    by pi_rules ( 123171 ) on Thursday July 25, 2002 @04:52PM (#3953892)
    Trained Monkeys.

  • Here's a website that talks about the language: http://expect.nist.gov/. I bet there's other sites so check google. O'Reiley also has a book about it. I still think I'd use Applescript under MacOS, but under Linux I'd give expect a try. I played with it a little under Solaris and it looked pretty useful. I was able to automate the setup for US Robotics modem racks and Cabletron ethernet hubs using expect. Never did get that stuff totally debugged and my script would go crazy, but that's not the language's fault.

  • Depends on your budget. Can you afford a thousand monkeys?
  • You could probably roll your own rather quickly with the CGRemoteOperation API: /System/Library/Frameworks/ApplicationServices.fra mework/Frameworks/CoreGraphics.framework/Headers/C GRemoteOperation.h

  • F-Script (Score:3, Informative)

    by Tomy ( 34647 ) on Thursday July 25, 2002 @05:02PM (#3953963)

    For Mac OS X have a look at F-Script. [fscript.org] You can inspect a running program and call any method in that program and any method on any object in the Objective-C runtime. This should allow you to generate events or enter text in text fields, etc.

    Of course this will only work for programs developed using Cocoa. Can't help you with Carbon apps. You might be able to write something using the Apple Event manager.

    Last time I had to do this for Linux (and Gtk+) I had to roll my own.

  • I'm sure it can be easily adopted to what you're trying to do!

    Google [google.com]

  • A while back I found an app called xmacro. Just a simple app to send mouse clicks and such. If you can't find a package to do what you want, you might be able to roll your own from it.

    Xmacro [sourceforge.net]

  • Funny (Score:3, Interesting)

    by psocccer ( 105399 ) on Thursday July 25, 2002 @05:13PM (#3954038) Homepage
    I just posted a set of tools for automating X at my website [hoopajoo.net] that I recently created to make a bot that could play Yahoo! Spelldown. There are 2 parts to the tools, finding things on screen using a "visual grep" program that finds images inside of images, and xte, an app built on the XTest extension that allows scripting of moving the mouse, clicking, dragging, keypresses, etc. Together you can use them to make scripts that can identify buttons on screen or letter or words or numbers, whatever, and interact with programs. Still a pre-1.0 version, but it does everything listed above so far.

    BTW, don't use xse, which is a wrapper for XSendEvent, if you roll your own. There are so many places where it just doesn't work, or doesn't work right, because the XSendEvent function just doesn't work reliably. I lost a lot of sleep over that before I gave up and just wrote xte.
  • i think compuware's QARun or QADirector work on *nix
    • Holy piss, madd scientist, I can't believe you recommended that ;)

      QADirector is not a testing language in itself. It is a harness from which to run the script generated by script writers. It used to be a Unix app, but when the switch was made to Wintel there were a few missteps. If you are looking for a test harness that is comprehensive (for Windows anyways) in its command-line options, look into QAD, otherwise I would go with Test Director by Mercury or maybe Segue's new product (I believe its called SilkPlan or something similar).

      QARun on the other hand is a pain in the ass. If you are looking for automation, don't use it. It's designed for straight-up "record and playback" style testing. If you wish to deploy these tests to client machines (something which some, but not all people want to do), do not use QARun. You can look into XRunner (it is the predecessor of WinRunner) by Mercury if it is still being sold/supported for *nix testing. I honestly don't know about Mac, our product doesn't run on it.

      If you want a serious evaluation document, I highly suggest heading over to http://www.qaforums.com [qaforums.com] and head to the Automation section. Post a query about the Mac and you'll probably receive a response from one of the moderators with a white paper that was written comparing all of the available tools.

      DISCLAIMER: My group uses QARun, QAD, Segue Silktest, and WinRunner, so as not to be accused of bias.
  • for mouse events, you can try QuicKeys X [cesoft.com] from CE Software for Mac OS X, anyway. It can enter text, too, and launch applescripts.
  • For the right amount of payment I've got a couple of friends who'd love to bang on the mouse all day! I tend to prefer the keyboard myself, so I'll do the typing. Best of all, we're entirely cross-platform (among platforms can handle physical input, that is).

    Finally, I've discovered a use for this strange box on my desk.

  • Over the years I've asked Rational salespeople if they plan on porting some of their more popular debugging tools to Linux, such as Purify, Pure Coverage, Quantify, etc. The earliest responses to the question were, "What's Linux?" In later years, the response has simply been "No". I do not understand why, but it has been a major sticking point for me. More and more, people are developing software for Linux, and it's hard to understand why companies like Rational won't embrace it.

    I'll tell you, it's a major hassle to develop a product for Linux while being forced to debug it on Solaris because that's the only place our debug tools will run. The first company to make decent Linux tools that are similar to Rational's will have my business. And Rational won't.
    • Rational haven't even ported (properly) their products to VS.NET yet... their 'VS.NET compatible' version of Purify is just an external app that you have to call up and load your EXE in, which sucks hard.

      I'm not impressed with their testing software - we paid for Rational Robot (£20,000 - something like 6 months software budget for a small company like ours) and a programmer has been trying to work out how to use it for another 6 months. It still can't test reliably. For that time/money we could have just had the programmer actually doing the damned testing themselves.

  • Applescript, QuicKeys for Mac OS X and OSA (Score:5, Informative)

    by plsuh ( 129598 ) <plsuh@noSpAM.goodeast.com> on Thursday July 25, 2002 @05:23PM (#3954125) Homepage
    AppleScript [apple.com] is great, but the app that you are testing must be scriptable. Many are today, but some are not. Furthermore, in this situation you want to test the GUI, not just the underlying object model which is what AppleScript talks to.

    However, AppleScript is not the only scripting environment for the Mac. Underlying all of the native scripting languages on the Mac is something called the Open Scripting Architecture [apple.com]. This allows any OSA component (i.e. scripting language) to talk to any other OSA component.

    One OSA component is QuicKeys [quickeys.com], a great product that actually does simulate mouse clicks, keystrokes, etc., and is completely scriptable. It integrates completely with AppleScript and via the power of the OSA, with shell scripts and Perl as well.

    On top of this, Apple provides a powerful GUI building tool for AppleScript called AppleScript Studio [apple.com], which is free. Plus, AppleScripts can talk across a network to do RPC. Imagine, you can have an AppleScript Studio front end that drives scripts on a set of machines over the network (so that you can assess performance under load). The individual scripts rely on AppleScript to drive the logic while using QuicKeys sequences to drive the GUI.

    The only issue here is how easily you can assess results. There's no general way to check what the GUI is showing other than a pair of Mark I mod 0 eyeballs, since grabbing data out of the app being tested (even simulating a copy-paste) only talks to the underlying data structures and does not address GUI bugs. A possible way is to arrange the windows in a known manner, turn off the menu bar clock, take a screen shot, and use a graphical comparison tool to XOR the bitmap with a known good screen shot to highlight differences, but this won't work if you're working with variable or randomly generated data.

    Disclaimer: I work for Apple, but these are my opinions only and do not represent any sort of official endorsement.

    --Paul

  • If you want a cross-platform automated testing tool that is:

    1. Localizable

    2. Non-Intrusive (tests the environment on which the user runs).

    Then check out TestQuest [testquest.com].

    While a bit pricey - consider it a capital expense - this system is fantastic. It was originally developed for testing medical devices and systems, where the FDA requires that what you test is what you ship exactly .

  • Although not a software solution, we use Test Quest [testquest.com] to automate our reboot and application execution tests. Basically the hardware in an external controller machine (a second PC) simulates keyboard and mouse actions, along with a bunch of discrete lines that can be set high and low for things like reboot, power switch, and floppy drive. The system can also read back discrete lines for things like "is the system powered on?", and can take a screen capture to compare bitmaps, or do OCR to read results. It's all controlled by windows DLLs that can be written by anybody with a basic understanding of C, or pseudo-automatically written with the TestQuest recorder system.

    They may have other newer systems, or a software solution for now, but for what we do it works great. We only test on PC hardware, but they have a variety of modules and interfaces for touchpads, VGA, svideo, Mac and Sparc hardware as well.
  • Here's a site which will give you opinions and feedback on a wide variety of tools.
    QA Forums [qaforums.com].
    Scroll on down until you get to the Testing Tools forum.
  • Java has a Robot class that does this. It's pretty good, the only downside being that you have to specify locations by the pixel.

  • Check out java.awt.Robot [sun.com].

    ~Kevin
    :)

  • A lot of open source projects could benifit from automated testing. The wine project [winehq.com] has spent nine years working on translating the windows API to X and reproducing the windows GUI.

    Many people [google.com] think wine is a vital tool for greater use of linux. They rely on people downloading new versions and testing windows programs using whatever versions of X, system libraries and windows DLL's they happen to have and then posting bug reports in a newsgroup. [ms-windows.wine] There are still lots of quirks with relativly simple windows programs not redrawing or displaying dialogs correctly.

    I believe that wine would progress much faster if the wine developers could make a change then click test and later, with no more effort get a report saying "3 tests which failed previously now pass, 1 test which passed on the last version now fails".

    I will be very impressed if somone comes up with a free automated testing tool that allows tests to be set up easily and is powerfull enough to provide human readable results "the bitmap is drawn 1 pixel to the left", "text as expected but wrong font" "dialog appeared behind other window" etc.

  • DejaGnu is a TCL/Expect testing harness provided as free software by the Gnu Project. Not sure if it has tk hooks (which would allow you to test X-based programs) so you might want to check. If you're trying to test programs on the postscript display of a Mac instead of X, well, I don't know if it would work. Also, check to be sure there is a recent version of TCL/TK and Expect for your machine before you pursue this too far.

    There is also LTP, the Linux Test Project, which may be of use to you as well. (Do a Google search for "linux test" and you'll find it quick.)

    If you're not a TCL programmer, you're probably not going to like DejaGnu very much. If this is the case, good luck with the GUI tool search; you can expect to spend some serious money for decent test tools. (The pun is simply there. If you look at it too long or pay too much attention to it, it may or may not vanish. Just chalk it up to some crazy cat and don't worry about it.)

    Best wishes,

  • Expect and Xtest have already been mentioned.
    Android runs on top of these and can be downloaded free for Linux and Unix

    See the July 2001 issue of "Dr. Dobb's Journal"
    for an overview of Android and how to download.

    Plays back keyboard and mouse events for most UNIX implementations. (maybe even Mac OS X)
    It is a little bit roll-your-own
    Expect to spend some time writing perl to support your android scripts.

  • ARexx used to do this on my Amiga back in '89. I remember that to launch applications using tooltypes defined in the icons (and to avoid launching via command line tooltypes in the startup-sequence), I ran an ARexx script that double-clicked on my drive, double-clicked on the Applications folder, and launched the apps I wanted running after I logged in. The script would then place focus in a new CLI window. All by scripting mouse events.

    *Sniffle, wipes tear from eye* Ah, the good old days.

  • WinRunner (Score:2, Informative)

    by jishak ( 571556 )
    My company uses WinRunner for Windows GUI testing. I know that Mercury Interactive which makes WinRunner and LoadRunner another GUI Testing utility creates Unix version of their software in addition to the Windows versions. I don't know about Mac though.
    Mercury Interactive [mercuryinteractive.com]
    Hope that helps
    Joe
    • Ugh!!

      Don't use WinRunner..

      We use it at work (actually, I just got back from spending the entire day working with the software). It works perfectly until you try to "replay" something you have "recorded". Then all hell breaks loose.

      They use a kind of "property map" which they use to locate GUI objects. Problem is, the GUI map keeps changing on you because the default properties they pick vary from load to load (especially with the Java add-in, IMHO). Also, it can be very tricky (i.e. almost impossible) to find the right set of properties that will uniquely identify your UI objects, and that won't change from load to load, so that you can successfully regress your software.

      Blechhk! Add to that, it costs an arm and a leg.

      If I had to make the decision all over again, I would say skip the UI testing, and do unit testing on your classes instead. Then hire some monkeys (verification types) to sanitize the UI part on a semi-regular basis.
  • Rational suite, including tests was a piece of crap when we worked with it. We switched to XP and everyone started having great time coding, due to allowance of concurrent programming...
    If you want to get job done right, I would recommend eXtreeme Programming, over most other ideologies.
    2c,
    p.
  • I'm not completely sure about this, but I think you can use ATK (accessability toolkit) for sending events and getting widget contents to GTK apps. That works only onn GTK2, though, so perhaps it's not helpful to you, but at least I think the functionality is there.

  • Gerd? (Score:1)

    by Gerald ( 9696 )
    Tim Janik came out with a utility called Gerd [gtk.org] a while back that lets you script GTK+ applications. It appears to have been abandoned at version 0.0.3, however.

Slashdot Top Deals

Neutrinos have bad breadth.

Close