Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 



Forgot your password?
typodupeerror
Programming Software

Ask Slashdot: Best Test Case Manager Plugin For JIRA? 70

Bomarc writes: I've been working with software testing ... for a few years now. And there seems to be a serious lack of QA — Test Case Management (TCM) tools. The company that I'm working for needs a good test case manager. Currently JIRA is the tool of choice for other aspects of project management. I'm not asking to jump ship from JIRA, but the Atlassian TCM "Zephyr" has several problems, some of the key ones include: It does not have (any) matrix capabilities, no test case suite capabilities, if you change one test case (including assignments) the system changes all of the runs from that test case, the integration between the defect tracker and the TCM is archaic (at best), the number of actions to pass/fail a step (or test case) are annoying (way to many). Whoever designed it doesn't use it. If you watch the "Introduction" for Zephyr – it is amusing to see how the person performing he demo skips over and fumbles when dealing with the flaws I've mentioned above.

I have used the product "TestLog" which is a well-thought-out product; has test matrix capabilities (and other good features); however it does not have any integration with JIRA. (Hint, hint: Atlassian, this is what you need!).

Is there any company that makes a "plug-in" for JIRA with a similar features to TestLog – test case management that is well thought out, not just an afterthought?
This discussion has been archived. No new comments can be posted.

Ask Slashdot: Best Test Case Manager Plugin For JIRA?

Comments Filter:
  • Enterprise Tester (Score:5, Informative)

    by Lunix Nutcase ( 1092239 ) on Sunday May 31, 2015 @12:41PM (#49809869)

    The last few places I've worked that use Jira for have used Enterprise Tester. Seems to do an okay job for the QA people.

    http://catchsoftware.com/testi... [catchsoftware.com]

    • In what way is my post a "troll"?

  • Make your own (Score:3, Informative)

    by neumayr ( 819083 ) on Sunday May 31, 2015 @01:16PM (#49809961)

    Okay, Testlog looks like an awful proprietary Windows-native tool. But if you can find a TCM with your required feature set and a documented API, it shouldn't be too big of a deal to make an own Jira plugin that interacts with that hypothetical TCM.
    Maybe Testrail has those features?

    • by mridoni ( 228377 )

      It might be proprietary and Windows-only, but a single license costs US$ 99, I doubt that this money (or even the money for a dozen of licenses) would cover the costs associated with developing a custom in-house tool.

      • by neumayr ( 819083 )

        I'm not talking about a whole tool, "just" an interface. Still, it'll be more than 99, definitely.

  • by war4peace ( 1628283 ) on Sunday May 31, 2015 @01:25PM (#49809993)

    I have been testing TFS and got a parsing error at "(way to many)".

  • how about (Score:2, Offtopic)

    by wbr1 ( 2538558 )
    How about a test case manager for the crooks at dice? Stealing projucts, beta, and now pop overs? Fuck you
  • We're just getting ready to publish or Test Case Management plugin to the Atlassian marketplace. I can provide trial installation packages to people interested in trying it out. Collaborative editing, tight integration with Jira, great UI. https://www.evernote.com/l/AAL... [evernote.com] Comment if interested, I'll set up trial link.
    • by Ash-Fox ( 726320 )

      tight integration with Jira

      Does this mean I can use JQL with the test cases and produce pretty reports and graphs using JIRA's regular built in dashboards?

  • TestRail (Score:4, Interesting)

    by ThomK ( 194273 ) on Sunday May 31, 2015 @01:58PM (#49810131) Homepage Journal

    We use TestRail (http://www.gurock.com/testrail/) with much success. It integrates with Jira (or not, up to you) and has a very simple and intuitive interface. I am a former Zephyr user and that experience is, in part, why I ended up with Testrail. I'm surprised it hasn't been mentioned yet.

    Good luck.

    • by Ash-Fox ( 726320 )

      When I last used TestRail, it did not actually expose it's data to JIRA directly, therefore making anything like JQL queries useless which made JIRA dashboard and JIRA reporting integration non-existent.

  • After a long break in research I've been doing client work again. This client is pretty big, a small European airline company. For some reason they have a lot of trouble getting the Jira suite of products to run stable. Stash is offline complete afternoons. I find this quite bizarre. But is this really the Jira software, or does it have to do with the client's sysadmin team?

    • by quetwo ( 1203948 )

      We use JIRA at Apache for all the Apache projects. I wouldn't say it's the best software in the world, but it is better than most of the other ones I've personally used. They don't seem to have the stability issues you listed . Some of my direct clients use JIRA as well, and they have never mentioned stability issues with it either...

    • by neumayr ( 819083 )

      First off, it's not a either the software or the sysadmins. Lots of things can break in between, I suppose that's especially true when there are strict processes in place, as I imagine the workings of an airline company.
      That said, I've worked with Atlassian products (lots of Jira, Confluence, and Fisheye, but only a little Stash), and while I'm not a big fan (I find them bloated), I can't complain about their stability. Our admin team wasn't especially large, one could say understaffed, so I doubt they coul

    • I've found Atlassian's products to be great, but the latency when used from Europe (at least Norway) is so bad that there is just no way for us to use it :-( It's not always slow, but at least for some hours of the day we're talking 4-20 seconds before a page refreshes. We have a confluence site up that nobody uses just because of this issue. I know we could host it ourselves, but I have neither the resources nor the patience (Jira seems to need a lot of tlc to keep running).
      • by plopez ( 54068 )

        Not my experience with their wiki where searching and markups are a joke. Stash seems to work better but I would really like to ditch their wiki.

      • I've found Atlassian's products to be great, but the latency when used from Europe (at least Norway) is so bad that there is just no way for us to use it :-( It's not always slow, but at least for some hours of the day we're talking 4-20 seconds before a page refreshes. We have a confluence site up that nobody uses just because of this issue. I know we could host it ourselves, but I have neither the resources nor the patience (Jira seems to need a lot of tlc to keep running).

        Here in the UK we don't have that same problem using their hosted JIRA, so probably this is local issue to you guys in Norway.

    • by vanye ( 7120 )

      We use JIRA and Confluence OnDemand and Stash on-premise (to replace their subversion hosting we initially used).

      We evaluated gitlab before deciding to go with Stash - based largely on JIRA reputation.

      I found stash to need way more memory than they claimed was needed

      I think the VM is configured for 16GB - it was 2GB initially.

      We are using the built in database - primarily because I can't be bothered to worry about getting a real one setup.

      Our main repository is about 800MB - ~8000 commits

    • by radish ( 98371 )

      We use JIRA & Confluence. I'd say they were pretty stable normally, but upgrades seem to have problems more often than I'd expect, particularly when you have a bunch of plugins installed.

    • by Ash-Fox ( 726320 )

      But is this really the Jira software, or does it have to do with the client's sysadmin team?

      The software can be a pain at times, but it's certainly not problematic as you described usually and this is JIRA experience taken from large multi-national firms.

    • by Hallow ( 2706 )

      I have had to run Atlassian products a few places. They're much better than some of the more proprietary packages on the market. The only problems I ever had were due to running on windows virtual machines, and an overtaxed db server. Pretty much all their stuff runs much better on real hardware, running Linux.

  • by sheetsda ( 230887 ) <[moc.liamg] [ta] [steehs.guod]> on Sunday May 31, 2015 @02:44PM (#49810331)

    I understand that TestTrack TCM [seapine.com] has been integrated with JIRA at some shops.

    Full disclosure: I work on this product.

  • by Anonymous Coward

    Try ReqTest. As a developer I mostly use their bug tracker, but their test management seem competent and it has Jira integration.

  • by flippet ( 582344 ) on Sunday May 31, 2015 @03:35PM (#49810565) Homepage

    We use RMsis [optimizory.com]; it's not the slickest interface in the whole world but it does everything we (medium size engineering company) want to do so far.

    The people who support it are very good, and seem to be very amenable to requests for features and support.

  • We switched a couple years ago to YouTrack from JIRA and have been quite happy with it. JIRA is a load of shit.

    • by Anonymous Coward

      As someone who's had to support Jira at the backend, YouTrack looks a LOT more mature purely from the upgrade procedure on their website.

      The Jira upgrade is generally a multipage spaghetti process which I have never seen someone get right the first time, and these are smart people who operate complex systems on a regular basis...

  • We use standalone Zephyr in conjunction with JIRA, Confluence (both hosted OnDemand) and Stash.

    What I can't work out is the justification for their pricing.

    I pay Atlasssian about $10 per user per month.

    Zephyr wants $80 per user per month, others mentioned are $25 per user per month.

    Is test management really worth 5-10times as much as I pay for bug and content management ? Or 1000 times as much as I pay for source code and code review ? (Stash is $10 for a year for 10 users)

    No.

    So we continue to use the Commu

  • ...if this Slashvertisement would tell us what the hell JIRA is!

  • Test cases have little to no place in Agile development. They are the finest example of waste. I know that a lot of places still feel that more test cases equals better quality, that is clearly not the case. Quality is derived from process, which can involve test cases, but again test cases are very inefficient at catching and preventing bugs. I have evaluated multiple solutions and they are all lacking.

    While I do advocate some Scenario based testing, which usually are a single line like "Can you place an
    • by plopez ( 54068 )

      How do you feel about Behave, Spock, Cucumber, and Thucydides? My experience is that they work well with behavior driven testing. We are testing both APIs and integrating with Selenium using these tool sets.

      My major gripe with automated testing is that it is just regression testing. It does not give you new information. Automation and exploratory testing are often at odds with each other. And management wants pretty reports and test case numbers and so carve out little time for exploratory testing.

      • by Ash-Fox ( 726320 )

        My major gripe with automated testing is that it is just regression testing. It does not give you new information.

        You are correct and it has other flaws such as pesticide paradox.

        And management wants pretty reports and test case numbers and so carve out little time for exploratory testing.

        Take a look at session based testing, which provides lots of pretty metrics that are understandable. James Bach wrote a considerable amount of information on why reporting things like test case numbers and relying on how

    • by Ash-Fox ( 726320 )

      Test cases have little to no place in Agile development.

      But they do when certain auditing requirements are involved. I do believe that any step based tests should be automated however and a better approach where auditing requirements are not involved would be to make use of developers doing unit testing and coverage using methodology like test/behaviour driven development.

      While I do advocate some Scenario based testing, which usually are a single line like "Can you place an order", traditional test cases ar

  • Didn't you get the memo?

  • I'm really sorry, but the best test case management plugin for Jira is Zephyr.

    My suggestion is that you actually better more flexibility out of creating custom issue types with fields and for your test cases and depend on issue linkage to handle things like traceability matrixes.

    I would suggest adding the following text field types:

    Steps To Complete
    Expected Outcome
    Actual Outcome

    Ensure to build whether the test case succeeds or fails into the work flow for the ticket type. You can then create pretty dashboar

  • I would like to point you at Klaros-Testmanagement [testmanagement.com] which features a close integration with Jira even in the free Community Edition. Working on the implementation of it I am of course biased.

To do two things at once is to do neither. -- Publilius Syrus

Working...