Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
GUI Software Operating Systems Unix

Conducting a Unix Desktop Usability Study? 313

Posted by Cliff from the not-something-you'd-generally-find-in-the-HOWTO-db dept.
cyclop asks: "I am a close friend of a Ph.D. student on human interface usability. She's now working to tailor a KDE-vs-Gnome usability study (a pretty hot topic these days), and I have been called to help, as a long time GNU/Linux desktop user. What kind of advice -- both technical and theoretical -- would you give us on conducting a deep and objective study on the Unix desktop, that can be useful for the developers and the OSS community?"
"She has installed GNU/Linux and used both KDE and Gnome to get to know them, while I provided her a number of links on background information and previous usability studies on both DE, and advised her to subscribe to relevant mailing lists of both projects. However, I feel that it's not enough and that there are a lot of potential pitfalls and misconceptions that wait for us, me being a geek and she being a Linux newbie. Moreover, she found that most of the previous studies on the web were quite sloppy, in comparison with common usability research standards."
This discussion has been archived. No new comments can be posted.

Conducting a Unix Desktop Usability Study? More

Conducting a Unix Desktop Usability Study?

Comments Filter:

  • Allow power-users to tweak settings first. (Score:4, Insightful)

    by Anonymous Coward on Thursday December 15, 2005 @08:08PM (#14268689)
    No windowing interface is as efficient as it can be out of the box. (for example, for some use-cases, In the Windows world that usually means making things like the File viewer actually show you the files and extentions).


    The out-of-the-box setup is a compromise at best; and shouldn't be used to judge the overall usability for people who use the system more than once.

    • Well, it would make a comparison useless IMHO.

      I think we should go for defaults instead. I feel the distro closest to vanilla desktop settings are Gentoo and Debian, we'll probably run one of these two, but if you have advices please tell me.

      • "Generic" defaults, like Gentoo or Debian, don't cut it for this.

        You don't want to compare GNOME to KDE to Windows because GNOME and KDE aren't operating systems. You should compare *Fedora* against *Ubuntu* against *Mac OS* against *Windows*.

        Fedora and Ubuntu make customizations to GNOME because they feel they are doing a better job of usability than upstream. Fine, let them. Tell us how they compare.

        No one (sane) will give a newbie an uncustomized Gentoo box or Debian setup, so compare realistic things
      • Personally, I consider the tweakability of an IDE to be a key aspect of the usability itself, so if you make me use the defaults, I probably couldn't tell you whether I liked it or not.

    • Tainted vs Ignorant users. (Score:5, Insightful)

      by khasim ( 1285 ) <brandioch.conner@gmail.com> on Thursday December 15, 2005 @08:31PM (#14268840)
      The first thing to understand is that you will have 3 groups of users:

      #1. The ignorant users: These have never used a desktop before. These aren't as easy to find anymore. I worked with one woman back in the mid 90's who could not even use a mouse. She had to hold it still with one hand while she clicked the button with her other hand. After a week of solitare, she had the necessary muscle coordination to start learning the system.

      #2. The tainted users: These have experience with systems other than the one you're testing. If your system isn't 100% like the one they're used to, they'll waste time clicking around where the functions are on their systems.

      #3. Friends: These have worked on the system that they're being evaluated on.

      Now, a system that is easy to learn for the "Ignorant" class may be incredibly un-friendly for more advanced "Friends".

      Determine what functionality you want to measure and what GROUP you want to measure it for.

      The real "ease" on an interface comes down to 2 things:
      a. Can you quickly guess where a function is based upon your existing experience with it?

      b. Once you know where a function is (you guessed at it before, you asked someone, you went to training), how easy is it to remember that 24 hours later, 1 week, 1 month, 6 months later?
      • c. And once you've used the interface for months, can you use the function quickly and efficiently?
      • >#1. The ignorant users: These have never used a desktop before.

        I totally disagree with this. what percentage of your potential audience has "never used a desktop before". Very close to zero, and getting lower all the time. MSFT did do one thing, and that was teach everday people what a desktop was, and how to more-or-less get around a computer.

        There is no reason to try to design anything for this class of user, it is such a small portion of the populace that it can be said to be zero.

        • In the USofA, that is mostly correct. (Score:4, Insightful)

          by khasim ( 1285 ) <brandioch.conner@gmail.com> on Thursday December 15, 2005 @09:31PM (#14269179)
          But there are lots of countries with people who still haven't used computers.

          Not to mention that an interface with settings 1-10 (1=brand new user, 10=expert) would make a lot of non-experts more comfortable with their computers.

          An ideal interface would evolve with the user's experience level. Not trap an experienced user with a pre-school interface nor confuse a new user with expert-level options.

          Tailor the choices available to the level of the user and let the USER choose how complex the interface he uses is. Just like books. When you started reading, you didn't read the books you read today. Those books followed very careful patterns on what words were used and how often they were repeated.

          But since none of the interfaces out there are doing that yet, it really doesn't matter for this discussion.

        • Re:Tainted vs Ignorant users. (Score:4, Informative)

          by Jon-o ( 17981 ) on Thursday December 15, 2005 @11:20PM (#14269640) Homepage
          I would add that there are loads of people that use computers every day, but have never learned the basics of the interface they see all the time. I saw someone at work today who, despite doing most of her work in Outlook, and having many folders of e-mail, had no idea that you could collapse and expand the folder tree. If you only learn the tasks as a step-by-step set of actions, and don't learn how to apply those steps to any other tasks, then you really don't know the desktop you use. I think there are a LOT of people that fit this category.
          • Exactly.
            My sister's owned a computer since 1998. She's a whiz when it comes to email, using Word, or manipulating her photos in Photoshop. Her first machine was a Gateway monstrosity that she paid way too much for. Now she's got a G5.

            A couple of months back, she was absolutely flabbergasted when she saw me switching between open windows... all this time she's had no idea that one can minimize a window to the taskbar and continue on in another.

      • You missed one last (Score:5, Interesting)

        by DrYak ( 748999 ) on Thursday December 15, 2005 @09:55PM (#14269278) Homepage
        #4. I have no social life : These have seen so much different systems that they understand well the general concept and can adapt themself very quickly to whatever system you throw at them. As open to new solutions as #1, but very quickly get as agile to whatever system as #3 is.
        (In fact that was my case when I started learning both KDE and Windows 95 in parallel)

        The best system ever should be as easy as possible for #1 out of the box, but need to be very easily configurable to whatever complex system #3 and #4 need.

        If the system is newbie-friendly but can't evolve you'll end with Clippy and this kind of stuff that gets in the user's way with pointless tips (tips that would have helped a total beginner, but sorry now I know exactly what I want). Attracts #1 users, but repels #3 and #4.

        If the system is configurable to extreeme you end up with emacs or vi : the most powerfull tool around you can't ever dream of in your wildest dreams, but you can't do anything without unless you've spent the first year learning it the harsh way. #3 only are interested, #4 must ponder if they want to re-learn everything once again (albeit they do it faster), #1 will prefer to commit suicide.

        #2 are pointless, they won't accept anything that isn't their original system, they're the one that will never switch to MacOSX or Linux because it's not Windows+Office, and they'll cry each time MicroSoft revamps the interface and everything is moved around (Windows 3.11 -> Windows 9x -> Windows 2k -> Windows XP -> Windows Vista and same for the Offices). Just wait until the next "GUI is completly changed one more time" period, and they'll be as good as #1 users (or #4 if it's not their first change around).

        • Re:You missed one last (Score:3, Insightful)

          by Raphael ( 18701 )

          #2 are pointless, they won't accept anything that isn't their original system, they're the one that will never switch to MacOSX or Linux because it's not Windows+Office

          I disagree. Group #2 is probably the most important group. Keep in mind that being tainted does not only mean being exposed to Windows+Office, but also to any other desktop environment, including Linux, OS X or others.

          This includes for instance the Linux FVWM users who will be frustrated that the default GNOME window manager does not

    • I think it would be enlightening to see it both ways.

      But for the tweaking, I would not answer questions about how you can set up things a certain way.

      You could even conduct a usability study on how easy it is to change the settings; in that case, you *should* start with defaults.

  • Simple (Score:5, Insightful)

    by captain_craptacular ( 580116 ) on Thursday December 15, 2005 @08:10PM (#14268699)
    Get a slashdot poll on the topic and read the insightful comments.

  • Good Lord (Score:2, Insightful)

    by Anonymous Coward

    a deep and objective study on the Unix desktop

    Ha. Ha ha. Ha ha ha heh heh hee hee hee hee ha ha HAA HAA HAA HAAA HARRRRRR. You might as well do a study on whether apples are better than oranges, or settle the One True God question once and for all. It's hardly possible to do a deep and objective study on the merits of the Linux kernel vs. the FreeBSD kernel even though that's reducible to a purely technical inquiry.

    It *does* sound like perfect academic paper fodder.

  • for meaningful results... (Score:5, Interesting)

    by smash ( 1351 ) on Thursday December 15, 2005 @08:11PM (#14268711) Homepage Journal
    I'd ensure first up that the study runs for "long enough".

    3-6 months perhaps?

    "Usability" imho, in the usual meaning of the term, is a load of wank.

    Who cares if the first time someone uses the environment that it takes a little orientation to get used to? In the real world, if a couple of weeks of pain makes you much more productive after that, it's a net benefit imho - the remainder of your time using the environment outweighs the significance of the learning time.

    I'm not saying that initial learning is not important, but I think that these studies need to show both sides of the equation...

    smash.

    • An occasionally used website (like an airline) or kiosk (photo stickers?) must be useable on first contact.

      A work desktop must be quickly learnable and facilitate productivity of intermediate to advanced users.

    • Learning should be built into the system. (Score:4, Insightful)

      by khasim ( 1285 ) <brandioch.conner@gmail.com> on Thursday December 15, 2005 @08:37PM (#14268890)
      IMHO - the "best" interface would be one that starts off at a child's level and allows the user to set the degree of functionality and complexity based upon his/her knowledge and needs.

      How about, how many repetitions of instruction does it take for an average user to remember how to perform one action after a week of not using it?

      And, once one function is explained, how quickly can the average user deduce/guess at related functions? This is how you select "bold" text. Then let them find "italics" and "underline".
    • Most usability test would propably rank f.ex emacs pretty low, but when you compare the speed at which a veteran with emacs can code compared to a veteran of MSVC or the like...

      3-6 months isn't enough, 3-6 years sound much more resonable.

    • Exactly. On the face of it, KDE and GNOME can appear to be similar to some people. You may even dislike KDE's Qt interface for a while until you get used to it. However, KDE has amazingly powerful technology underneath. Spend a few months doing all your work on KDE, giving yourself time to discover (discovery learning is good) its hidden strengths, and you'll never look back.

      I don't often agree with Linus, but what he said about GNOME and KDE was spot-on: when you take the time to try them both, you'll s

  • My thoughts (Score:5, Insightful)

    by gid13 ( 620803 ) on Thursday December 15, 2005 @08:13PM (#14268721)
    Get people who are not experts, see how many problems they run into doing simple tasks that they're familiar with on Windows. See how many of these they can solve themselves. Start half of them on Gnome and move them to KDE, do the other half in the reverse order.

    It is probably also worth noting that most people (apparently including Linus) consider KDE more powerful, so KDE is kinda at a disadvantage.

    • Re:My thoughts (Score:5, Interesting)

      by Anonymous Coward on Thursday December 15, 2005 @08:35PM (#14268876)
      It might be better to test it on childern.

      most people have used windows at one time and expect things to have that type of layout.

      children who havent gotten used to what windows is like might find it a bit easier/harder to move around in.

      you could have 2 groups of children

      set one group of children to use gnome for the first week/month/year and kde for the second week/month/year whatever
      and set the other group to use kde for the first week/month/year and gnome for the second week/month/year

      and compare there reviews of how easy it is to move around.

      however it might be better to test it out on teenagers are they will be able to take more infomation in.

      • Re:My thoughts (Score:3, Insightful)

        by EvanED ( 569694 )
        I think testing on children, while it would give interesting results, isn't what they're aiming for. In the real world most people WILL be moving from Windows, so if you're interested in doing this study to help, for instance, distributions or new users choose between them, or help the developers fix problems, this is what you should test.

        Also, a lot of the tasks you'd be interested in testing a child wouldn't know about. If you were to ask, for instance, the user to write a formal letter in a word processo
    • Linus seemed to be saying that he wants all the functions exposed and available. That's why he didn't like the GNOME approach of hiding what they considered "un-necessary" options.

      But ... for a user who is unfamiliar with the system, a stipped down interface with only the functions that they'll be using would be the easiest to learn.

      We have people at work who are really scared that they'll do something wrong with their computers. If ANYTHING changes, they need to be walked through it for a few days.
      • So what GNOME really needs is a means of ramping novice users up to power users. Unfortunately, this is sadly lacking. For instance, let's try navigating in nautilus as a power user:

        Open Parent -> Alt+Up
        Open Location -> Ctrl+L
        Close Parent Folders -> Ctrl+Shift+W
        Close All Folders -> Ctrl+Q
        Close -> Ctrl+W
        Home Folder -> Alt+Home

        Notice that we're using Alt key combinations, Ctrl key combinations, and Ctrl+Shift key combinations. My biggest problem with using nautilus effectively is mixing u

  • The Nipple? (Score:3, Funny)

    by mukund ( 163654 ) on Thursday December 15, 2005 @08:14PM (#14268722) Homepage
    You've got a lady friend and you have been called to help on conducting a deep and objective human interface study on the desktop?

    Go for it!

    (If you're wondering about the subject of this comment, the nipple is one of the most intuitive human interfaces btw).

  • Long-Term Efficiency (Score:5, Insightful)

    by Anti-Trend ( 857000 ) on Thursday December 15, 2005 @08:17PM (#14268741) Homepage Journal
    I for one would like to see a study involving not just how easy it is to learn an interface, but once learned how productive one can be in said interface. For instance, I am proficient in both KDE and Gnome (and a myriad of other WMs which aren't mentioned here), but I feel I can get the most work done faster in KDE. Of course I do tweak quite a few aspects of KDE, but I digress. I would really like to see a productivity evaluation between already proficient users, confident with their skills on their respective interfaces, performing a series of common tasks and comparing the results.
    • My thoughts exactly.

      I find the gnome interface a little "cleaner", but several things in KDE just work better for me. The KDE IOslaves (fish, etc) just rock - excellent for web development, etc - they're a huge productivity boost for me.

      Ripping CDs in whatever format with full CDDB support etc with drag and drop to another folder just rocks.

      Having said that, right now at home I'm running ubuntu 5.10 with gnome. I find it less cluttered to navigate, but in terms of actual application use, right now K

    • Re:Long-Term Efficiency (Score:3, Insightful)

      by JanneM ( 7445 )
      For me it's the exact opposite. I'm a lot more happy in Gnome; it doesn't get in my way. From the default Ubuntu installation I don't need to tweak anything desktop related, it all just works.

      As for doing a usability study, you first have to decide what to measure, then to decide what your users are. "everything" and "everybody" are hopeless non-answers.

      On what to measure, you could focus on several things: time spent dealing with the desktop rather than your work; number of desktop related problems run int

  • My advice? (Score:5, Insightful)

    by rampant mac ( 561036 ) on Thursday December 15, 2005 @08:20PM (#14268757)
    "I have been called to help, as a long time GNU/Linux desktop user."

    My advice? Don't have someone who's been a long time GNU/Linux user assisting her. Chances are, you're fond of either KDE or Gnome. Before the study has even started, I'm alarmed by potential bias. Let her do the study on her own, gather the facts and come up with her own conclusion. Isn't that what Ph.D.'s do?

    "[...] while I provided her a number of links on background information and previous usability studies on both DE, and advised her to subscribe to relevant mailing lists of both projects."

    To me, the study is already flawed. You've dropped a load of information onto her lap, while a complete "newbie" doesn't have that same luxury. How can a usability study be unbiased in this manner? Who's to say you didn't provide her with REALLY good links to KDE information, while giving half-assed links to Gnome?

    • Re:My advice? (Score:4, Informative)

      by cyclop ( 780354 ) on Thursday December 15, 2005 @08:30PM (#14268827) Homepage Journal

      You didn't understand. She will conduct the study, but that's not she will judge what's more usable and what's not. This would not be a usability study, it would be a -1,Flamebait article. She plans instead to put categories like WinXP-proficient people,MacOSX-proficient people and total computer noobs (if any still exists) in front of Unix desktop enviroments and see their reactions and if and how they can be proficient with them. She's using them to understand them and for obvious curiosity, and I gave her info to help her tailor the study.

      • Not good. (Score:3, Informative)

        by khasim ( 1285 )

        She plans instead to put categories like WinXP-proficient people,MacOSX-proficient people and total computer noobs (if any still exists) in front of Unix desktop enviroments and see their reactions and if and how they can be proficient with them.

        No. All she is "testing" there is how closely the desktop they're being "tested" on resembles the one they're used to.

        Novell did this already.
        http://linux.slashdot.org/article.pl?sid=05/10/11/ 146202&tid=223&tid=106 [slashdot.org]

        So, to send email ... where's Outlook? Wh

    • To me, the study is already flawed. You've dropped a load of information onto her lap, while a complete "newbie" doesn't have that same luxury. How can a usability study be unbiased in this manner? Who's to say you didn't provide her with REALLY good links to KDE information, while giving half-assed links to Gnome?

      No, thats actually a good start. User X gathered some information from various links and tries to work with this little knowledge. Its the same in Windows. There are tons of Windows help sites, ti
    • your girlfriend needs to become an expert in _all_ systems,
      in order to make a comparison.

      she needs to _locate_ suitable dumb-idiots who haven't been thingied.
      biased. and intelligent people. etc.

      but she needs to _become_ a geek - to know the pitfalls and
      advantages of the various setups - in order to not _accidentally_
      introduce bias.

      if she were one of the _subjects_ of the test, that would be
      a different matter.

    • Re:My advice? (Score:4, Insightful)

      by Froggy ( 92010 ) on Friday December 16, 2005 @03:44AM (#14270439) Homepage
      To me, the study is already flawed. You've dropped a load of information onto her lap, while a complete "newbie" doesn't have that same luxury. How can a usability study be unbiased in this manner? Who's to say you didn't provide her with REALLY good links to KDE information, while giving half-assed links to Gnome?

      The researcher won't be a subject. You can't do a usability study that way. You need to recruit a bunch of people who match the kind(s) of user you're studying, get them to do a range of tasks, and observe various aspects of their performance. If you're your own subject, you're not doing research. You're just airing your opinion.

      Speaking as a PhD student, one of the most important things we are expected to do is a literature survey. That means we have to go out and read studies that are relevant to our research topic, and critique them. If the researcher fails to discuss them, her supervisors should ding her for not having done her reading. If she can't judge the worth of the studies for herself, she's not working at PhD level yet. She should have a good grasp of what constitutes good research by now.

      I know I'd be thrilled to have a load of pointers to relevant studies dropped in my lap. I'll judge their degree of assedness myself, thank you very much.

  • objectivity schmuctivity (Score:5, Insightful)

    by abes ( 82351 ) on Thursday December 15, 2005 @08:22PM (#14268779) Homepage
    It's not clear that one can easily do an objective study on usability, as it can mean very different things to different people. It should at least be done with segregated populations (e.g. power-users vs. novices).

    Some examples:
    * A novice might look for how obvious it is to do a certain task, whereas an expert user might instead prefer what can be done fastest (e.g. notepad vs. emacs).

    * Related: How much time does this person use a computer/this application can be an important factor. If I rarely do 3d design, I want to be told how to do everything, and have obvious controls (i.e. > 3 parameters might boggle my mind). However, if I work for Pixar, the verbose messages, and dumbed down controls (i.e. 30 parameters might just not cut it for what needs to be done).

    * Certain paradigms might make sense to people who are used to using certain types of systems. Files and folders make perfect sense to many people, but certainly not to everyone (e.g. my mother). We think these simplified analogies work better for novices, but that isn't always the case. People think differently, and different analogies will make more/less sense dependent on their world view.

  • Some suggestions (Score:3, Interesting)

    by ardor ( 673957 ) on Thursday December 15, 2005 @08:23PM (#14268780)
    *) Get total PC newbies and Windows users to try using them and observe what they do, what they try, how and why they fail to succeed in their attempts. Let them talk verbosely about what they are doing.

    *) Count the times you had to look in a manpage, in google, and how often you had to fire up a console for doing simple things (like creating a shared folder, browsing the internet, installing some plugins like flash etc.) Keep in mind: SIMPLE things! Trivial tasks done by the casual user.

    *) For each system you need to learn how to use it, thats a fact. Unix users have to learn the concept behind the filesystem (nothing too fancy, but basic knowledge about what mounting is for example). This is comparable to the knowledge about the drive letters in Windows, the usage of backslash for separation in paths, that .exe are apps etc.

    *) Review the help system and documentation. Among other things, look for technical mumbo-jumbo. This is a common error. Stuff like SSH, SSL, CORBA, FUSE, pthreads etc. should never occur in enduser documentation.

    *) Have a look at the menus. Are they cluttered or usable? How long did you have to search something in the menu?

  • don't do it! (Score:5, Insightful)

    by schwaang ( 667808 ) on Thursday December 15, 2005 @08:24PM (#14268787)
    But if you must, at least don't do KDE vs. Gnome. What's the best possible outcome of that? ("So in summary, Gnome tended to be less confusing for newbies, but power users preferred the configurability of KDE...")

    Instead compare either or both against Windows or Macintosh for tasks that your _specific target userbase_ would do. [If you haven't defined one or more use cases you've already lost.] This would be much more valuable.

    Better yet, switch your topic to focus exclusively on accessibility (a11y). Every DE out there needs some accessibility love.
    • In fact KDE and Gnome are kind of "subdistributions", the kernel is another.

      The problems of Linux are usually not on that "KDE or Gnome" level anymore. KDE serves all my needs, I also like Gnome and I am more productive than on Windows on both.

      The comparison is flawed because, I mean, think of webbrowsing. I use Konqui, Gnome users would use Epiphany or Galeon. But here I currently use Firefox on KDE.
      So what use has a KDE vs. Gnome comparison here?

      What me annoys on Windows is that opening a PDF in Acrobat o
  • Personally I think Apple has done a quite decent job of building a GUI on top of a UNIX core (the Darwin flavor of freeBSD).

    Currently they have it working on two different processor families (the IBM Power series, and Intel).

    Yes, it is proprietary, but that does NOT mean that "Aqua" is not a GUI desktop running on a UNIX system.

    Why not compare the other UNIX desktops with what may be the best UNIX desktop running?

    Don't get excited, it's just an honest question. After all, just because it was done by a comme
    • IMHO KDE (3.4) is better than Aqua. But that's for the tasks that I do. YMMV.
    • > Personally I think Apple has done a quite decent job of building a GUI on top of a UNIX
      > core (the Darwin flavor of freeBSD).

      Well, this is no real option.

      Because for Apple users it is irrelevant whether it is build on foosys or Unix. When you run a C64 emulator on Linux which is distributed as a game console to play games you cannot say "this is a real Linux" only because a techie can open a linux console. For the average use it is C64. And for the manufacturer it it was a rather technical choice wh
  • Worked perfectly. Email, Internet, Applix Word, Spreadsheets, etc. Putting KDE with OpenOffice on it would only make it that much more useable and interesting.

    Frankly, I don't see why this is needed...people have been using it for years.

    There...Can I please have my doctorate now?
  • Try questioning the slashdot community. They're sure to offer up objective advice on the KDE vs. Gnome debate!

    *cough!*

  • You want an unbiased opinion?

    Don't ask us.

    For one its not GNU/Linux Desktop. It's KDE vs GNOME.

    KDE runs under BSD as well as Linux. Gnome runs under Solaris.

    Of course the GNU people probably want to start calling *BSD running GNOME GNU/*BSD because of all the GNU code in it.

    Oh, wait. Its not that popular.

  • Comment removed based on user account deletion
    • Yeah, thats true. manpages are NOT for endusers. Definitely. I do like Microsofts CHM format a lot, but I don't know if it can be used freely. Then there is DocBook, which could be used for a standard documentation. For Linux, maybe a modular documentation system would do fine. Basic documentation common to all distros, and on top of that KDE documentation, K3B docs, KDevelop, Konqueror etc. as modules. AFAIK only devhelp features this, but devhelp is very gnome-centric.
      • At least you have to make sure that all KDE applications have man pages. This is a quality problem. The documentation can be improved and standardised. Also a quality problem. But most problems users face originate not from the desktop environment but from the level below the desktop environment which cause trouble within the desktop environment. You cannot listen to sound because your sound driver is wrong configured or you need to install a special library.
  • That's not how you spell A-P-P-L-E!
  • Linux/Unix users are a self-selected bunch. You need to decide whether your user persona should be Joe Clueless (who is put in a room and has to perform basic functions in Linux) or the Power User.

    The Power User may turn out to be the more typical linux user (from the standpoint of HP/IBM), so the reactions of Joe Clueless may just not be useful. Good to have a specific goal in mind while running this study. Are you trying to help developers understand power users better?
    Or trying to help a company make ma

  • Don't assume. (Score:3, Interesting)

    by MikeFM ( 12491 ) on Thursday December 15, 2005 @08:34PM (#14268867) Homepage Journal
    Don't go into it with the expectation that more like Windows or Mac OS is better. Sometimes they may be and sometime they may not be. Study people ranging from no experience up through experts.
  • I wonder if she might better user her time doing something else. There is so much catcalling and pointless arguing in the nix community, and especially between the Gnome and KDE factions, or at least their fans. Nothing seems to be looked at objectively or is taken seriously except as grist for yet another propaganda offensive. Open source developers always have the ultimate get-out if something is subpar, namely that they aren't writing for a market or to a set of standards but for pleasure or their peers.
  • Well we all know Linus's opinion [slashdot.org]. It's rather insightful: GNOME is overly simple and for dumb users, KDE is for smart people, haha. But, seriously, it is all about configurability. One of the nice thigns about both KDE and GNOME is all of the configuration you can do to them. The question is, how "stripped down" of a configuration are you going to start a user out with? Are you going to set up some nice buttons or put some useful help-guides on the desktop? For instance, I'm starting a cute little Web sit
  • That must be some universtity, where a researcher asks slashdot users about usability. slashdot is where usability goes to die. Viva the CLI! Users are idiots! Consistency is for the simple minded! et cetera ad infinitum.

    For god's sake, woman. Read some Norman for the theoretical background (his older book, not his newer shite which pisses all over his previous work without any real reason to other than to shine his own "i'm a high priced consultant" knob now. Then, and this is serious though it's

  • Some suggestions: (Score:4, Interesting)

    by laughingcoyote ( 762272 ) <barghesthowl.excite@com> on Thursday December 15, 2005 @08:40PM (#14268908) Journal

    Get some novices (people who have never used the interface before) in, and a list of tasks to complete. Let some of them muddle through it on their own, and give others pointers on use of the help system, google, and man pages. (One of the tasks for the first group could be -finding- help on some of the things they won't be able to complete on their own.) This will help represent the range of people coming into it-some will have absolutely no idea what they're doing, others will have at least some support from other knowledgeable users who can at least point them in the right direction.

    You also might want a broad cross-section of users-some who rarely or never use a computer at all, some who use one relatively frequently, and some "power users" from other operating systems. This could lead to a very interesting picture-do those that already "know how" on a different interfacee have a harder time with something new, or are they able to translate most of their knowledge and pick up more quickly?

    As to a comparison between the two, you may wish not only to time how long it takes the users to complete their task lists, but also include feedback from them-were the help pages actually helpful, or did they just confuse the issue more? Was the experience relatively smooth and welcoming, or aggravating and frustrating? Was there anything the user expected to be/work a certain way that did not do as expected? Did the user find it necessary to work in CLI at any point, and if so, was this disorienting or frustrating, or relatively smooth? Did they ever think they had done something right when they really had screwed it up, and were any clues/warnings given them to this effect if so?

    All these are factors in usability, and I'm sure anyone can list plenty that I missed. In the end, usability is determined by-well, the user. Since it is somewhat subjective (I find working in a command line far easier and more convenient then use of a GUI most of the time, but there are many that would disagree!), focus on what the end-user, presented with the interface for the first time, thinks of it overall.

  • Make sure the study becomes large and well known enough so MS's payoff for you to "adjust" your findings is higher.
  • I would think she would want to decide what type of user you are testing for. The needs of a large or medium sized business user would deviate greatly from those of a home user.
  • Watch who you take funding from. It doesn't matter if you're as objective as you can possibly be, if you get funded by anything even remotely associated with one camp or the other, the hardcore geeks won't trust a word you say.
  • 1. Desktop Environments are today not the loopholes of Linux usability. Both are superiour to Windows and easier to use.

    2. Linux Desktop is always perfect when it works. The real problems are: It takes to much time to get it to work.

    * unsupported hardware and broken hardware detection
    * what happens if one component breaks, how are problems in hardware handled. (how to get rid off popup annnoyances), e.g. your cd drive is not detected, how does your music player handle the problem and help you, during instal
    • 1) no, they are not.

      2) Windows is always perfect when it works.

      * better integration of distribution functionality with the DE. DE or both DE are able to set standards and dictate them to all distributions which do not want to hack, they should better do that.
      * quality checks. E.g. manpages for all applications? Translations 100%? x-projects
      * system conformity checks.
      * unify DE registry and standardize setting file data formats.
      * buildserver
      * bridge some desktop functionality

      sooo make it more like windows?
      • I would rather say I am more productive on KDE than on Windows. All problems I face are not real problems of KDE despite some polishing. Most problems originate from unnecessary distribution differences. Consistency is a huge advantage despite the alleged inconsistency, - Windows is always perfect when it works? No, it is not. Because Windows is also messy and inconsistent, esp. when 3rd party software and hardware drivers get involved. Just install a drive for your digital camera. Integration of compo
  • We need more information.

    Is she testing against newbies or experts or in between? Choice of distro will make a HUGE difference here.

    An absolute newbie will have an easier time with Ubuntu even compared to windows (windows does not have a gui to download and install new apps without thought.)

    While ubuntu will cause fits to 15 year Linux vetrans that cut their teeth on making their own distro or slackware.

    Everything depends heavily on the perception of the users tested and the distro used. KDE and Gnome und
  • Your friend probably has more knowledge on HCI than all of the comments here combined. Trust her, don't trust /., because what is true for cryptography is also true for usability - it's easy to get it wrong and hard to get it right, and it takes an expert to spot the difference.

  • improve apon betterdesktop (Score:3, Informative)

    by goon ( 2774 ) <peterrenshaw@sel ... al.com minus bsd> on Thursday December 15, 2005 @08:47PM (#14268964) Homepage Journal
    `...What kind of advice -- both technical and theoretical -- would you give us on conducting a deep and objective study on the Unix desktop? ...`


    Improve apon betterdesktop [del.icio.us]. The site is a collection of usability data with a focus on Linux apps. The front page gives more detail ...

    `... is a project dedicated to sharing usability data with Linux developers. Over the past year, we have conducted many usability tests on different parts of the KDE and GNOME desktops. We created this site to serve as a place where developers can watch videos of these tests. Here you will find over 200 videos of people using Mozilla Firefox, Evolution, Open Office, Banshee, F-Spot and other applications. ...`


  • Mac OS X (Score:2, Insightful)

    by Slackdog ( 893543 )
    KDE vs Gnome? hey dude, you missed Mac OS X, the perhaps the best operating system based on UNIX http://www.apple.com/macosx/features/unix/ [apple.com]
  • Just have a set of tasks, like configure email client or download and install said software. Then, ask them how hard it was to do this. Give them no more instructions than the very simple sentances I gave earlier and make sure to have a group of people doing the testing that is pretty representative of the public (read not all Ph D. students in CS). You might want to come up with a list of questions that get you the comments you desire for each activity, but that should be a short list and let the testers k

  • OS X (Score:2, Insightful)

    by planetfinder ( 879742 )
    Use a commercially successful UNIX desktop as a reference point.
    While OS X doesn't occupy the majority of desktops it is
    a commercial success.
  • One thing I tend to find is that Windows users react differently to Linux desktops than do, say, Mac users. Windows has a rather peculier UI (from a strict usability theory standpoint), so not accounting for whether your test subjects are Windows or Mac users can make the results pointless.

    One thing might be to get longtime Mac users, longtime Windows users, and complete novices (though, this might be hard, because most people have used a computer at some point, and usually it was a Windows machine), and se

  • The Study should be Double Blinded (Score:4, Insightful)

    by sweetnjguy29 ( 880256 ) on Thursday December 15, 2005 @08:58PM (#14269028) Journal
    Neither the participants nor the study coordinator should know what operating system the test subjects are using! You might laugh, but all you need are people who have only used Windows or Mac!

    Also, make sure to use more than Gnome or KDE! Use XFCE, Fluxbox, and other XWindows managers.

    And don't forget to make sure that the study has the appropriate "power"!

    And make sure that everyone is using the same system configuration (motherboard, processor, underlying flavor of linux)

  • The most overlooked item is (Score:4, Interesting)

    by geekoid ( 135745 ) <dadinportlandNO@SPAMyahoo.com> on Thursday December 15, 2005 @09:00PM (#14269043) Homepage Journal
    a benchmark.

    Using the OOMA method, let's say it takes a user on one system 2 minutes to figure out how to send an email, and 1.95 min on anther system. What the hell does that mean?

    If you use other items as some sort of bench mark, people might begin to get a feel for the numbers.

    Lets say it take 45 seconds to figure out a new blender, 5 minutes to use a new remote, 20+years to set the clock on the VCR. Now people reading your study have a reference they can relate to.
    It would also help companies trying to make applience computer to know where they stand in relation to appliances.

  • A simple usability study consist of a series of steps:

    1. Make a list of tasks to be performed (this can be done with the subject if you would like an objective list).
    2. Think about the range of problems that can occur during the performing of the task. Make a subtask for every possible system failure.
    3. Confirm that your subject is in total control.
    4. Tell your subject that he or she is not to be judged according to his/her skills, but that the program (or in this case, distribution) is being judged. Faults

  • Don't ask us (Score:2, Insightful)

    by Milton Waddams ( 739213 )
    If this chick is doing a PhD on usability, then she probably knows tons about how to do an objective usability study. If she doesn't then people on Slashdot definitely won't :)
  • ... take her out, get her drunk and then bang her. Your deep and objective method is way too complicated.
  • 1) Coming here to ask your question is a bad idea. Not necessarily because of the quality of most answers, but rather because /. readers represent such a miniscule portion of the real population.

    2) Which brings me to my next point. Hire HCI experts, or take some classes on HCI. Testing OSS interfaces isn't any different than testing those of commercial software. You can do either user evaluations or predictive evaluations (w/o users). In fact, doing the latter first AND then the former is [usually] the b
  • What kind of advice -- both technical and theoretical -- would you give us on conducting a deep and objective study on the Unix desktop, that can be useful for the developers and the OSS community?"
    Well, for starters ... don't bother ask Slashdot. (Who wants to bet that Cliff made this submission up to get a flamewar started?)
  • one _very_ interesting test to do is to disconnect _all_ machines from the internet;
    windows, gnome, kde - and see whether people find it useable or even useful.

    another:

    http://kde-look.org/ [kde-look.org] and http://gnome-look.org/ [gnome-look.org] (when they're back up/online)

    set something up that is MAC-like. see how much it takes to set up a MAC look-alike
    (use kroller.sez - search for it on kde-look.org or even just kroller)

    use the MAC kde theme (baghira i think it is).

    try to do the same thing on gnome (which is near impossible).

    but mo

  • Don't get my hopes up. (Score:4, Interesting)

    by Hosiah ( 849792 ) on Thursday December 15, 2005 @09:46PM (#14269240)
    would you give us on conducting a deep and objective study on the Unix desktop

    Well, since Unix has *NEVER* had an objective study of it's desktop done, you will make history as a pioneer. Since it's survived so many smear campaigns, yours will, unfortunately, just add to the hot air. What, exactly, is the *point* of such a study, anyway? What does it change? I have yet to read a single such study that swayed my choices one iota.

    Sadly, you're off on the wrong foot already. KDE-vs-Gnome. Hey, Dr Kinsey, there's just a few other test subjects you're failing to interview: http://xwinman.org/ [xwinman.org]. So actually, you're flunking already. You are not doing a "Unix desktop study". You are doing a "KDE-vs-Gnome" study, and your results will no more be applicable to Unix in general than a study of Coke-vs-Pepsi would apply to all beverages.

    It does not go without saying: Don't be paid Microsoft shills. Don't be paid by *anybody* for that matter.

    Now, if I studied dogs, I wouldn't start with everything I know about cats and try to fit it all around that by comparing dogs with cats at every possible point. Similarly, Unix never gets taken as an operating system on it's own right. Everything is instead stated "It is not as good as or just like or better than Microsoft." How about judging something just once based on it's own merit, the way anybody studying anything else is expected to do in any other field? Consider your subject as if other operating systems did not exist. God knows, Microsoft is talked about in this manner.

    Unfortunately, the focus will of course be on KDE and Gnome, the Heckyl and Jeckyl whose sole point of contention is "I'M a perfect clone of the Windows environment!" "No, I am!" "No, me!" "NO, ME!" So in fact, you're not the least bit interested in considering even KDE or Gnome on it's own right - this will be a Windows-impersonator contest. Never mind that counting from the invention of computers: http://ei.cs.vt.edu/~history/Babbage.html [vt.edu], computers have been around for one hundred and eighty-two years, and only the last 20 years http://members.fortunecity.com/pcmuseum/windows.ht m [fortunecity.com] has seen the existence of a desktop system known as Windows. For a ratio of 0.10989011 of computer's history, you are going to compare the one system whose sole claim is that it made a lot of money in the United States to two other desktops expressly written to mimic it.

    I'm really sure the world will be enlightened.

  • Problem, Action and Result (Score:4, Funny)

    by Stan Vassilev ( 939229 ) on Thursday December 15, 2005 @10:50PM (#14269499)
    Problem: Unix GUI usability low for casual users.

    Action: Perform UNIX GUI usability studies every few days, post repeatedly on Slashdot.

    Result: UNIX GUI usability studies improve, UNIX GUI usability stays same.

  • KDE, Gnome and other (Score:3, Insightful)

    by Jon-o ( 17981 ) on Thursday December 15, 2005 @11:16PM (#14269627) Homepage
    My advice is to not just look at KDE and Gnome! Many people, myself included find both of them rather annoying and cluttered. There's a lot to be said for the customizability and simplicity of not using them. It's not perfect either, but I do wish people wouldn't assume that it's no longer an option.

  • Phrasing may be important too (Score:3, Interesting)

    by realnowhereman ( 263389 ) <andyparkins@nOsPam.gmail.com> on Friday December 16, 2005 @03:35AM (#14270422)
    In the back of the lastest Linux Journal is a report about a usability study at Novell. It was a description of how a user has trouble sending email because they where pressing the "Send" button instead of the "New" button. It made me wonder if the way the instructions are given are influencing the user. Would the results have been different in each of the following cases?
    • "Send an email"
    • "Create a new email then send it"
    • "Write an email"
    The answer from my point of view is "I don't know", but I'd be interested to know. Of course you need a load more people and a load more tests. But this is science, it's always tough :-)

  • Tell her to check back her notes. (Score:3, Insightful)

    by jotaeleemeese ( 303437 ) on Saturday December 17, 2005 @08:31AM (#14279115) Homepage Journal
    For goodness sake, usability is not checked in websites, mailing lists or blogs. The poster would be excused about following this path of action because he is not a human interface specialist, but I hope the PhD student is wiser about this.

    Human interface usability is properly tested one way and one way only. By watching real people interacting with them. You can use muckups (drawn or computerized), storyboards, etc, but nothing beats putting the fat asses of a few users in front of a computer and collecting their impressions.

    If you possibly can you tailor your investigation to a particular group of people, ideally one that would make the study useful to you (if you are testing usability for software in kindergartens you don't want to do your usability test in a bank's trading floor).

    The laughable suggestions to use children only, experienced users only or unexperienced users only as the correct or more accurate way of gauging usability is, as the British say, a load of pants. People suggesting this should jump of a clift like the gerbills they are.

    One would do such a think only if there is no choice or if one has particular reasons for doing so, but never as the preferred criteria for a generalized useful study, what may be good for children may be crap for old timers and viceversa, experienced users may find some things annoying that new users find useful and viceversa.

    The first thing that many people fail to understand is that usability is a wholy subjective thing. Linus loving KDE (yeah, that Linus) is only probe that for Linus (yeah, that Linus) KDE is more usable. All the KDE zealots implying that this is the God given truth regarding usability in Linux should be forced to use Gnome untile the know better.

    Usability should be studied only on groups with similar patterns of usage for it to be any useful. The wider you make your target group study, the more difficult it will be to find meaningful results.

    If you target all Linux users, then you are in for the most subjective, meaningless, most likely useless study.

    If you target Linux users with less than one year experience using Linux then you are into something. If you target Linux users with more than 10 years experience degrading penguins you would also find more useful results.

    Target your audience and you will find good results for that group of people.

    Make your sample too wide and be welcome to the scrapyard of useless studies.

Slashdot Top Deals

Beware of Programmers who carry screwdrivers. -- Leonard Brandwein

Close