Forgot your password?
typodupeerror
Cellphones Handhelds

Ask Slashdot: Developer Responsibility When Apps Might Risk Lives? 100

Posted by timothy
from the gps-makers-have-been-trying-to-kill-me-for-years dept.
First time accepted submitter bashaw writes "What ethical responsibilities do software developers have in determining the role that mobile devices take in our lives? As performance increases, size decreases, and the only limitation is the software available, mobile devices have expanded into new areas of our lives for which they were not designed. This raises the ethical question of who decides what software is available, and therefore what role these devices should take. I am a software developer at the Canadian Avalanche Centre. We recently issued a warning about mobile avalanche search applications that are marketed as avalanche rescue systems. Three smartphone applications are presenting themselves as economical alternatives to avalanche transceivers, the electronic device used by backcountry users to find buried companions in case of an avalanche. The applications are not an adequate replacement for an avalanche transceiver for many reasons, and we are concerned about the use of this software in lieu of a specifically-designed avalanche transceiver. When it is a question of public safety, does the onus fall on the developers, a government agency or the users themselves?"
This discussion has been archived. No new comments can be posted.

Ask Slashdot: Developer Responsibility When Apps Might Risk Lives?

Comments Filter:
  • by gnasher719 (869701) on Friday October 25, 2013 @07:24PM (#45241209)
    Unless someone shows otherwise, the apps mentioned seem to do what the software developers who created them made them do. But the publisher of these apps tries to sell them for uses that they are not fit for. That's the publisher's problem, not the developers'.
    • Re: (Score:2, Insightful)

      by Qzukk (229616)

      Unless someone shows otherwise, the apps mentioned seem to do what the software developers who created them made them do.

      Does the app cause the phone to broadcast on the international avalanche transceiver standard 457kHz band? No? Then enjoy hearing the rescuers crunch by overhead while they look for you.

      • Does the app cause the phone to broadcast on the international avalanche transceiver standard 457kHz band? No? Then enjoy hearing the rescuers crunch by overhead while they look for you.

        You have problems understanding what you read, right? Did the developers ever try to make the app broadcast? No. Did they ever claim it could do so? No. Did the publishers claim it could do so? Yes. So whose fault?

        • by rtb61 (674572) on Friday October 25, 2013 @08:12PM (#45241561) Homepage

          Well technically speaking, it would be likely the fault of the specific Government Consumer Protection Authority, whose jobs it is to monitor claims about products and if it finds them false, seek fiscal redress for the risk it puts the public too and ensure the public are warned. The end consumer should never really be put in this position because reality is the only find the failure when they try to apply the product and seeking legal redress can be all too late. I find it all too annoying to get product after product that fails to achieve the levels of performance claimed and really like the idea of an agency that puts the breaks on this, by bankrupting deceitful company after deceitful company as well as those companies executive teams.

          • by sumdumass (711423)

            I'm generally against the governments doing things like that but I agree this is one time some controlling authority steps in and initiates legal action to make sure the apps are not marketed as something they are incapable of. It would be like me marketing a bungy cord as a replacement passenger restraint system (seat belt) for cars that had them cut either by vandalism or by emergency personnel after a crash.

            Given the information in the submission and one of the linked files, these apps should be required

          • by slick7 (1703596)
            Unless, of course, you are Monsanto and you are not responsible for your products.
        • The ones who lied? Honesty isn't a 100% ironclad defense, there are things you can do with full disclosure that are still unethical; but if I program 'tweet_while_U_die' a program that uses your accelerometer to detect a probably-fatal vehicular collision and then tweets about it for you while you bleed out, I'm in poor taste; but hey, it is what I say it is.

          If somebody brands it as 'Personal Safety Notifier Pro' and insinuates that EMS dispatch is somehow going to find you based on this behavior, I'd li
          • by sumdumass (711423)

            If somebody brands it as 'Personal Safety Notifier Pro' and insinuates that EMS dispatch is somehow going to find you based on this behavior, I'd like to see them charged with negligent homicide each time that fails to happen.

            They would probably be sued out of existence by a few wrongful death lawsuits before anyone prosecuted them. The thing is, you can make claims, you can obfuscate claims that lead people to believe something not directly implied, but when it is a matter of life and death and the later p

            • by dbIII (701233)

              They would probably be sued out of existence by a few wrongful death lawsuits before anyone prosecuted them

              That takes a lot of time and a lot of money from people that care about the dead. If the faulty product only kills people who are not extremely rich or the manufacturers have deep pockets (eg. Union Carbide) the threat of being sued is of little or no consequence.

              • by sumdumass (711423)

                In Ohio for instance, the law requires the executor of an estate to file any probable wrongful death lawsuits on the behalf of the estate. If they fail to do so, the legal heirs can go after the executor of the estate for whatever they think the wrongful death claim would have brought.

                Most law firms will flip the bill for the lawsuit and take a percentage of the suit if won. But it might not be up to the people that care about the dead. It may be something the law requires even if it does cost a lot more of

        • Re: (Score:3, Interesting)

          While I generally agree with you, I think it's worth looking at the example given by electronic chart programs. They all make you click on a "The prudent mariner will have properly updated paper charts" notice on startup. Once they learned about the publisher's advertising, the developers could make a "notice to idiots" one has to acknowledge on startup saying something like "99.9% of phones made in 2013 don't have the hardware to broadcast on the 457kHz avalanche transceiver band, and this app doesn't tr
      • You'd be lucky to have signal in avalanche country in the first place, much less through the snowpack.

        • by pspahn (1175617)

          I theorized about this exact type of app about five years ago. First off, there would be no requirement to have cell data accessible. You'd simply create an ad-hoc network with WiFi and/or Bluetooth and the devices will communicate over that, using signal strength and GPS coordinates as a locator.

          Ultimately, though, the same drawbacks I considered years ago are the same ones mentioned in the review by the CAC, battery life being the most problematic.

          After deciding that an avy beacon app would simply not w

  • Users (Score:2, Interesting)

    by Anonymous Coward

    App did not warn me about tornado. [xkcd.com]
    Seriously, people have to take responsibility for their own choices.
    We're too litigious nowadays; we ought to set the standard that grownups are required to think.

    • People themselves should be responsible for choices they make. In this scenario a smartphone app could act as a supplement but not a replacement for an avalanche tracker.
      • by gl4ss (559668)

        how would people know that it's not a proper safety device though?

        I mean, you have to know something technically to know that he phone can't transmit on the required frequency. and all the consumer knows is that it has gps, glonass and lte and all kinds of fancy shit words on it.

        (having 2100, 1800, 900mhz etc detectors on the rescuers would be a great idea though!)

    • by Jmc23 (2353706)
      Thinking for yourself would interfere with your police state. Move to another country if you don't want to be surrounded by dumbasses.
    • You buy car. The car says it has brakes. The car only has brakes that work, though, when you are going less than 20mph.

      Bummer for you, I guess. You should have known better. You should take responsibility for your own choice to buy that car! Why didn't you get under there and check the brakes thoroughly first? What, you want to sue? Everything they said was completely true, the car has brakes...

      Admittedly, this is a silly and exaggerated example, but I personally have no doubt that a lot of advertis

  • All (Score:5, Informative)

    by ebno-10db (1459097) on Friday October 25, 2013 @07:25PM (#45241221)

    does the onus fall on the developers, a government agency or the users themselves?

    Yes.

    • by tompaulco (629533)
      First, a developer with any morals would not develop such an app (and it sounds like they didn't). Second, the discerning public should know that their phone is not capable of broadcasting on the 457 kHz band, by government regulation, that their phone is not even warranted for use in temperatures where an avalanche might be a possibility, and that if they were in an avalanche, the phones stray droplet of water detector would instantly render it useless.
      If and only if all of those things fail, then maybe t
      • by sumdumass (711423)

        One of the problems is that how do you know those facts? Of course you are not a dummy and looked into it a bit and applied knowledge you probably already possessed, but have you done so before this story came about?

        Well, suppose you have looked into it and when you did, you saw an app for your phone that said it could send help if ever in an avalanche. If you were an average consumer, would you know based on that exchange that the app couldn't be a substitute? Forget what you know about phones and think li

  • by Herkum01 (592704)

    As soon as executives are financially responsible for the money they lose or swindle from customers.

  • by Holmwood (899130) on Friday October 25, 2013 @07:38PM (#45241307)

    On the one hand, we can crack down hard on anyone who tries to even hint at some medical or safety purpose for a particular app. On the other we can be wild and free-booting and allow people into precisely the sort of trap that the poster outlines.

    These apps may well be better than nothing (though they are not tested in any meaningful sense, nor are they compliant in any meaningful sense), but to the extent that they give a false sense of security, they are dangerous.

    Personally, I lean towards crystal clear disclosure, and, in Canada, and restrictions on marketing. I do not favour an outright ban, since I could see that as having unpleasant consequences.

    Look forward ten years. Suppose my smartphone has a ~90% reliable software and sensor package to tell me if I'm suffering from a heart attack. Suppose also that I'm part of a demographic group that by gender, age, fitness, weight, diet is highly unlikely to be suffering one. (There have been cases before where software has successfully diagnosed heart attacks in situations where physicians didn't believe it -- consider the case of psychologist Helen Smith a fit 37 year old woman who came close to dying since humans didn't believe she could be having a heart attack).

    It would not make rational sense in that case for me to purchase a $1000 bespoke medical device to monitor me, but a $5 app might make sense even if it wasn't as reliable.

    Similarly if I ski only occasionally and in areas highly unlikely to suffer an avalanche, it might make sense for me to not purchase a transceiver. (For those who say they'd spend anything to protect their lives, even on extraordinary low probability, I suspect you may have some irrational optimizations in your life.)

    Offering consumers informed choice seems key; if they are marketing their apps as the equivalent of Avalanche transceivers, that clearly is not informed choice.

    Similarly, I'd pressure Google and Apple and Blackberry to come up with a common standard for fine grid device location that these apps could use.

    The OP raises some interesting points; I still come down somewhat on the libertarian side of things.

    • There's a lot of regulatory framework in place already for medical devices, medical device software, etc. Apps that are trying to be medical devices should be following those rules (they're really more like laws than guidelines...) I'm sure lots of other life-critical areas have similar structures in-place.

      Generally speaking, if you follow those best practices, you're doing what's expected and should be in the clear if you've actually done everything you are supposed to. Of course, it's all very fuzzy an

      • by Holmwood (899130)

        This is a very good point. In the past I have developed in the avionics and old-school telecoms area. (Half an hour unscheduled downtime permitted in 40 years, in the latter case). The former tends to be life-critical, the latter not far off.

        I am very aware of the kind of requirements that medical software and devices require though have very deliberately steered well clear of that market.

        It is my belief that developers should be educated, ethical, but that there is also a place for apps, even devices that

        • by sjames (1099)

          There does need to be a middle ground. Arguably, the current situation is already in ethical trouble because it prevents people who can't afford or otherwise access certified equipment from accessing good enough or better than nothing. Because of that, they get nothing.

    • by Dunbal (464142) *

      consider the case of psychologist Helen Smith a fit 37 year old woman who came close to dying since humans didn't believe she could be having a heart attack

      Or my case, where I actually did die (ventricular fibrillation) because humans didn't believe I could be having a heart attack. At 28. They sure changed their tune quickly though when I hit the floor without a pulse.

    • I think that there is one additional factor to consider (that does strongly affect this case; but might be weak or nonexistent in others): In the case of an "Avalanche Transceiver", that's not a generic description of any radio beacon designed for avalanches, it's a fairly specific set of standardized minimum capabilities and interoperability characteristics, recognized by both hardware vendors and significant mountaineering organizations.

      Even the insinuation that you are talking about a similar thing e
    • by Kjella (173770)

      Except it seems these apps are rather battery-intensive in any automatic trigger mode and you don't want to have to manually activate them, meaning you'll probably only turn them on if you think you might possibly need them. Second since they're totally non-standard, they only do any good if you've agreed with your travel companions to use this as your avalanche search app, any other bystanders or rescue crews will look for a transponder if you have one or search based on visual observation if you don't, th

  • but let them go through otherwise.

    As long as they're claiming "better than nothing" and not "as good as an avalanche beacon!" then I have no real problem with it.

  • We recently issued a warning about mobile avalanche search applications that are marketed as avalanche rescue systems. Three smartphone applications are presenting themselves as economical alternatives to avalanche transceivers, the electronic device used by backcountry users to find buried companions in case of an avalanche. The applications are not an adequate replacement for an avalanche transceiver for many reasons, and we are concerned about the use of this software in lieu of a specifically-designed avalanche transceiver.

    Assuming they work to some degree its probably a matter of whether the buyer was accurately informed about their performance, not what the level of performance actually is. To be honest I expect that the users of such apps are those who would otherwise go into the backcountry with no device at all, are the apps better than nothing at all?

    • "To be honest I expect that the users of such apps are those who would otherwise go into the backcountry with no device at all, are the apps better than nothing at all?"

      I had thought this too, thinking 'blah blah article presents false dichotomy... blah blah implying users would otherwise buy very expensive avalanche beacons etc'. Then I looked up the price of a real avalanche transceiver.

      They are cheap. Much cheaper than most of the skiing gear (even most of the clothing) that back country skiers wear. In
  • I would say the onus falls upon the user. In the absence of accepted standards or regulating entities I think it's buyer beware man. On the other hand, If you have standards dictating required operational parameters, then it's quite obviously the developer or designer I think. My post is over simplified and ignores the hardware/software integration question, but I think I'm on the right track. At least by slashdot standards. I welcome argument and rectification.
    • by sumdumass (711423)

      The Avalanche Rescue System actually does have a predicated standard that is recognized internationally. Supposedly, you purchase one anywhere, regardless of the manufacturer, and it is completely comparable anywhere in the world that uses the system.

      That being said, I do think this is a government regulation type thing in at minimum, they should be required to clearly disclaim any possibility of being a substitute for the real thing.

      Imagine you were at a pool or lake and there was a flotation device hangin

  • A short anecdote (Score:5, Interesting)

    by arielCo (995647) on Friday October 25, 2013 @07:53PM (#45241431)

    The headline reminded me of a story in a book of mine:

    When Brunel's Ship the SS Great Britain was launched into the River Thames, it made such a splash that several spectators on the opposite bank were drowned. Nowadays, engineers reduce the force of entry into the water by rope tethers which are designed to break at carefully calculated intervals.

    When the first computer came into operation in the Mathematisch Centrum in Amsterdam, one of the first tasks was to calculate the appropriate intervals and breaking strains of these tethers. In order to ensure the correctness of the program which did the calculations, the programmers were invited to watch the launching from the first row of the ceremonial viewing stand set up on the opposite bank. They accepted and they survived.

    • by tapi0 (2805569)
      Not to take anything away from the principle underlying the anecdote, but SS Great Britain was launched at the Bristol Docks with nothing of great note occurring (apart from getting wedged in the lock leading to the Avon)
      • by arielCo (995647)

        Yes, WIkipedia tells me that the Great Britain was floated out. It may have been the 1898 battleship HMS Albion, whose launching washed ~200 people standing on a rickety temporary "bridge" at a slipway on the opposite bank, of which 34 drowned.

        Do you know anything about this "controlled launch by tether breaking"?

  • This is the best story/discussion topic in a long while here on slashdot. I hope many chime in with their serious thoughts.
  • "13. Note on Java support. The software may contain support for
    programs written in Java. Java technology is not fault tolerant and is
    not designed, manufactured, or intended for use or resale as online
    control equipment in hazardous environments requiring fail-safe
    performance, such as in the operation of nuclear facilities, aircraft
    navigation or communication systems, air traffic control, direct life
    support machines, or weapons systems, in which the failure of java
    technology could lead directly to death, pers

    • yup, company I work for publishes medical books for veterinarians and we have a disclaimer saying we are not responsible. We put it in our mobile apps as well.
  • If the developer is the owner of the software and has provided warranty for the software and/or as a selling points, uses the accuracy and reliability of the software and, assuming this, that the users of the product are following all the rules and guidelines and the software fails.. then, there could be a point towards this. So, this becomes a question of what are you selling, how much are you willing to back it up and finally, if there are any disclaimers, are they easily understood? Nothing is infalli
  • Why do you hate the free market?
    Everybody can make their own standard and just let the consumers sort it out (or not).

    • by dbIII (701233)

      Why do you hate the free market?

      Because I really dislike exploding acetylene bottles. Given the option a lot of people would accept the risk of cheaper and more dangerous ones in the expectation that accidents happen to other people. However I was one of those free market hating pricks that blocked the import of some very dangerous ones from India in the 1990s.

      • by mspohr (589790)

        Yes, I agree with you completely.
        My comment was meant as sarcasm but that probably wasn't clear.

  • Ok, I am not a lawyer, and I am not a Canadian, but as far as I can see there are a few different questions:

    • Who (if anyone) has criminal liability
    • Who can get sued?
    • Who can get successfully sued? And
    • Do you have a moral obligation independent of the law?

    Where I am (the UK) criminal liability is likely to lie with the company that developed the product (or that sold it, if it was misrepresented). If they are selling safety-significant software, they should have appropriate processes in place to ensure the so

  • There are always (and more so in modern times) people that cannot adequately estimate risk and will do stupid things without adequate protection. If some app-makers bank on that and claim properties which are not true or gross exaggerations, by all means charge them with involuntary manslaughter when their trash kills somebody. But the app-makers are only opportunistic parasites here, the real problem is people grossly overestimating their own skills. Just let them do it to themselves, its evolution at work

  • by Anonymous Coward

    I think the developers are ethically responsible for being crystal clear about the capabilities, reliability, and limitations of what is produced. They are responsible for reasonably thorough testing of any claimed capabilities on the hardware/OS combos it is stated to run on.

    I think it is also a question of reasonable liability. In the case of liability, I would say it depends. As stated earlier consumers need to be aware of the reliability of the application, its capabilities, and limitations. Developers

  • It's alarming how there's a certain group of people who are scared of the "danger" of radiation from mobile phones, when in fact mobile phones can save lives -- 911 calls, emergency alerts, and now apps that may help folks out of an avalanche.

    For now the focus needn't be on which apps are best for various purposes, but on providing cellular service globally, anti-radiation crackpots be damned.

  • by FrozenFrog (539212) on Friday October 25, 2013 @10:41PM (#45242353)

    I have my own company, with 1 business partner. We write software for 911 dispatch systems. Some of our clients require us to carry "Errors and Omissions" liability insurance, which costs us upwards of $15k a year. Along with with the insurance, we have a pretty detailed EULA agreement covering bugs, etc.

    If you're writing any kind of software that could directly affect the safety of others, insurance is a must.

    Frog

    • At least you're using insurance responsibly - does the company ever ask for any kind of audit?

      I quit a job at the largest medical center in my area because the PHB's were ignoring the advice of the technical staff and insisting on buying inferior database software that was going to cause medication errors. We estimated a rate of seven mix-ups per year (due to single phase commits among disparate systems) and the bosses calculated that it would be cheaper to settle the lawsuits than to do it the right way.

  • Facebook, foursquare, and just plain old texting, all impact people's safety because of their improper use, mainly by drivers. Are we going to go after texting app developers when people get hurt while using such software?

    • by aokoye (1628741)

      Facebook, foursquare, and just plain old texting, all impact people's safety because of their improper use, mainly by drivers. Are we going to go after texting app developers when people get hurt while using such software?

      None of the apps and websites that you mentioned make claims that they can save your life via enabling others to locate you under snow based on wifi, gps, and/or bluetooth connections. Yes texting and browsing Facebook while driving impact people's safety but, as you said, doing such is improper use. The apps brought up in the documents created by the Canadian Avalanche Centre are, unsurprisingly, likely to not enable you to be found when using the device correctly while being stuck in an Avalanche.

  • by Anonymous Coward
    It falls on you, assuming your license sucks, but just make sure your Professional Liability coverage handles that sort of thing. I make search-and-rescue alerting software for the aerospace industry, my premiums are about 33% more.
  • by jandersen (462034) on Saturday October 26, 2013 @03:33AM (#45243311)

    ...does the onus fall on the developers, a government agency or the users themselves...

    If by "the developer" you mean the company that sells the product, then the answer is clear: it is the developer that is responsible. The government can at best issue guidelines for how safe and fit for purpose a product should be, and it is not realistic to expect all users to understand the full ramifications of their choices, when it comes to things that are likely to be well outside their general competencies.

    On the other hand, if by "the developer" you mean the engineer who designed and coded the app, then I would say not so much, but it depends on the circumstances. In general, software is not sold directly by the developers; there is a business organisation between, that decides what to produce and how to sell it. They are far more likely to be the real culprits.

  • That's the way the legal system works. There is liability in EVERYTHING. You carry E&O insurance so you don't lose your house.
  • Stalled in the Mojave desert 106F in the heat of the day, a 2011 VW decided all three keys were ' not in range'. Two hours later I reached water, tow truck and ride 67 miles to civilization. It remains to this day whether the memory was wiped clean from extra-terrestial sunspot activity, fault ECU or programmed to lock-out the car at specified date/mileage.

    Lexus' judgement renders the ECU behavior to be the vehicle ' owner' responsibility in California. So there's definitely a need to get tools, connecti

  • Snotty, over-priced stock software corporation + millions of product copies in the field + deaths = lawyers = guess what, your stock becomes like a car company's.

    There are ways to deal with this, but it involves massive process and redundancy and code reviews and design reviews and detailed checks of Lint, QAC, Polyspace, a dozen other checkers, software watchdogs that, by the way, damned well never actually be needed, etc.

    Ya better put down the keyboard and get some training for a few months.

  • by mjwalshe (1680392) on Saturday October 26, 2013 @01:20PM (#45245695)
    I. Fundamental Canons Engineers, in the fulfillment of their professional duties, shall:

    Hold paramount the safety, health, and welfare of the public.
    Perform services only in areas of their competence.
    Issue public statements only in an objective and truthful manner.
    Act for each employer or client as faithful agents or trustees.
    Avoid deceptive acts.
    Conduct themselves honorably, responsibly, ethically, and lawfully so as to enhance the honor, reputation, and usefulness of the profession. http://www.onlineethics.org/Resources/ethcodes/EnglishCodes/9972.aspx [onlineethics.org]
  • what about where the is a chain of Contracts / subs where they all just pass the responsibility on.

  • In Canada, under the various provincial acts (and a National act that keeps them largely consistent), professional engineers (note, the word "engineer" is legally protected in Canada, like Medical Doctor or Lawyer, unlike in the US.) must do any work that involves human safety. That INCLUDES computer/technical related work. The classic example is software for air traffic control systems or software on space shuttle modules.

    One of the problems for the engineering regulatory bodies (Professional Engineers O

    • by msobkow (48369)

      Bullshit.

      I've done software development for such systems, and I am not a "professional engineer". I'm a professional computer programmer with a BSc in computer science.

The one day you'd sell your soul for something, souls are a glut.

Working...