Ask Slashdot: Developer Responsibility When Apps Might Risk Lives? 100
First time accepted submitter bashaw writes "What ethical responsibilities do software developers have in determining the role that mobile devices take in our lives? As performance increases, size decreases, and the only limitation is the software available, mobile devices have expanded into new areas of our lives for which they were not designed. This raises the ethical question of who decides what software is available, and therefore what role these devices should take. I am a software developer at the Canadian Avalanche Centre. We recently issued a warning about mobile avalanche search applications that are marketed as avalanche rescue systems. Three smartphone applications are presenting themselves as economical alternatives to avalanche transceivers, the electronic device used by backcountry users to find buried companions in case of an avalanche. The applications are not an adequate replacement for an avalanche transceiver for many reasons, and we are concerned about the use of this software in lieu of a specifically-designed avalanche transceiver. When it is a question of public safety, does the onus fall on the developers, a government agency or the users themselves?"
Developer or publisher? (Score:4, Interesting)
Re: (Score:1)
Using dynamically typed languages should be a capital crime.
Unfortunately the variable was set with a lower-case varaint and the program threw an exception.
Re: (Score:2, Insightful)
Unless someone shows otherwise, the apps mentioned seem to do what the software developers who created them made them do.
Does the app cause the phone to broadcast on the international avalanche transceiver standard 457kHz band? No? Then enjoy hearing the rescuers crunch by overhead while they look for you.
Re: (Score:1)
Does the app cause the phone to broadcast on the international avalanche transceiver standard 457kHz band? No? Then enjoy hearing the rescuers crunch by overhead while they look for you.
You have problems understanding what you read, right? Did the developers ever try to make the app broadcast? No. Did they ever claim it could do so? No. Did the publishers claim it could do so? Yes. So whose fault?
Re:Developer or publisher? (Score:5, Informative)
Well technically speaking, it would be likely the fault of the specific Government Consumer Protection Authority, whose jobs it is to monitor claims about products and if it finds them false, seek fiscal redress for the risk it puts the public too and ensure the public are warned. The end consumer should never really be put in this position because reality is the only find the failure when they try to apply the product and seeking legal redress can be all too late. I find it all too annoying to get product after product that fails to achieve the levels of performance claimed and really like the idea of an agency that puts the breaks on this, by bankrupting deceitful company after deceitful company as well as those companies executive teams.
Re: (Score:2)
I'm generally against the governments doing things like that but I agree this is one time some controlling authority steps in and initiates legal action to make sure the apps are not marketed as something they are incapable of. It would be like me marketing a bungy cord as a replacement passenger restraint system (seat belt) for cars that had them cut either by vandalism or by emergency personnel after a crash.
Given the information in the submission and one of the linked files, these apps should be required
Re: (Score:1)
Re: (Score:2)
Re: (Score:3)
If somebody brands it as 'Personal Safety Notifier Pro' and insinuates that EMS dispatch is somehow going to find you based on this behavior, I'd li
Re: (Score:2)
They would probably be sued out of existence by a few wrongful death lawsuits before anyone prosecuted them. The thing is, you can make claims, you can obfuscate claims that lead people to believe something not directly implied, but when it is a matter of life and death and the later p
Re: (Score:2)
That takes a lot of time and a lot of money from people that care about the dead. If the faulty product only kills people who are not extremely rich or the manufacturers have deep pockets (eg. Union Carbide) the threat of being sued is of little or no consequence.
Re: (Score:2)
In Ohio for instance, the law requires the executor of an estate to file any probable wrongful death lawsuits on the behalf of the estate. If they fail to do so, the legal heirs can go after the executor of the estate for whatever they think the wrongful death claim would have brought.
Most law firms will flip the bill for the lawsuit and take a percentage of the suit if won. But it might not be up to the people that care about the dead. It may be something the law requires even if it does cost a lot more of
Re: (Score:2)
I think you are confusing me with someone else. I'm not necessarily a libertarian and I have already stated in another thread you have replied to that I think government should regulate it.
However, the begger in Soviet Russia doesn't need to know anything about the US. Soviet Russia should have the laws in place too. This system is internationally recognized the world around and is in use in Russia too. But a lawsuit over international persons or shell companies only gets complicated, not impossible.
You are
Re: (Score:2)
Re: (Score:2)
Yes, I do know so little about Ayn Rand. What is your point? I already said I'm not the libertarian you think I am.
Re: (Score:3, Interesting)
Re: (Score:2)
You'd be lucky to have signal in avalanche country in the first place, much less through the snowpack.
Re: (Score:2)
I theorized about this exact type of app about five years ago. First off, there would be no requirement to have cell data accessible. You'd simply create an ad-hoc network with WiFi and/or Bluetooth and the devices will communicate over that, using signal strength and GPS coordinates as a locator.
Ultimately, though, the same drawbacks I considered years ago are the same ones mentioned in the review by the CAC, battery life being the most problematic.
After deciding that an avy beacon app would simply not w
Users (Score:2, Interesting)
App did not warn me about tornado. [xkcd.com]
Seriously, people have to take responsibility for their own choices.
We're too litigious nowadays; we ought to set the standard that grownups are required to think.
I agree (Score:1)
Re: (Score:2)
how would people know that it's not a proper safety device though?
I mean, you have to know something technically to know that he phone can't transmit on the required frequency. and all the consumer knows is that it has gps, glonass and lte and all kinds of fancy shit words on it.
(having 2100, 1800, 900mhz etc detectors on the rescuers would be a great idea though!)
Re: (Score:1)
Re: (Score:3)
You buy car. The car says it has brakes. The car only has brakes that work, though, when you are going less than 20mph.
Bummer for you, I guess. You should have known better. You should take responsibility for your own choice to buy that car! Why didn't you get under there and check the brakes thoroughly first? What, you want to sue? Everything they said was completely true, the car has brakes...
Admittedly, this is a silly and exaggerated example, but I personally have no doubt that a lot of advertis
All (Score:5, Informative)
does the onus fall on the developers, a government agency or the users themselves?
Yes.
Re: (Score:2)
If and only if all of those things fail, then maybe t
Re: (Score:3)
One of the problems is that how do you know those facts? Of course you are not a dummy and looked into it a bit and applied knowledge you probably already possessed, but have you done so before this story came about?
Well, suppose you have looked into it and when you did, you saw an app for your phone that said it could send help if ever in an avalanche. If you were an average consumer, would you know based on that exchange that the app couldn't be a substitute? Forget what you know about phones and think li
Sure (Score:2)
As soon as executives are financially responsible for the money they lose or swindle from customers.
Re: The Onus (Score:2)
Re: (Score:2)
So what happens when you write a marco polo game using GPS and then a month later you see that some jackass in marketing has claimed it is the perfect way to find someone buried in an avalanche?
Certainly an increasing danger. (Score:5, Insightful)
On the one hand, we can crack down hard on anyone who tries to even hint at some medical or safety purpose for a particular app. On the other we can be wild and free-booting and allow people into precisely the sort of trap that the poster outlines.
These apps may well be better than nothing (though they are not tested in any meaningful sense, nor are they compliant in any meaningful sense), but to the extent that they give a false sense of security, they are dangerous.
Personally, I lean towards crystal clear disclosure, and, in Canada, and restrictions on marketing. I do not favour an outright ban, since I could see that as having unpleasant consequences.
Look forward ten years. Suppose my smartphone has a ~90% reliable software and sensor package to tell me if I'm suffering from a heart attack. Suppose also that I'm part of a demographic group that by gender, age, fitness, weight, diet is highly unlikely to be suffering one. (There have been cases before where software has successfully diagnosed heart attacks in situations where physicians didn't believe it -- consider the case of psychologist Helen Smith a fit 37 year old woman who came close to dying since humans didn't believe she could be having a heart attack).
It would not make rational sense in that case for me to purchase a $1000 bespoke medical device to monitor me, but a $5 app might make sense even if it wasn't as reliable.
Similarly if I ski only occasionally and in areas highly unlikely to suffer an avalanche, it might make sense for me to not purchase a transceiver. (For those who say they'd spend anything to protect their lives, even on extraordinary low probability, I suspect you may have some irrational optimizations in your life.)
Offering consumers informed choice seems key; if they are marketing their apps as the equivalent of Avalanche transceivers, that clearly is not informed choice.
Similarly, I'd pressure Google and Apple and Blackberry to come up with a common standard for fine grid device location that these apps could use.
The OP raises some interesting points; I still come down somewhat on the libertarian side of things.
Re: (Score:2)
I do not favour an outright ban, since I could see that as having unpleasant consequences.
Such as? These apps literally enable the ignorant to get themselves killed, as you point out>
Did you actually read the rest of my post? Blanket regulations and bans tend to have unintended consequences and can be quite sweeping in effect. I gave a specific example of a situation (future cardiac monitor app) that might be quite beneficial for a certain segment of the population to have access to, even if it was less reliable than a dedicated device.
Such an item, if one were to blanket-ban apps based on medical and safety claims, would be unavailable in highly regulated countries, likely to the detri
Re: (Score:2)
There's a lot of regulatory framework in place already for medical devices, medical device software, etc. Apps that are trying to be medical devices should be following those rules (they're really more like laws than guidelines...) I'm sure lots of other life-critical areas have similar structures in-place.
Generally speaking, if you follow those best practices, you're doing what's expected and should be in the clear if you've actually done everything you are supposed to. Of course, it's all very fuzzy an
Re: (Score:2)
This is a very good point. In the past I have developed in the avionics and old-school telecoms area. (Half an hour unscheduled downtime permitted in 40 years, in the latter case). The former tends to be life-critical, the latter not far off.
I am very aware of the kind of requirements that medical software and devices require though have very deliberately steered well clear of that market.
It is my belief that developers should be educated, ethical, but that there is also a place for apps, even devices that
Re: (Score:2)
There does need to be a middle ground. Arguably, the current situation is already in ethical trouble because it prevents people who can't afford or otherwise access certified equipment from accessing good enough or better than nothing. Because of that, they get nothing.
Re: (Score:2)
consider the case of psychologist Helen Smith a fit 37 year old woman who came close to dying since humans didn't believe she could be having a heart attack
Or my case, where I actually did die (ventricular fibrillation) because humans didn't believe I could be having a heart attack. At 28. They sure changed their tune quickly though when I hit the floor without a pulse.
Re: (Score:3)
Even the insinuation that you are talking about a similar thing e
Re: (Score:2)
Except it seems these apps are rather battery-intensive in any automatic trigger mode and you don't want to have to manually activate them, meaning you'll probably only turn them on if you think you might possibly need them. Second since they're totally non-standard, they only do any good if you've agreed with your travel companions to use this as your avalanche search app, any other bystanders or rescue crews will look for a transponder if you have one or search based on visual observation if you don't, th
crack down on false claims (Score:2)
but let them go through otherwise.
As long as they're claiming "better than nothing" and not "as good as an avalanche beacon!" then I have no real problem with it.
Re: (Score:1)
Is a parachute with a giant hole in it better than no parachute?
Interestingly, for round parachutes a parachute with a hole in the apex is better than a parachute with no hole at all.
As for your question, yes a parachute with a giant hole is better than nothing. It still slows you down and depending upon what you land upon that slowing may make a difference. There are occasional incidents where parachutists survive parachute failures.
Re: (Score:2)
That depends.
How big of a hole in relation to the surface area of a parachute do you consider giant?
Better than nothing ? (Score:2)
We recently issued a warning about mobile avalanche search applications that are marketed as avalanche rescue systems. Three smartphone applications are presenting themselves as economical alternatives to avalanche transceivers, the electronic device used by backcountry users to find buried companions in case of an avalanche. The applications are not an adequate replacement for an avalanche transceiver for many reasons, and we are concerned about the use of this software in lieu of a specifically-designed avalanche transceiver.
Assuming they work to some degree its probably a matter of whether the buyer was accurately informed about their performance, not what the level of performance actually is. To be honest I expect that the users of such apps are those who would otherwise go into the backcountry with no device at all, are the apps better than nothing at all?
Re: (Score:2)
I had thought this too, thinking 'blah blah article presents false dichotomy... blah blah implying users would otherwise buy very expensive avalanche beacons etc'. Then I looked up the price of a real avalanche transceiver.
They are cheap. Much cheaper than most of the skiing gear (even most of the clothing) that back country skiers wear. In
Lacking Official Critiera (Score:2)
Re: (Score:2)
The Avalanche Rescue System actually does have a predicated standard that is recognized internationally. Supposedly, you purchase one anywhere, regardless of the manufacturer, and it is completely comparable anywhere in the world that uses the system.
That being said, I do think this is a government regulation type thing in at minimum, they should be required to clearly disclaim any possibility of being a substitute for the real thing.
Imagine you were at a pool or lake and there was a flotation device hangin
A short anecdote (Score:5, Interesting)
The headline reminded me of a story in a book of mine:
When Brunel's Ship the SS Great Britain was launched into the River Thames, it made such a splash that several spectators on the opposite bank were drowned. Nowadays, engineers reduce the force of entry into the water by rope tethers which are designed to break at carefully calculated intervals.
When the first computer came into operation in the Mathematisch Centrum in Amsterdam, one of the first tasks was to calculate the appropriate intervals and breaking strains of these tethers. In order to ensure the correctness of the program which did the calculations, the programmers were invited to watch the launching from the first row of the ceremonial viewing stand set up on the opposite bank. They accepted and they survived.
Re: (Score:2)
Re: (Score:2)
Yes, WIkipedia tells me that the Great Britain was floated out. It may have been the 1898 battleship HMS Albion, whose launching washed ~200 people standing on a rickety temporary "bridge" at a slipway on the opposite bank, of which 34 drowned.
Do you know anything about this "controlled launch by tether breaking"?
Re: (Score:1)
A Great Post in a Long While (Score:1)
That is what disclaimer are for... (Score:2)
"13. Note on Java support. The software may contain support for
programs written in Java. Java technology is not fault tolerant and is
not designed, manufactured, or intended for use or resale as online
control equipment in hazardous environments requiring fail-safe
performance, such as in the operation of nuclear facilities, aircraft
navigation or communication systems, air traffic control, direct life
support machines, or weapons systems, in which the failure of java
technology could lead directly to death, pers
Re: (Score:2)
Depends. (Score:1)
Die of stupidity... (Score:2)
Why do you hate the free market?
Everybody can make their own standard and just let the consumers sort it out (or not).
Re: (Score:2)
Because I really dislike exploding acetylene bottles. Given the option a lot of people would accept the risk of cheaper and more dangerous ones in the expectation that accidents happen to other people. However I was one of those free market hating pricks that blocked the import of some very dangerous ones from India in the 1990s.
Re: (Score:2)
Yes, I agree with you completely.
My comment was meant as sarcasm but that probably wasn't clear.
It depends on a lot of things. (Score:2)
Ok, I am not a lawyer, and I am not a Canadian, but as far as I can see there are a few different questions:
Where I am (the UK) criminal liability is likely to lie with the company that developed the product (or that sold it, if it was misrepresented). If they are selling safety-significant software, they should have appropriate processes in place to ensure the so
People who do stupid things often die... (Score:2)
There are always (and more so in modern times) people that cannot adequately estimate risk and will do stupid things without adequate protection. If some app-makers bank on that and claim properties which are not true or gross exaggerations, by all means charge them with involuntary manslaughter when their trash kills somebody. But the app-makers are only opportunistic parasites here, the real problem is people grossly overestimating their own skills. Just let them do it to themselves, its evolution at work
Re: (Score:2)
Avalanche beacons definitely help. And the technology has dramatically improved in the last twenty or thirty years. They are definitely easier to use and much more reliable nowadays.
Having said that, there are some fairly serious limitations on how well they can work. From what I remember of avalanche school, about fifty percent of avalanche victims die of traumatic injuries during the event, so obviously beacons can't help in those situations (my own personal experience in an avalanche included the tail
Honesty and Testing (Score:1)
I think the developers are ethically responsible for being crystal clear about the capabilities, reliability, and limitations of what is produced. They are responsible for reasonably thorough testing of any claimed capabilities on the hardware/OS combos it is stated to run on.
I think it is also a question of reasonable liability. In the case of liability, I would say it depends. As stated earlier consumers need to be aware of the reliability of the application, its capabilities, and limitations. Developers
But... but... radiation! (Score:2)
It's alarming how there's a certain group of people who are scared of the "danger" of radiation from mobile phones, when in fact mobile phones can save lives -- 911 calls, emergency alerts, and now apps that may help folks out of an avalanche.
For now the focus needn't be on which apps are best for various purposes, but on providing cellular service globally, anti-radiation crackpots be damned.
I write EMS/Police/Fire 911 Software (Score:3, Interesting)
I have my own company, with 1 business partner. We write software for 911 dispatch systems. Some of our clients require us to carry "Errors and Omissions" liability insurance, which costs us upwards of $15k a year. Along with with the insurance, we have a pretty detailed EULA agreement covering bugs, etc.
If you're writing any kind of software that could directly affect the safety of others, insurance is a must.
Frog
Re: (Score:3)
At least you're using insurance responsibly - does the company ever ask for any kind of audit?
I quit a job at the largest medical center in my area because the PHB's were ignoring the advice of the technical staff and insisting on buying inferior database software that was going to cause medication errors. We estimated a rate of seven mix-ups per year (due to single phase commits among disparate systems) and the bosses calculated that it would be cheaper to settle the lawsuits than to do it the right way.
avy rescue community already knows about this (Score:3)
This is from May of 2012...
http://avyupdates.blogspot.com/2012/05/iphone-avy-beacon-app-jokes-on-you.html [blogspot.com]
What about texting apps? (Score:2)
Facebook, foursquare, and just plain old texting, all impact people's safety because of their improper use, mainly by drivers. Are we going to go after texting app developers when people get hurt while using such software?
Re: (Score:1)
Facebook, foursquare, and just plain old texting, all impact people's safety because of their improper use, mainly by drivers. Are we going to go after texting app developers when people get hurt while using such software?
None of the apps and websites that you mentioned make claims that they can save your life via enabling others to locate you under snow based on wifi, gps, and/or bluetooth connections. Yes texting and browsing Facebook while driving impact people's safety but, as you said, doing such is improper use. The apps brought up in the documents created by the Canadian Avalanche Centre are, unsurprisingly, likely to not enable you to be found when using the device correctly while being stuck in an Avalanche.
Professional Liability Insurance (Score:2, Insightful)
The peddlers (Score:3)
...does the onus fall on the developers, a government agency or the users themselves...
If by "the developer" you mean the company that sells the product, then the answer is clear: it is the developer that is responsible. The government can at best issue guidelines for how safe and fit for purpose a product should be, and it is not realistic to expect all users to understand the full ramifications of their choices, when it comes to things that are likely to be well outside their general competencies.
On the other hand, if by "the developer" you mean the engineer who designed and coded the app, then I would say not so much, but it depends on the circumstances. In general, software is not sold directly by the developers; there is a business organisation between, that decides what to produce and how to sell it. They are far more likely to be the real culprits.
Whoever has the deepest pockets is "responsible" (Score:2)
Automotive judgement (Score:2)
Stalled in the Mojave desert 106F in the heat of the day, a 2011 VW decided all three keys were ' not in range'. Two hours later I reached water, tow truck and ride 67 miles to civilization. It remains to this day whether the memory was wiped clean from extra-terrestial sunspot activity, fault ECU or programmed to lock-out the car at specified date/mileage.
Lexus' judgement renders the ECU behavior to be the vehicle ' owner' responsibility in California. So there's definitely a need to get tools, connecti
Softwhere? (Score:1)
Snotty, over-priced stock software corporation + millions of product copies in the field + deaths = lawyers = guess what, your stock becomes like a car company's.
There are ways to deal with this, but it involves massive process and redundancy and code reviews and design reviews and detailed checks of Lint, QAC, Polyspace, a dozen other checkers, software watchdogs that, by the way, damned well never actually be needed, etc.
Ya better put down the keyboard and get some training for a few months.
National Society of Professional Engineers (NSPE) (Score:3)
Hold paramount the safety, health, and welfare of the public.
Perform services only in areas of their competence.
Issue public statements only in an objective and truthful manner.
Act for each employer or client as faithful agents or trustees.
Avoid deceptive acts.
Conduct themselves honorably, responsibly, ethically, and lawfully so as to enhance the honor, reputation, and usefulness of the profession. http://www.onlineethics.org/Resources/ethcodes/EnglishCodes/9972.aspx [onlineethics.org]
what about where the is a chain of Contracts / sub (Score:2)
what about where the is a chain of Contracts / subs where they all just pass the responsibility on.
In Canada Professional Engineers Must Do The Work (Score:2)
In Canada, under the various provincial acts (and a National act that keeps them largely consistent), professional engineers (note, the word "engineer" is legally protected in Canada, like Medical Doctor or Lawyer, unlike in the US.) must do any work that involves human safety. That INCLUDES computer/technical related work. The classic example is software for air traffic control systems or software on space shuttle modules.
One of the problems for the engineering regulatory bodies (Professional Engineers O
Re: (Score:1)
Bullshit.
I've done software development for such systems, and I am not a "professional engineer". I'm a professional computer programmer with a BSc in computer science.