Should I Take Toyota's Software Update? 750
kiehlster writes "I'm a software developer, and I know that most software has bugs, but how much trust can we put in the many lines of code found in our automobiles? I have a 2009 Camry that is involved in both of the recent Toyota recalls. As part of the floor-mat issue, they're offering to install a software update that would cause 'the brake pedal to take precedence over the gas pedal if both were pressed,' or, as their latest notice states, 'would cut power to the engine if both pedals were pressed.' In the computer world, we're all taught to install firmware updates only if there is a real problem because a large percentage of firmware updates actually brick the hardware or cause other unforeseen consequences. On a base of 100 million lines of code, can I really trust a software update to work safely when it is delivered in a three-month development cycle? My driving habits don't cause the floor mat to slide much, so I see the update as overkill. What do you think? If it doesn't void the warranty, should I tell them to skip the update?"
If it bricks, it's their fault. (Score:5, Informative)
First, this is about your safety.
Second, if the update bricks your car, that would be Toyota's fault, not yours and I'm pretty sure they would resolve the issue for you free of charge.
Or, you can keep driving a potentially unsafe vehicle on "firmware update" principles.
Get the Flash (Score:5, Informative)
There's a lot of cars that have the 'brake takes precedence' feature. The only real reason to not have such a feature is because of trail-braking or hell-toe shifting. Both are racing/performance driving techniques you won't be doing in your Camry. Plus, it is a pure software feature in that if it detects you braking, it will cut throttle. So there's no big issue there.
Also, cars have their computers updated all the time, and it has never been a big deal in the past. The Nissan GTR was the last example that made the news (to cut down on the RPM the launch control used). But really, cars are reflashed all the time. Its not a big deal.
Apply the update (Score:5, Informative)
Many other manufacturers have already added a similar piece of code. It really doesn't take to long to debug an interlock. Your primary failure mode will be: if the brake pressed switch fails (ie: the tail lights are stuck on), then the car won't run.
Every interlock has a strong tendency to fail into the safe state. Conversely, omitting interlocks tends to result in fail-dangerous failures, which is what Toyota is experiencing.
Seriously? (Score:5, Informative)
Take the update.
My driving habits don't cause the floor mat to slide much, so I see the update as overkill.
Perhaps, but didn’t I read about some people who died in a Toyota, presumably from this exact bug, whose floor mat was found secure in their trunk, exactly where Toyota recommended them to put it when they thought the floor mats were causing the accelerator bug?
Re:You're looking at it wrong. (Score:5, Informative)
It's not 100M lines of handwritten code! Every time this comes up everyone (especially those that work with embedded systems) seem to think that there are a ton of code monkeys locked away coding in C or assembly.
I'd be willing to bet that almost all of it is auto generated. Toyota (and nearly everyone else) uses Matlab & Simulink extensively.
The MathWorks tools help Toyota design for the future [mathworks.com] (PDF)
Toyota Racing Development Makes Faster and More Efficient Engineering Decisions with MATLAB [mathworks.com]
A simple PID controler with saturation and limits could easily take up 50 "lines of code".
And it's not like Toyota is Mathworks' sole customer. Boeing, GM, Chrysler, Ford, etc ALL use Mathworks.
Just like nearly everyone that works with CAN uses Vector CANape [vector.com]. Everyone that develops ICE powertrains uses AVL [avl.com]
When you start to get to specialized software like what Matlab, CANape, AVL, etc all do, there aren't a ton of options (and no open source solutions). It's cheaper for all of these companies to buy X product and use it than try to write their own.
Re:huh? (Score:5, Informative)
Bill
Re:Their new slogan (Score:4, Informative)
Where was the Spanish Inquisition errr... Congress when Ford had to recall 4.5 million cars [cnn.com] a few months ago due to their cruise control causing fires?
Re:He is looking at it wrong... (Score:3, Informative)
You are currently modded funny, but I would prefer not to purchase a car that prohibited me from pressing the brake and throttle at the same time and expecting power and braking. You don't need to be James Bond to do left-foot braking, you just need to understand when it is to be used (on the racetrack only). Obviously this situation doesn't apply to a Camry, and I don't know if any of their high performance cars have this same issue. If purchasing a high performance car I would expect the brake and throttle to work independently.
Heck, I even set up my racing pedals on my computer at home to be independent to allow for LFB.
Re:You're looking at it wrong. (Score:5, Informative)
As a user of these software programs, I can tell you how they are Really used:
PHD Uses matlab and simulink to create their motor control algorithms. They port program to the processor of choice and test their algorithm.
Once their algorithm is proved, the firmware engineer uses that code as a template. They re-write all the code to play nicely with the other required code and to improve efficiency. (WTF? Another Memcopy? GARGH! Stop hogging all of my cycles!)
It is a great program for a rapid prototype and proof-of-concept, but it totally fails on actual implementation. I have been to a few microcontroller workshops where people have told the horror stories about the atrocious code created by these programs. In the end, it is just not production quality code.
Re:Jane, you ignorant slut... (Score:3, Informative)
While I disagree with the 'large percentage of firmware updates actually brick' bit, he's correct that it's pretty common practice not to update firmware unless there's a known bug that -is- affecting you.
However, that applies to non-mission-critical appliances like home routers and not to death machines like cars or any device that could cost someone a -lot- of money if it goes down.
And you should never do the firmware update on a 'live' system for the same reason. So if he's actually driving the car while he updates the firmware, that would be bad. Otherwise, it should be done.
Re:You're looking at it wrong. (Score:5, Informative)
Which articles were that?
The one I saw was this:
http://www.caranddriver.com/features/09q4/how_to_deal_with_unintended_acceleration-tech_dept [caranddriver.com]
The speed where brakes+full throttle didn't eventually stop the car was 120mph.
And their conclusion:
http://www.caranddriver.com/news/car/10q1/toyota_recall_scandal_media_circus_and_stupid_drivers-editorial [caranddriver.com]
Re:You're looking at it wrong. (Score:2, Informative)
Re:He is looking at it wrong... (Score:3, Informative)
>Manual transmission drivers don't have three feet, they can't hold the break, clutch and gas at the same time.
No, but they can use the handbrake, which is what I do for hill starts on steep hills.
Re:He is looking at it wrong... (Score:2, Informative)
Re:You're looking at it wrong. (Score:4, Informative)
Re:You're looking at it wrong. (Score:5, Informative)
My background is as an RF engineer, and I have a reasonable familiarity with EMI engineering.
The utter fucking cluelessness of that article scares me.
"Professor Liu, the story says, compares it to the problem with the jamming of signals on military aircraft.
"The problem is, the expertise for preventing signal jamming rests in the Department of Defense, not the automakers or their suppliers,' Professor Liu says. "
There's a MASSIVE difference between trying to prevent jamming of communications/radar signals, and basic EMI protection engineering of wired electronic circuits. There is PLENTY of experience with the latter in the civilian world, especially within the automotive industry.
Yes, cell phones can cause EMI problems with unshielded equipment, especially GSM phones. The critical systems in a vehicle are without any doubt *shielded*. More details on that later...
Satellite radios are RECEIVERS. (With the exception of satphones - these are incredibly rare.) They can be jammed, but you have to SERIOUSLY fuck up for one of them to interfere with something else. Same for GPS receivers. The most likely way for either of these systems to affect a car negatively is for them to short out and pull excessive current from their power supply. That's what fuses are for.
Large restaurant microwaves are subject to the same restrictions from the FCC as home microwaves. Yeah they can leak a little and they'll jam 2.4 GHz communications, but you could most likely take the magnetron from a microwave oven, point it at a car, and no adverse effects to critical systems would happen.
Why? Because the ignition system within a car is typically the #1 source of interference to anything in or near a car. A malfunctioning ignition system (old spark plug wires, loose spark plug wire connections) is tantamount to a high power spark gap transmitter. Automotive engineers have been dealing with internally generated EMI since the beginning of their industry.
Nice fact free sound bite (Score:3, Informative)
Take a look at the statistics for death causes for people under 60, and you will find almost everyone who doesn't die old dies in a car.
Nonsense. [wikipedia.org] Yes, motor vehicle accidents are the leading cause of death in the US for those between the ages of 15 and 34 (peaking at around 1 out of 3 deaths for the 15-24 age group) but it is nowhere close to "almost everyone" no matter what age group you choose. But don't let actual data get in the way of a good sound bite.
Look at what wasted labor there is in society, and you will find that producing and maintaining one high-price high-waste transportation system per citizen is quite a bit of work when horses managed do to better than that quite some time ago...
If horses were actually more efficient economically, we would still be using horses. If you think horses are cheap as a means of transportation, you clearly have never tried to use them. Yes there is a cost to modern infrastructure but there is a bigger (economic) cost to lacking it. The biggest obstacle to the growth of many nations (India is a good example) is a poor quality road infrastructure.
not to mention electricity and electric computer system transport. And PRT more recently.
You think a PRT is seriously a solution which makes sense for more than a few high density urban areas? Nice for airports but it isn't going to be much use on a farm.
Then read about pollution, and oil wars.
Yep, there is a downside to fossil fuels. Fossil fuels have serious problems in need of serious solutions. However there is a huge upside too which I note you are conveniently forgetting. I'd also like you to point out the magical technology you think will eliminate pollution. Solar and wind come closest but even they pollute. (you didn't think the steel in that turbine came without an environmental cost did you?)
Re:He is looking at it wrong... (Score:5, Informative)
Re:You're looking at it wrong. (Score:5, Informative)
Ok. Case in point, here is a VERY simple switch block. (And this could really be all that they did)
Brake_Override.jpg [exstatic.org]
If brake is 1, then 0 gets sent to the throttle, otherwise what ever the throttle is gets sent to the throttle.
How many lines of code would you guess that is?
157. [exstatic.org] (including blank lines between functions).
Want to wager how many the .h file [exstatic.org] has?
901.
For that little model right there, there were almost 1000 lines of code. Now do you see how you could easily get 100M?
*This is also quick and dirty, I didn't turn on any optimizations it's just the default C generated code to make a .exe (I didn't target any specific embedded device).
**Now in real production these would pull from sensors and it'd probably use a few more lines of code. (You have to read from the A/D, etc)
Re:You're looking at it wrong. (Score:3, Informative)
I think the point is that while in general "bugs per lines of code" is not a terribly useful metric, bugs per generated/assembly/etc lines per code is even less useful.
It seems like a scare tactic.
Re:You're looking at it wrong. (Score:3, Informative)
The thinking is still fundamentally flawed...
You see... taking an update the yes or the no is questioned because it could cause flaws when the current version doesn't fail.
Well guess what, no-brainers; the current version is flawed.
Just take the damned update and maybe you won't cause a fscking accident. The update could cause a security fail, but it is certain version does cause it.
Re:Not only on the race track (Score:3, Informative)
Sometimes folks step on both pedals to start up steep inclines. You can use the emergency brake as an alternative though.
Doing it wrong.
And yes, I drive a manual.
It's not the floormats! (Score:3, Informative)
Firstly, it's not the floormats. Even Toyota has backed away from that as an explanation. The current theory is that it's the accelerator pedal sticking, but that doesn't jibe well with all of the incident reports either. Given that, I wouldn't count on your driving habits or removing the floormats to solve the problem.
You should also consider that if you have a problem later and the update hasn't been done, guess what they'll blame?!
In general, the modification sounds like a very good idea. If for whatever reason your car decides to go full throttle against your wishes, I'm sure you'd like one extra chance to convince it otherwise.
As others have pointed out, you have already accepted 100 million lines of their code without knowing anything about their software practices.
Heel-and-toe shifting (Score:3, Informative)
Manual transmission drivers don't have three feet, they can't hold the break, clutch and gas at the same time.
You've never done a heel-and-toe [wikipedia.org] shift I guess. Not really disagreeing with your main point (regarding rollback) - just being pedantic and pointing out that it is quite possible for two feet to control three pedals at once. In fact before synchronized transmissions [wikipedia.org] became common it was nothing unusual to need to engage in some fancy footwork. Some race cars still do.
Re:Just try using one on a hill! Re:1st bug found (Score:3, Informative)
He has to release the brake for 1 to 2 seconds so that the car recognizes the brake pedal has been released before it allows the Gas pedal to apply any acceleration to the engine when you start moving.
Citation needed. According to the press release,
Nowhere does it say that you have to let the brakes up for 1-2 seconds before you can use the accelerator.
Re:Cars and Computers (Score:3, Informative)
"Older" as meaning before mid-1970s.
Even those late 70s / early 80s automobiles that seem sans of computers very likely have at least one or more to help meet emission requirements.
Ron
Re:Seriously? (Score:3, Informative)
Citations would have been good. Here they are for reference. There could be more.
Take the update. I got it for my 2009 Camry. (Score:4, Informative)
1) Chopped off about 4cm from the end of the gas pedal. It looks like they did it with a hack saw. The air near the brake pedal smelled like hard plastic that has just been cut.
2) Replaced the old floormat with looked like this:
+-----------+
| |
| |
| |
| |
| |
| |
+-----------+
To one that looks like this:
+---+
| |
+---+ +---+
| |
| |
| |
| |
+-----------+
That way there is a lower chance of the gas pedal touching the floormat. It also means, that the carpet underneath your gas and clutch pedals will get soiled.
3) Updated the firmware. After the update, I did a test where I got the car going 30Mph, and then pressed and held the accelerator. While the accelerator was depressed, I applied the brake with my left foot. After about 1.5 seconds, the engine RPM went down to idle speed. I repeated this test 2 more times. Same result each time.
The firmware update appears to work at least in 3/3 of my test cases.
Re:You're looking at it wrong. (Score:3, Informative)
Which is why I don't like push-button ignition. If my car ever goes into hyperdrive because of a stuck throttle, I take comfort in knowing I still have a kill switch, and I grew up driving tractors and cars without power steering or power-assist braking, so I can cope. How can I trust that that push-button ignition will still shut off the car? I know it's conceivable that even a key-start ignition might turn all ignition control over to an ECM, but who's done that?
Push-button ignition can be turned off by holding down the button (kind of like with a computer). Push-button ignition doesn't stop you from putting the car in neutral.
Re:You're looking at it wrong. (Score:3, Informative)
I hope they didn't use your simple (and informative) example, because if you're stopped at the top of a steep hill (see: San Francisco, city of) you need to use both brakes and accelerator even with an automatic transmission.
As far as I can tell in my re-flashed Camry, hitting the brakes while pressing the accelerator does *not* cut the engine RPM. Of course, I haven't tried this at runaway speeds.
Re:You're looking at it wrong. (Score:1, Informative)
Matlab has an open source equivalent. It is called Octave.
Re:He is looking at it wrong... (Score:2, Informative)
Hill starts are a large part of a UK driving test. If you roll back at all you'll fail. That's what the handbrake is for.
Re:DO-178B for Cars (Score:2, Informative)
I used to work for a automotive software company that does work for the likes of Ford, Mazda, Volvo and thy do pretty much test safety critical parts of the system as much as aviation.
The big element in the gap is aviation using formal methods for verification of the design.
And most of the good players have testing sufficiently automated and systems of design, change, test with reviews at every stage.
Testing typically covers functional unit testing, module testing, system testing. In several ways, on a simulator, on the real hardware being amongst them. Plus the code is usually subject to strict coding standards that would make most programmers weep about being able to express their individual creativity and other crap.
Then there is the extensive use of static analysis and code coverage to make sure that every line of code has been exercised with the tests and if not that review has signed off that it really really is an unreachable piece of code.
You don't move tool chains because by the time you have finally released you know the bugs and have worked around them.
Safety with software in cars amongst 5 car companies I've seen inside of is taken very very seriously. Remember too most of these people drive their own dog food and that includes taking their families in them. So if you trust your quality of work enough to trust your families lives to it good on you.
So I would certainly be taking the updates. That said I like that my motorbike runs on carb's and no ECUs.