20% chance of false drunk driving reading? That's not good. Follow

Let me preface this by saying, I have been hit by a drunk driver, and I'm of the oppinion that if you drive while drunk, it should be legal to kill you in the most painful manner possible at will. That being said, the system needs to have accountability. A drunk driving conviction should destroy your life. A false drunk driving conviction should never be a possibility, as the technology to determine it is very straightforward. The errors in software discovered by the independant review will likely lead to thousands of drunk driving convictions across the U.S. being overturned on appeal, and many of those people will end up driving again.

http://www.schneier.com/blog/archives/2009/05/software_proble.html

May 13, 2009
Software Problems with a Breath Alcohol Detector
This is an excellent lesson in the security problems inherent in trusting proprietary software:

After two years of attempting to get the computer based source code for the Alcotest 7110 MKIII-C, defense counsel in State v. Chun were successful in obtaining the code, and had it analyzed by Base One Technologies, Inc.
Draeger, the manufacturer maintained that the system was perfect, and that revealing the source code would be damaging to its business. They were right about the second part, of course, because it turned out that the code was terrible.

2. Readings are Not Averaged Correctly: When the software takes a series of readings, it first averages the first two readings. Then, it averages the third reading with the average just computed. Then the fourth reading is averaged with the new average, and so on. There is no comment or note detailing a reason for this calculation, which would cause the first reading to have more weight than successive readings. Nonetheless, the comments say that the values should be averaged, and they are not.
3. Results Limited to Small, Discrete Values: The A/D converters measuring the IR readings and the fuel cell readings can produce values between 0 and 4095. However, the software divides the final average(s) by 256, meaning the final result can only have 16 values to represent the five-volt range (or less), or, represent the range of alcohol readings possible. This is a loss of precision in the data; of a possible twelve bits of information, only four bits are used. Further, because of an attribute in the IR calculations, the result value is further divided in half. This means that only 8 values are possible for the IR detection, and this is compared against the 16 values of the fuel cell.

4. Catastrophic Error Detection Is Disabled: An interrupt that detects that the microprocessor is trying to execute an illegal instruction is disabled, meaning that the Alcotest software could appear to run correctly while executing wild branches or invalid code for a period of time. Other interrupts ignored are the Computer Operating Property (a watchdog timer), and the Software Interrupt.


Basically, the system was designed to return some sort of result regardless.

This is important. As we become more and more dependent on software for evidentiary and other legal applications, we need to be able to carefully examine that software for accuracy, reliability, etc. Every government contract for breath alcohol detectors needs to include the requirement for public source code. "You can't look at our code because we don't want you to" simply isn't good enough.

I would never submit to a breath test. Now I have yet another reason to justify it.

You are allowed a blood test at the station if you refuse the breath test.

I'm relatively sure (though not even close to certain and a quick google could not find the answer) that in Australia the preliminary breath test is taken, and then a blood test.

Although the blood test may be for prosecuting purposes as opposed to on the spot testing for alcohol.

But still scary and stupid that they use such imprecise technology (same as speed camera's, there was an outcry against dodgy speed cameras a few years ago that cost the government $19mil in fines repaid)

Not necessarily. There are systems in which you want to use an averaging process like that. Some measuring devices are wildly inaccurate at first, then "settle" on a value over time. The resulting output will look like a wave form with a wide amplitude over the first X amount of time, then decrease towards a normative middle over time. If you take measurements of that waveform at regular intervals during a given time frame, you can get very inaccurate results because of this effect.

When you have a form like that, you want to average in the manner mentioned. It puts more weight on the "settled" values at the end of the wave form than the more variant measurements at the beginning. It's a little hard to explain, but if you see the results on a graph, you'll understand why it's a useful technique.


I suspect this is more of a documentation error than an actual process error.


The more relevant point is whether or not you think the system has flaws based on the software design, but whether or not the results consistently meet the requirements. Kao wrote in his title about a 20% chance of a false reading, and maybe I'm missing something, but I don't see where that's written in either the story he linked, or the link contained on that page to the court case.

Does the device fail to produce accurate results? That's the only relevant question. They could use the most bizarre process imaginable, but if it's able to consistently recognize when people are too drunk to drive, then it's working properly.

Now I'm just a layman but if you are driving around after drinking even 60% of a bucket of beer (even that wussy American ****) then you definately deserve to lose your licence.

Well, to Nit Pick, there is no "Legal Limit" read the laws if interested.

I'm not quite sure I take your point. In California the law reads as follows:

Driving Under Influence of Alcohol or Drugs

23152. (a) It is unlawful for any person who is under the influence of any alcoholic beverage or drug, or under the combined influence of any alcoholic beverage and drug, to drive a vehicle.

(b) It is unlawful for any person who has 0.08 percent or more, by weight, of alcohol in his or her blood to drive a vehicle.

For purposes of this article and Section 34501.16, percent, by weight, of alcohol in a person's blood is based upon grams of alcohol per 100 milliliters of blood or grams of alcohol per 210 liters of breath.

In any prosecution under this subdivision, it is a rebuttable presumption that the person had 0.08 percent or more, by weight, of alcohol in his or her blood at the time of driving the vehicle if the person had 0.08 percent or more, by weight, of alcohol in his or her blood at the time of the performance of a chemical test within three hours after the driving.

(c) It is unlawful for any person who is addicted to the use of any drug to drive a vehicle. This subdivision shall not apply to a person who is participating in a narcotic treatment program approved pursuant to Article 3 (commencing with Section 11875) of Chapter 1 of Part 3 of Division 10.5 of the Health and Safety Code.

(d) It is unlawful for any person who has 0.04 percent or more, by weight, of alcohol in his or her blood to drive a commercial motor vehicle, as defined in Section 15210.

Or do you really think the final output of the machine can either show .06% or .12% and nothing in between. If whomever did that math was correct, there should be no way to get a value in between.

No.

No math classes required for what you do, either, I guess.

Sorry but the test equipment you use needs to be sent out to be calibrated and isn't likely to be of the quality needed for scientific testing.

When I was kid, it was like comparing General Radio noise meters to Brüel & Kjær from Denmark. My father only would use equipment from B&K as the General Radio meters were so inadequate as to be useless for his purpose. Then he was on the forefront of Acoustics for housing and airport noise abatement and nothing else would do.

These days there are many more brands to choose from, but if you want the best results from your testing equipment, you best be ready to pay for it and make sure you have it re calibrated each month by qualified technician. (My ex was the best o-scope technician, and could calibrate more equipment faster then anyone else in the shop, at Ft Meade until he started to live full time as woman.)

I initially though I'd be snarky with my response, but I'll assume you honestly don't know what equipment I'm talking about.

We test SOC packages. That stands for "system on a chip". These are basically computer chips used in a variety of communications equipment. The testers we use cost upwards of a million dollars apiece, have counterweights weighing around a ton, and most definitely are not "sent out" for calibration. We have our own in-house maintenance staff who, among other things, do regular calibrations every 30 days. Full NIST cals are done on site (with specially manufactured and calibrated robots) every 6 months on each system.

We specifically use a mix of mostly Teradyne Ultraflex and Verigy V93000 testers. We also have an assortment of wafer probers in use (Accretech and Seiko-Epson IIRC), so we can actually probe raw wafers before they are cut up and put into a package.

Outside of clean rooms at fabrication plants, there just isn't much more precise "scientific testing" you can do.


If you'd like, I could go into detail about how we use yield analysis software to produce accurate sample set results while only testing a small fraction of the whole sample. Sometimes, methods that don't appear to be very accurate to the lay observer are actually mathematically provable to be very very accurate. I'm not going to assume that the methods used in the Breathalyzer must be producing inaccurate results, just because a vague description of the methods seem like they should be.

Some background on how the machine works:
http://www.nj-dmv-dwi.com/parts/breathtest.html
Should be noted that is a potentially biased site, but their description of the basic mechanics seems accurate based on what I know.

There is also a picture of the unit. These are apperently rather old, and likely do not have much in the way of processor horsepower. I suspect they deliberatly limited the range of possible values to attempt to increase unit speed.

I'll see if I can find the source code somwwhere, but they seem to be holding it pretty close to the vest.

That's the part I comprehended immediately. So what the heck is your problem? You're angry because I correctly identified what was going on?

You didn't correctly identify what was going on. You wrongly assumed something was being argued that wasn't. Because you either didn't understand the math, or can't read. Having read thousands of your posts, it's really a toss up, but I'd have to say that your reading comprehension is superior to your math ability, which really doesn't speak well for that latter one.

Also, I'm not angry. I don't think I've ever been angry at anything you post. I frequently laugh at the idea that people take you seriously, but for you to move me to anger would require some level of accurate insight on your part that I was uncomfortable with. Joph's made me angry before, you, as of yet, have not.