Deconstructing a problem using QL competencies

The problem

The question and answer below appeared in Marilyn von Savant’s column in the April 10 edition of Par= ade magazine. Its brevity and apparent simplicity belie the deep roots of knowledge that a reader interested in the accuracy and veracity of the an= swer must possess. In this analysis of the problem I will explore the elements= of both the question and the answer in detail. My goal is to expose the high= er order skills that are required to understand where the reader went wrong, test the accuracy of the calculations that are embedded in the answer, ma= ke judgments about the reasonableness of the conclusion, and show how, as Marilyn herself states, “When statistics like this are taken out of conte= xt, they can be misleading.”

The question

“If half of the children 14 and under who die in car crashes are = not buckled, boostered or otherwise restrained, doesn’t this mean that half of the children are appropriately secured? If so, wouldn’t this also mean th= at the chances of a child surviving a crash are 50-50, restrained or not?”

The answer

“Yes to the first question, but not to the second. When statistics like this are quoted out of context, they can be misleading. You need more information. To illustrate, suppose that 90 percent of children involved = in car crashes (not just fatal ones) are secured, and 10 percent are not. Now say that 20 percent of these accidents cause a fatality, half with the children secured and half with a child who is not. This would mean that e= very unrestrained child involved in an accident was killed, but only one out of nine restrained children was killed. You’d draw a very different conclusi= on, wouldn’t you?”

Comprehending the rea= der’s question

The first part of the reader’s question seems straightforward. The group of interest includes children u= nder 14 who die in car crashes. The first part of the question states the appa= rent fact that half of that group was not secured. The second simply asks whet= her the other half was secured. It implies that there is no other alternative= to being secured or not. That seems logical, but we will see later that, fro= m a data perspective there is a third alternative, which is that we just do n= ot know.

The second question the reader ask= s is more complicated. Whereas in the first case the group of interest included only those who died in car crashes, this question is really drawing a conclusion about the fraction surviving. Therefore, the group of interest= is all children involved in car crashes. In infers something about one group from data about another group and that is where the reader goes wrong. As Marilyn says, “You need more information.”

Comprehending the ans= wer

Marilyn states that the answer to = the first question is yes. As indicated above, this assumes no third alternat= ive, a fair assumption in reality, but not from a data perspective. In any cas= e, all the alternatives have to account for 100% of the group.

The next part of the answer says t= hat statistics like these can be misleading when quoted out of context and th= at more information is necessary. The statistic itself is not misleading—it = is a fact that has been used incorrectly because reader did not account for the fact that the group of all fatalities is not the same as the group of all students involved in crashes. He jumped from correctly interpreting the statistic about one group to making an inference about another group with= out enough information.

The remainder of the answer is dedicated to working through a hypothical set of data. It supposes some a= dditional percentages that, when combined with what the reader knows, can be used to draw conclusions. The task in this deconstruction now becomes one of prov= ing that her conclusions are correct.

Structuring the probl= em for solution

The first supposition is “…that 90 percent of children involved in car crashes (not just fatal ones) are secured, and 10 percent are not.” This means that we are now dealing with= a group that includes all children involved in all car crashes and that they have been divided into secured (90%) and not secured (10%).

The second supposition is “…that 20 percent of these accidents cause a fatality, half with the children secur= ed and half with a child who is not.” We are now dealing with a group that includes all car crashes, only 20% of which involve a fatality.

So, to sum up, we have statistics = on three groups: the first is the group of children in which everyone dies, = the second is all children involved in accidents, and the third is all accide= nts. Structuring all this information into a form that lends itselfs to drawing conclusions is the challenge that we now face. It is a this stage that the higher order skills embodied in the literacy part of quantitative literacy come into play. That is you should have been exposed to something somewhe= re in your education that seems similar to the problem at hand.

For example, when faced with a collection of groups like this one might dredge up the concept of Venn diagrams from a high school or college math course. The stimulus is, of course, the idea that fractions of these groups have some things in common and some things not in common. One example might be that of the 90% of all children involved in car crashes some were secured, a characteristic they share with half of the 20% of all children who died in car crashes. Before reading on I suggest that you try to construct some diagrams to capture t= hese intersections.

If you attempted to construct some Venn diagrams you should have come up with something like the one shown here. If you tried at all to represent the proportions of children in the various groups by size of the various circles or squares, you probably found it pretty confusing. These diagrams do not lend themselves to solving this problem with the numbers = as presented so some other appoach is necessary. They do provide a clue, however.

One of the fundamental QL Core Competencies is that students should be able to read, interpret and draw conclusions from tables. They are similar to Venn diagrams in that you can overlap characteristics, but they lend themselves quite nicely to quantitative solutions. Somewhere in your education you should have been = exposed to this task. If using tables came to mind then you can much more easily structure the problem for solution.

Consider the following steps in wh= ich the cells of the tables correspond to the sets in the diagram above. The color coding of the four cells in the middle corresponds to the colors of= the sets in the diagram above.

Step 1. Where to put the first supposition<= /span>

“To illustrate, suppose that 90 percent of children involved in car crashes (not just fatal ones) are secured, and 10 percent are not.” These two percentages are the row total= s in the following table since they include both non-fatal and fatal injuries.= You can see immediately from where these numbers go why the Venn diagram did = work very well. It is the color coded numbers that we need, not the “marginal” numbers.

<= /td>	Percent of children in= car crashes
	Non-fatal	Fatal	Total
Secured			90%
Not secured			10%
Total			100%

Step 2. Where to put the second supposition=

“Now say that 20 percent of these accidents cause a fatality, half with the children secured and half with a child who is not.” The total percentage of fatal injuries out of all inju= ries is 20% so that number is entered as a column total, another “marginal” nu= mber. Since half the children in the fatal injury column are secured and half a= re not, the 20% gets distributed equally between the two conditions. These a= re the first two data cells that can be filled.

<= /td>	Percent of children in= car crashes
	Non-fatal	Fatal	Total
Secured		10%	90%
Not secured		10%	10%
Total		20%	100%

Step 3. Fill in the remaining cells of the table<= /o:p>

Since all the row and column totals have to add up so the table total is 100%, the last three cells are fille= d in by simple subtraction.

Thus, as Marilyn concluded, the 0%= in the Not Secured/Non-fatal cell indicates that no unsecured child survives. Or, said another way, 100% (10%/10%) of the unsecured children are in the fatal column.

<= /td>	Percent of children in= car crashes
	Non-fatal	Fatal	Total
Secured	80%	10%	90%
Not secured	0%	10%	10%
Total	80%	20%	100%

Now that we have constructed the hypothetical, let’s go back to the reader’s questions and re-examine the answers. The first was “…doesn’t this mean that half of the children are appropriately secured?” Marilyn’s answered said that the answer to this quesiton was yes. This is true if you only consider the fatal column, however. So that answer is simply a repeat of the statistic that prompted= the question in the first place. The way the hypotheical is constructed, she actually set it up so 90% of ALL children are appropriately restrained.

The second question was, “If so, wouldn’t this also mean that the chances of a child surviving a crash are 50-50, restrained or not?” The answer is clearly no since she set the hypothetical up so that 20% of the injuries were fatal, restrained or not= .

It is clear that the answer she wa= nted by providing “more information” was the 0% survival for unrestrained children. If this were true, it would certainly be strong motivation for = all parents to restrain all kids, but is it true? As a matter of common sense would one really expect no survivors among all the unrestrained children = who are injured? Not really.

Testing the veracity = of the hypothetical

When thinking about this kind of p= roblem it is important to ask whether the numbers make sense or not, i.e., to ma= ke a judgment. If not, then what does make sense?

I expect that very few people actu= ally carry accident statistics around in their heads so a little research is required. Though it was not necessarily a straightforward search I found = the relevant statistics on the web= at = http://www-nrd.nhtsa.dot.gov/pdf/nrd= -30/NCSA/TSFANN/TSF2005.pdf. (Traffic Safety Facts 2005: A Compilation of Motor Vehicle Crash Data from the Fatality Analysis Report= ing System and the General Estimates System, National Highway Traffic Safety Administration, National Center= for Statistics and Analysis, U.S. Department of Transportation, Washington, DC 20590.)

This document contains a wealth of information. The most appropriate table can be found on page 119 (135 in PDF). From that table we can reconstruct tables similar to the ones above. There is a catch, however. The real data include the fact that it is often unknown whether a child was restrained or not. It turns out that this additional (lack of) information does not change the conclusions significantly when more appropriate calculations of risk are done, howeve= r.

The following tables show the corr= ect data for children 15 and under. The first one includes the actual numbers= and the second shows the calculated percentages.

<= /td>	Number of children <=3D 15 injured in car/light truck crashes
	Non-fatal	Fatal	Total
Restrained	191,000	744	191,744
Unrestrained	25,000	740	25,740
Unknown	13,000	133	13,133
Total	229,000	1,617	230,617

	Number of children <=3D 15 injured in car/light truck crashes
	Non-fatal	Fatal	Total
Restrained	82.8%	0.3%	83.1%
Unrestrained	10.8%	0.3%	11.2%
Unknown	5.6%	0.1%	5.7%
Total	99.3%	0.7%	100.0%

The first thing to notice is that = the numbers of fatalies among restrained and unrestrained children are essentially equal. This agrees with the statistic that prompted the reade= r’s original question. But that is about the only thing that corresponds to the hypothetical, however. It is clear that the assumption of 20% of all inju= ries being fatal is not even close to reality! It is actually less than 1%. No= r is the 90%-10% split between restrained and unrestrained correct.

At first glance the numbers and percentages in these tables seem to diminish the point that Marilyn was apparently attempting to reinforce, that being the there is a very real advantage to child restraints. There are a number of ways of drawing this conclusion from the real data and I leave it to the reader to do so. As a starting point one could, for example, simply ask how many lives would ha= ve been saved had all children sustaining injuries been restrained. There ar= e, however, much better calculations that can be done to assess the risk of fatalities when children are unrestrainged.