Fun Facts about Domestic Violence

Welmer points to a 2007 study on domestic violence by CDC researcher Daniel Whitaker, published in the Journal of Public Health. Here are the results:

Weighted Estimates of Violence Occurrence for Reciprocally and Nonreciprocally Violent Relationships

Variable	Overall %	Men %	Women %
All relationships
- Nonviolent	76.1	80.7	71.6
- Violent	23.9	19.3	28.4
Among violent relationships
- Reciprocal	49.7	46.9	51.5
- Nonreciprocal	50.3	53.1	48.5
Among cases with nonreciprocal IPV
- Perpetrated by men	29.3	25.1	32.3
- Perpetrated by women	70.7	74.9	67.7

A couple of thoughts on this. First, I will happily betray my middle-class innocence by saying that 23.9% sounds like an awfully high number of heterosexual relationships in which violence occurs. But note that the survey questions (“How often in the past year have you threatened your partner with violence, pushed or shoved him/her, or thrown something at him/her that could hurt,” and “How often in the past year have you slapped, hit, or kicked your partner”) include both threats of violence and attempts at violence in the same category as, you know, actual violence. (Though one could argue that the willingness of respondents to characterize mere threats and attempts as "violence" is a measure of their psychological impact.)

The study gives us an idea of the prevalence of actual violence by asking about injury. Again the results:

Weighted Estimates of Injury Occurrence by Reciprocity Status and Perpetrator Gender

Variable	Injury occurrency (%)
- Nonreciprocal	11.6
- Reciprocal	28.4
Perpetrator gender
- Men against women	28.8
- Women against men	18.1
Gender by reciprocity
- Men against women: nonreciprocal	20.0
- Men against women: reciprocal	31.4
- Women against men: nonreciprocal	8.1
- Women against men: reciprocal	25.3

Note that "injury" as used herein refers to a "sprain, bruise, or cut" and, presumably, anything worse. This, I submit, is a minimum threshold for what constitutes actual "domestic violence" as opposed to simple boorishness.

My second thought is: wow, women inflict 70% of the non-reciprocal domestic violence on men! But let's limit our consideration to violence causing injury. Looking at the non-reciprocal violence numbers, my back of the envelope calculation is that (.293)(.2)/[(.293)(.2) + (.707)(.081)] = 51% of non-reciprocal, injury-producing domestic violence is perpetrated by men, leaving the other 49% to be perpetrated by women.

But what about reciprocal violence, i.e. men and women deciding to have at it together? Waving aside, for the moment, whether this constitutes as morally deserving a category as non-reciprocal violence, we see that 31.4% of women in reciprocally violent relationships suffered injury, while only 25.3% of men in reciprocally violent relatonships suffered injury. (Keep in mind that, as the violence was reciprocal, these categories are not mutually exclusive -- no doubt some relationships included injuries to both the man and the woman.)

Looking at the raw numbers, we can combine the categories and see that of the injuries sustained in domestic violence, 55% were suffered by women while only 45% were suffered by men. This isn't as dramatic as the 70% mentioned above, but does illustrate that the popular notion that domestic violence is suffered exclusively by women appears to be a misconception.

A few caveats. First, the study only considers heterosexual couples ages 18 - 28. Second, while the study did include a racially representative sample, racial correlations were evidently not part of the study. (The original data can be downloaded here, I think.)

Finally, the study has only one injury category. It doesn't break out separately those DV victims receiving "the full Rhianna", i.e. victims suffering injuries of sufficient severity to merit either professional medical and/or law enforcement attention.

Still. This study provides a lot of nuance on the question of who perpetrates domestic violence.

GRE Percentiles: The Affirmative Action Effect

In my last post, I examined the graduate school exam percentile score data as published by Steve Sailer, and discovered that the low percentage of black college graduates scoring above the white median (~16% for the GRE) is partially but not fully explained as an artifact of the different distributions of ability between blacks and whites: specifically, that the black mean in cognitive ability is one standard deviation below the white mean.

In this post, I will examine whether affirmative action in college admissions affects these results. Based on my initial model, the answer, by inspection, would be: no effect. No student, black or white, magically acquires cognitive ability by the mere fact of college admittance. If +1 SD ability (i.e. an IQ of 115) is necessary to graduate from a competitive college, then admitees with lower ability would simply fail to graduate.

But this analysis ignores the fact that college admissions criteria only estimate a student's ability. There is no bias in these criteria, but there is error: as any college registrar can tell you, some students do, in fact, outperform their admissions portfolio . . . and other students underperform it. I myself saw students with 800 math SAT scores earn C's in calculus. I also saw students with a score of 540 earn the same C.

Thus, students taking the GRE have met two complementary standards: they tested well enough to meet the college admissions criteria, and then they performed well enough to graduate. So, if a college relaxes its admissions standards for black applicants, but maintains the standards for graduation, some of the affirmative action admits will succeed in meeting the higher standard. As has been amply documented, such policies come with great human costs: the greater majority of the AA admits will not graduate, but enter the work force with less work experience, a mountain of debt, and no credential.

Our extended analysis makes use of a two-dimensional probability distribution f as a function of tested ability t (in other words, their composite admissions portfolio as determined by the college) and demonstrated performance p (their actual ability in college). This pdf is defined by the equation:

Let's go over these variables one at a time: μ-bar is a vector of the tested mean μ_t and the performance mean μ_p. σ_t and σ_p are the tested and performance standard deviations. ρ is the correlation coefficient, a measure of how much variation exists between tested ability and demonstrated ability. If ρ = 1, then the admissions tests would absolutely predict performance; this was the unstated assumption in my previous post. If ρ = 0, then the two variables bear no relationship to each other. In practice, ρ will fall between 0 and 1. Σ is called the covariance matrix and is a convenient way of representing the data.

As before, we will model the white population as a standard distribution: μ_t = μ_p = 0 and σ_t = σ_p = 1. (There is no reason to suppose the tested and performance means and variances differ with respect to each other. I will arbitrarily set ρ = 0.5. For those who think this estimate is too low, let me say that from my experience, admissions criteria are not nearly as accurate a predictor as most people think. Using my earlier example, I once ran a regression between math SAT scores and performance on Calc I. The correlation coefficient was about 0.3. I imagine that other factors (GPA, extra curricular activities, etc) improve on that estimate, but I would be surprised if we exceed 0.5. So our 2D pdf is now:

As before, we will expect our GRE-takers to have met a +1 SD standard along both the tested and the performance dimensions. I have started to question this parameter, because the number of students meeting both standards is (by my calculation; your mileage may vary) only 6%, down from 16% for a single standard. This implies that more than 60% of students testing above +1 SD fail to perform at that level. Since graduation rates are higher than that at competitive colleges, then assuming my other parameters are correct, I am guessing that many if not most of their admissions committees build a margin of error into their admissions standards.

Returning to the GRE: assuming that GRE takers have successfully met the +1 SD standard along both dimensions, we can calculate the median performance p_med by:

where t_c = p_c = 1. I computed this as p_med = 1.5, slightly higher than the result for as single dimension.

We will model the distribution of black ability as:

We will then calculate the percentage of black test takers who exceed p_med:

If we assume that black test takers were admitted without affirmative action, i.e. the admissions cuttoff for blacks is the same for whites, t_cb = t_c, then the percentage of blacks meeting the white median climbs to 38%. But let's assume that the college has a quota for blacks, and admits them up from +1 SD from the black mean or up from the white mean of zero. If t_cb = 0, then %_med = 31%.

In conclusion, the affirmative action assumption, applied to our two-dimensional model, brings our calculated value for the black percentile a little closer, but still some distance from the actual percentile of 16%.

Affirmative Action-free GRE Scores

Steve Sailer publishes a table of graduate school admissions tests results, broken out by race. Steve explains:

To make all the numbers comprehensible, I’ve converted them to show where the mean for each race would fall in percentile terms relative to the distribution of scores among non-Hispanic white Americans. Most of us have some sense of what the distribution of talent is among whites—political correctness doesn’t demand we avert our eyes when it comes to whites—so I’ll use whites as benchmarks.

Here is is table:

Mean Score as Percentile of White Distribution

Test	Degree	Year	White	Black	Asian	Tot Hisp	Mex-Am
GMAT	M.B.A.	2008	50%	13%	55%	27%	24%
GRE-Verbal	Ph.D./M.A.	2007	50%	18%	47%	29%	28%
GRE-Quant		2007	50%	14%	66%	29%	28%
LSAT	J.D.	2006	50%	12%	47%	19%	29%
MCAT-Verbal	M.D.	2007	50%	10%	36%/td>	19%	21%
MCAT-Phys Sci		2007	50%	14%	61%	24%	25%
MCAT-Biol Sci		2007	50%	10%	54%	24%	25%
DAT	D.D.S.	2005	50%	16%	60%	27%	NA

Steve has his own commentary on what these numbers mean. What I want to examine is the median black score as a percentile of the white median. Looking at the GRE scores, for instance, we see that only 16% (cumulative estimate) of blacks scored above the white mean. This is extraordinary when you consider that the test is presumably taken exclusively by college graduates, among which the black representation presumably reflects their -1SD mean cognitive ability.

But what should the black percentile be? Our initial guess might be: 50%. After all, whatever affirmative action might be present in college admissions, blacks have presumably met the same graduation standards, correct?

Actually, no. First, there is abundant evidence that black graduates are overrepresented in less-demanding fields of study: elementary ed, say, rather than engineering. But second, even aside from academic specialty, there is additional reason to expect that black GRE scores would be lower: the distribution of test takers is not normally distributed; rather, they are distributed along the upper tail of a normal distribution.

Consider a hypothetical. Let's assume that college graduates taking the GRE possess a minimum +1SD academic ability above the white mean. This strikes me as reasonable. Granted, many more students than this attend college, but the dropout rate is high, and will will assume at present that the just-got-by students don't aspire to professional school. Thus, in the graph below, if the blue line represents the distribution of academic ability among the white population (μ = 0, σ = 1), the shaded area represents the distribution of white GRE test-takers.

It is from the shaded area only that the median white score (in SD's) must be calculated. For the math geeks, given a cutoff score of t_c = 1, we are trying to find the median score t_med:

For the bit-heads, the Matlab code looks like this:

t_c = 1;

t = t_c:.001:(t_c + 2);

t_med = ...

max(t((1 - normcdf(t))./(1-normcdf(t_c))>.5))

Given my assumptions, t_med turns out to equal around 1.4 SD. Thus, assuming no selection bias or other means of skewing the results, the black percentile score would be calculated by comparing the percentage of the black distribution (μ = -1, σ = 1) that exceeds the median white ability t_med = 1.4 to the percentage of the black population that exceeded the cutoff for taking the test, t_c = 1:

Again with the Matlab:

percentile = ...

(1 - normcdf(t_med,-1,1))/(1 - normcdf(t_c,-1,1))

This percentile score turns out to be about 35%. In other words, lower than 50%, but not as low as the measured value of 16%. How can we account for the difference?

Steve provides some additional data on the number of test takers among the different ethnic groups as compared to their representation in the population. It turns out that blacks have only half the representation among test takers (e.g. 47% for the GRE) as their percentage of the population would predict, which sounds low . . . until you consider that the black share of those with +1SD academic ability is only 14% of their percentage of the population.

Well, I did make up the +1SD standard. Is there an ability cutoff whereby black representation among test takers would be 50% of its representation in the population? For those of you keeping score at home, we are trying to find:

t = -1:.001:0;

t_c = ...

max(x((1 - normcdf(t,-1,1))./(1-normcdf(t))>.5))

t = t_c:.001:0;

t_med = ...

max(t((1 - normcdf(t))./(1-normcdf(t_c))>.5));

percentile = ...

(1 - normcdf(t_med,-1,1))/(1 - normcdf(t_c,-1,1))

The solution to the standard t_c to this equation turns out to be -0.68 SD. Not only is this number absurd (equivalent to pretending that an IQ of 90 is sufficient to graduate from college), but it still only gives a percentile score of 25%. In other words, it's closer to the measured value of 16%, but still not close enough. In fact, the only standard by which only 16% of black test takers would exceed the white median would be: no standard. In other words, if we were to administer the GRE to the entire population (or some random sample thereof), the white median would be zero (in a standard distribution), and 16% of blacks would exceed it. Bear in mind what we are measuring: as the standard goes up, the black representation among those that meet it decreases; however, among the blacks that do meet the standard, the percentile score compared to the white median improves.

In order to account for the black percentile score, it is necessary for us to drop the assumption of a single standard. Clearly, the GRE and other graduate school exams dip much further in the pool of black college graduates than they do in the pool of white college graduates. How much more? Well, let's return to our assumption of a t_c of +1SD, but apply it only to the white students. What would the standard applied to black test takers (herewith t_cb) be in order to lower the percentile score to 16%?

t_c = 1;

t = t_c:.001:(t_c + 2);

t_med = ...

max(t((1 - normcdf(t))./(1-normcdf(t_c))>.5))

t = 0:.001:t_c;

t_cb = ...

max(t((1 - normcdf(t_med,-1,1))./(1 - normcdf(t,-1,1))<.16))

The value thus calculated for t_cb is 0.64. This is equivalent to an IQ cutoff for black college graduates of 110 versus an IQ cutoff for white college graduates of 115. In conclusion, while we can partially account for the black percentile score on the GRE by improving our model of the distribution of test-takers, we cannot account for all of it. It is therefore fairly evident that Steve's analysis is almost certainly correct: the GRE and other graduate school exams dips much further into the pool of black college graduates than it does in the pool of white college graduates. Why this is, and indeed why there is a deeper pool from which they can draw, is a question that merits more examination.

On Bias

A while back, I made a comment over at Bobvis to the effect that the word bias had a very specific mathematical meaning: it was the tendency to consistently over-estimate or under-estimate a parameter.

To illustrate this concept, let's return to the Poisson distribution from my last math post. You will recall that the Poisson pdf,

p(n|a) = an·exp(-a) n!

(1)

is related to the probability of some number of events n, known to occur at average rate a, occuring within the time period specified by a. In my last example, we spoke of the number of busses that would drive by a bus stop. We could also use the example of the number of photons received by a telescope, given they arrive at average rate a. We also showed that the expected value of n was a, and that the variance of n was also a.

But: what if we don't know the parameter a? Can we calculate it? Indeed, the straightforward calculation of the expected value of a is:

E[a] = ó õ a p(n|a)p(a) p(n) da,

(2)

where

p(n) = ∫p(n|a)p(a)da

(3)

However, calculating the E[a] in this manner not only requires our Poisson distribution, but also requires a priori knowledge of p(a), the distribution of a. In practice, we may not have this formula.

In such a situation, we can't calculate the expected value of a directly. But if we have empirical measurements, we can estimate the parameter a from the data itself. There are, in fact, a number of estimators, but in this case, the estimator of choice would be the Maximum Likelihood estimate, defined as:

^ a   ML  = argmaxa p(n|a) = argmaxa   an·exp(-a) n!

(4)

The expression argmax_a means the that we will vary a until we maximize the value of the pdf, given that we have made a measurement n of some number of photons (let's say) striking our telescope.

The value of the pdf will take on its maximum value with respect to a when its derivative with respect to a is equal to zero; so

d da p(n|a) = d da æ è   an·exp-a n! ö ø = (n·an-1-an)exp(-a) n! = 0

(5)

Þ n· ^ a   n-1 ML  = ^ a   n ML

(6)

lnn + (n-1)·ln ^ a   ML  = n·ln ^ a   ML

(7)

ln ^ a   ML  = lnn

(8)

^ a   ML  = n

(9)

In other words, I can estimate the rate of photons a by counting the photons. You might expect that this would be a very poor estimate, and you would be right; it's roughly the same as estimating the mean IQ of China from the IQ of the first Chinaman I meet.

But what if we have a collection of measurements that we will call N = {Ni, i = 1¼M}. In our example, this is like having M telescopes that each make a reading of the number of photons Ni in the relevant time period. Because these measurements are independent, we can write:

^ a   ML  = argmaxa p(N|a) = argmaxa   Õ p(Ni|a) = argmaxa   Õ aNi·exp(-a) Ni!

(10)

= argmaxa  aåNi·exp(-Ma)· Õ 1 Ni!

(11)

Note: all S's and P's are over the M samples.

As above, the value of the pdf will take on its maximum value with respect to a when its derivative with respect to a is equal to zero; so

d da é ë aåNi·exp(-Ma)· Õ 1 Ni! ù û

(12)

= é ë å Ni·aåNi-1·exp(-Ma) - M·exp(-Ma)·aåNi ù û Õ 1 Ni! =0

(13)

®   å Ni· ^ a   åNi-1 ML  = M· ^ a   åNi ML

(14)

Therefore,  ^ a   ML  = 1 M · å Ni

(15)

This conclusion may seem unremarkable: that the maximum likelihood estimate for, say, the number of busses driving visiting my bus stop in an hour, or the number of photons collected by my telescope in an hour, is the average number of observations made during a succession of hours.

The question still be answered is whether or not this is an unbiased estimate. Not a perfect estimate, mind you; there of course will be errors. The pertinant question is whether these errors will tend to accumulate in a particular way, predicting too many photons, on average, or too few.

To answer this question, we must calcuate the expected value of the estimate:

E[ ^ a   ML  ] = E é ê ë å Ni M ù ú û = 1 M · å E[Ni] = M·a M = a

(16)

Conveniently enough the expected value of the ML estimate of a is ... a. We say then that the esimate is unbiased; in other words, while any single estimate will have an error associated with it, these errors will not, on average, tend either higher or lower in the long run. average of a sufficiently large number of estimates will not be wrong.

But how much can I expect my ML estimate to be wrong? We can gain some insight into this question by estimating the variance. Since the Ni's can vary, our estimate will vary while the true value stays constant. What is the extend of this variation?

var ^ a   ML  = E[( ^ a   ML  - a)2] = E é ê ë æ ç è å Ni M - å a M ö ÷ ø 2   ù ú û

(17)

= E é ë æ è 1 M · å (Ni - a) ö ø 2   ù û = E é ë 1 M2 · æ è å (Ni - a) ö ø 2   ù û

(18)

The next step requires some explanation. We have a square-of-sums term that we wish to convert to a sum of squares.

Normally this would generate all kinds of cross terms; however, because our samples our independent, we know that E[(Ni - a)(Nj - a)]=0, so we can write:

var( ^ a   ML  ) = 1 M2 ·E é ë å (Ni - a)2 ù û = 1 M2 · å E[(Ni-a)2]

(19)

= 1 M2 · å var(Ni)= Ma M2 = a M

(20)

Here is the significance of what we've just done: we've shown that as the number of our observations goes up, the variance associated with our estimate goes down. So the larger the sample size, the closer I will usually be to correctly estimating the parameter I am trying to find.

Why is this? Basically, signals reinforce, noise cancels! To deploy an example that will interest the readers of this blog, if I test the IQ of a random Chinaman, my guess that this IQ is the Chinese average will probably not be a very close one, although my estimate will be unbiased. If I meet a thousand random Chinamen, my guess that their average IQ is the Chinese average will be pretty darn close.

Update: Crap!Crap!Crap!Crap!Crap! My equations look like they went through a blender!

Here's the story: Last week I wrote my first paper in LaTex. Once you learn the syntax, formatting equations in LaTex is mildly easier than in MS Word's Equation Editor or MathType, since it can all be done with control sequences instead of mouse-work. But I was particularly taken with the possibility of converting a LaTex file into HTML using a little bit of freeware called T_TH. Unlike, say, the HTML conversion by MS Word, the equations are formatted as real HTML, not gif files.

But there turned out to be numerous problems. First, T_TH doesn't recognize all LaTex syntax. The second problem is blogger. When I open the T_TH output in a browser, it looks fine. When I save that same output into blogger and read it there . . . well, what you see above is the result of many hours of post-processing on a very complicated source file. And it still removed all my fraction bars. So all those over-under strings that look like they'd be fractions if they had a line beteen them, well . . . they once did.

Never again. If I'm writing for the Web, I'll use word.