Monday, June 29, 2009

Fisking Emily Glassberg Sands

I finished reading the much-discussed Emily Glassberg Sands study on sexism in the theatre this weekend (and even had a phone conversation with Sands afterward). I must report, however, that the conversation grew "contentious" (as Sands herself put it) as I pressed her on issues regarding the final section of her paper - the part devoted to Broadway productions.

In a nutshell, as I read her paper, I began to have doubts about her data - both her sample of "female-written shows" (which seemed to include musicals which had a female book writer, but a male composer and lyricist), and her "proxy" for the profitability of the shows she was considering. It hadn't really come clear to me when I viewed the PowerPoint presentation that Sands gave in New York that she didn't actually have any hard data about the profitability of these shows (although on a second look, you could probably construe this from the fine print of her tables, and she may have mentioned it during the actual proceedings).

But this issue struck me as key to her argument that producers had ended the runs of female-written shows "earlier" than those of male-written shows. After all, a simple explanation for this might be that producers ended the female-written shows because they weren't as profitable as the male-written shows - perhaps they had higher production costs, for example, which cut into their weekly revenues (or a major star left the production, etc.).

But Sands didn't have this data (which in the text of her study she admits openly - it's hard to come by). Instead, she estimated the variable of production cost by a "proxy" of show type (musical, straight play, one-person show, and "exception"). I couldn't make out from her study, however, how accurate these proxies might be - and I wondered, frankly, if the method she was using was truly valid given that she lacked this hard data.

I therefore felt I was looking at a study in which the final section relied on a possibly questionable sample and a problematic proxy for a central variable. My conversation with Sands, however, yielded little real insight into these issues - perhaps, as she often reminded me, because I'm not as trained in statistical analysis as she is (my "training" goes no further than the standard introductory college course, taken some twenty years ago).

So I'm throwing open an invitation to any reader with a sophisticated statistics background - would you care to more thoroughly fisk Ms. Sands's thesis? I can email you her study, which I've downloaded (contact me at hubreview@hotmail.com), or you can find it here. If my doubts about her methodology turn out to be ill-founded, of course, I'll happily publish those findings. (It's entirely possible, too, that if hard data on profitability were available, it would back up Ms. Sands.) In the meantime, I've emailed Ms. Sands my questions in the hopes that the explanation which eluded us in conversation can be found via printed text.

I'll be following up later this week, so stay tuned . . .

4 comments:

  1. You're making a big deal of this profitability thing, but she clearly states that she's using revenues as a proxy for profits, assuming that there are no inherent differences between the production costs of plays written by women and plays written by men. This is not an outrageous assumption, as variation within production costs between plays written by men and women would probably be a wash.

    You provide no evidence that production costs for plays written by men would be different from production costs for plays written by women.

    What say you?

    ReplyDelete
  2. Sands has no argument without profit data. Period. Her whole project is to show that discrimination exists in economic terms, and you can't do that without profit data. The production costs aren't available, so I don't have them, Emily doesn't have them, nobody has them. Therefore the profit data is unavailable, and without that, you can't do the analysis Emily is pretending to do. It's too bad, but that's all there is to it. I'm almost amused by the double consciousness of her supporters - on the one hand they say, "But she admits she has no data!", while on the other hand continuing to argue as if she had the data.

    Or, perhaps, you and her other supporters don't really understand the structure and requirements of this kind of analysis. To you, this is a bunch of technical gobbledygook that seems to align with your personal beliefs (a lot of pseudo-science operates like that). Indeed, I think that reaction was pretty much what Emily was counting on.

    ReplyDelete
  3. I hear what you're saying, but you didn't exactly answer my question. 1) What's so crazy about using revenues as a proxy for profits? And 2) what's so crazy about assuming production costs are about equal?

    ReplyDelete
  4. Look, Molly, you need certain data to make the math work - marginal analysis (which is what Sands is doing) simply requires data about marginal costs and profits, which Sands doesn't have. What she's doing is like trying to measure the footprint of an object, when the only data she's got is its height; maybe the two measurements are related - but maybe they're not. Or think of it this way: producers open and close shows because of profits. A show can have very high revenues, but if its costs are even higher, then it will close regardless, because it's not profitable. Therefore if you're trying to determine that a show closed because of bigotry (which is what Sands is trying to prove), you have to show that it was profitable, but closed anyway. To get that data about profitability, you have to have accurate measurements of production cost AND revenue. And you can't assume production costs are "about equal" for various genres, because, to be blunt, they're often not equal. Musicals are usually more expensive than straight plays - but then again, a musical like The Fantasticks has much lower costs than a play like The Coast of Utopia. If the effect Emily came up with her rigged data was a huge one, her case might be believable. But the effect she came up with was rather small - and the gaps in the data were huge. Her paper is obviously a con job. Obviously.

    ReplyDelete