{"id":1495,"date":"2008-12-26T11:03:18","date_gmt":"2008-12-26T15:03:18","guid":{"rendered":"http:\/\/shermandorn.com\/wordpress\/?p=1495"},"modified":"2008-12-26T11:03:18","modified_gmt":"2008-12-26T15:03:18","slug":"effect-size-matters","status":"publish","type":"post","link":"https:\/\/shermandorn.com\/?p=1495","title":{"rendered":"(Effect) size matters"},"content":{"rendered":"

Nathan Yau is not Edward Tufte. Yau is a doctoral student in statistics. Tufte<\/a> is a Yale professor emeritus. Yau’s list of his 5 best data visualization projects of 2008<\/a> has a common missing element (from four of the listed projects) that E.T. would pull tufts of hair out over: the images have no quantification. To Tufte, that is a cardinal sin, along with the “chartjunk” that infects so many graphs in USA Today<\/i> and other newspapers.<\/p>\n

I am generally on the side of Tufte on this issue: unless you’re a topologist, quantity matters and units matter. A common fallacy in manuscripts (and sometimes published articles and books) is the confusion between statistical significance and practical meaning. But if you are working with a sample size of 50,000 or more (common with a large epidemiological study or census microdata extracts), it is hard for many relationships not<\/b> to be statistically significant. But whether the relationship is meaningful depends on the size of the relationship. <\/p>\n

And here, the units matter! If you know that the multiple-regression coefficient between income and achievement is 1.5, that may or may not be notable. If you’re measuring income in thousands of dollars and achievement in scale score points when the range is 0-1000 and the standard deviation is 150, that’s a meaningless relationship (going up 15 points, or 0.10 of a standard deviation, when the income increases by $100,000). If you’re measuring income by natural log and achievement in standard-deviation units, that’s a substantial relationship (essentially moving a standard deviation up or down when the income doubles or is halved). <\/p>\n

In part stemming from the literature on meta-analysis, it is becoming more common for individual studies to identify effect sizes. While I still want to have a sense of concrete relationships, pushing authors to look at quantitifed relationships in perspective is always good. The same should be true for “data visualization.” Quantify, folks! <\/p>\n

(For the record, I don’t think Tufte is infallible. Far from it<\/a>.) <\/p>\n","protected":false},"excerpt":{"rendered":"

Nathan Yau is not Edward Tufte. Yau is a doctoral student in statistics. Tufte is a Yale professor emeritus. Yau’s list of his 5 best data visualization projects of 2008 has a common missing element (from four of the listed projects) that E.T. would pull tufts of hair out over: the images have no quantification. […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[4,18],"tags":[],"class_list":["post-1495","post","type-post","status-publish","format-standard","hentry","category-research","category-the-academic-life"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/pag0MB-o7","_links":{"self":[{"href":"https:\/\/shermandorn.com\/index.php?rest_route=\/wp\/v2\/posts\/1495","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/shermandorn.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/shermandorn.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/shermandorn.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/shermandorn.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1495"}],"version-history":[{"count":0,"href":"https:\/\/shermandorn.com\/index.php?rest_route=\/wp\/v2\/posts\/1495\/revisions"}],"wp:attachment":[{"href":"https:\/\/shermandorn.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1495"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/shermandorn.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1495"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/shermandorn.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1495"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}