Streetlights & Counting Clicks

Someone recently sent me this 2014 Time Magazine article on metrics for web readership by Tony Haile, the CEO of Chartbeat, which does web metrics.

My first impression: this article seemed oddly familiar and I realized I read a similar article about Chartbeat by Farzad Manjoo in 2013 (with some of the exact same/similar graphs; note that this is not so much of a critique as just amusing to me):

Different articles, same graph. Bonus: see what's on Seth's bookmark bar

Technically different but essentially the same graphs. 


Go read either or both of these articles and come back here for my thoughts.

My key impressions:

-Of course neither clicks nor social media shares are a completely accurate proxy for what we really care about: engaged readers who understand, learn, and remember the content of an article

-but clicks and social media shares are (most likely) useful proxies.

Ultimately this is my more optimistic view of the "streetlight effect" -- sometimes it makes sense to look for your keys in the the light, where it's easy. See also: fruit, low-hanging.

Compare:

Newspaper circulation: of course 1.4 million NY Times readers are not fully apprised of every topic discussed in each daily paper.

Symplur impressions: #ACEP14 had 33 million impressions; this does not mean that, on average, each emergency physician learned 1,000 things via Twitter during ACEP.

My take is that in all of these cases, of course there are limitations but these numbers are a "ceiling" -- an upper limit of theoretical impact, and they are still somewhat useful to compare numbers (e.g. one outlet's performance across time, or multiple outlets' performance against each other).

The rub is how accurate the surrogate marker is as a proxy. Compare impact factor, where it's probably useless to compare a clinical EM journal against an academic mathematics journal, where citations accrue at different rates and may signify more or less (for more on limitations of journal impact factor, see our piece on Altmetric).

Complete speculation: I suspect that clicks are a useful measure for Annals of EM, particularly as we have what seem to be fairly high visit durations (nearly 3 minutes) and 2.6 pageviews per visit (if I'm reading the data I have correctly).

I have not analyzed this formally in any way, but casually following Annals' top Altmetric score articles seems to give a handful of popular topics, in no particular order:

1) airway (always a popular topic, particularly on social media)

2) social media (tons of self promotion and mutual congratulations, as well as legitimate interest in social media)

3) public health (e.g. ACA, injury and violence, even Ebola)

I suspect the social-media-share-to-actual-reading ratio rises going down this list, particularly as there are a lot of overlapping online communities and general public interest in public health topics.

One last point about the Time article:

2/3 of reading is below the fold, which makes sense: the overwhelming majority of clicks result in 15 seconds or less of reading, but the small fraction who read below the fold spend much, much more time, just as half of Medicare spending is on 5% of beneficiaries.

Crowdsourcing: TVP

In an effort to demonstrate* the crowdsourcing potential of Twitter for EM medical education resources, I asked:

The crowd performed well and very quickly I got a number of useful responses:




Thanks everybody!

*to a skeptical resident

ACA Update: What do you want?

On June 13, I'm speaking at the Policy Prescriptions Health Policy Symposium at Baylor.

My talk is titled "The ACA: An Evidence-Based Update." To continue my new tradition of crowd-sourcing, what would you want included? What do you know that I should share? What don't you know that you want to?