Streetlights & Counting Clicks

Someone recently sent me this 2014 Time Magazine article on metrics for web readership by Tony Haile, the CEO of Chartbeat, which does web metrics.

My first impression: this article seemed oddly familiar and I realized I read a similar article about Chartbeat by Farzad Manjoo in 2013 (with some of the exact same/similar graphs; note that this is not so much of a critique as just amusing to me):

Different articles, same graph. Bonus: see what's on Seth's bookmark bar

Technically different but essentially the same graphs. 


Go read either or both of these articles and come back here for my thoughts.

My key impressions:

-Of course neither clicks nor social media shares are a completely accurate proxy for what we really care about: engaged readers who understand, learn, and remember the content of an article

-but clicks and social media shares are (most likely) useful proxies.

Ultimately this is my more optimistic view of the "streetlight effect" -- sometimes it makes sense to look for your keys in the the light, where it's easy. See also: fruit, low-hanging.

Compare:

Newspaper circulation: of course 1.4 million NY Times readers are not fully apprised of every topic discussed in each daily paper.

Symplur impressions: #ACEP14 had 33 million impressions; this does not mean that, on average, each emergency physician learned 1,000 things via Twitter during ACEP.

My take is that in all of these cases, of course there are limitations but these numbers are a "ceiling" -- an upper limit of theoretical impact, and they are still somewhat useful to compare numbers (e.g. one outlet's performance across time, or multiple outlets' performance against each other).

The rub is how accurate the surrogate marker is as a proxy. Compare impact factor, where it's probably useless to compare a clinical EM journal against an academic mathematics journal, where citations accrue at different rates and may signify more or less (for more on limitations of journal impact factor, see our piece on Altmetric).

Complete speculation: I suspect that clicks are a useful measure for Annals of EM, particularly as we have what seem to be fairly high visit durations (nearly 3 minutes) and 2.6 pageviews per visit (if I'm reading the data I have correctly).

I have not analyzed this formally in any way, but casually following Annals' top Altmetric score articles seems to give a handful of popular topics, in no particular order:

1) airway (always a popular topic, particularly on social media)

2) social media (tons of self promotion and mutual congratulations, as well as legitimate interest in social media)

3) public health (e.g. ACA, injury and violence, even Ebola)

I suspect the social-media-share-to-actual-reading ratio rises going down this list, particularly as there are a lot of overlapping online communities and general public interest in public health topics.

One last point about the Time article:

2/3 of reading is below the fold, which makes sense: the overwhelming majority of clicks result in 15 seconds or less of reading, but the small fraction who read below the fold spend much, much more time, just as half of Medicare spending is on 5% of beneficiaries.

Sketchy EBM – Great Evidence Based Medicine Site

sketchy header

This really slick new site came onto my radar when listening to my friend, Ken Milne’s (@TheSGEM) podcast ‘The Skeptics Guide to Emergency Medicine. Ken’s podcast has surreptitiously taught me a great deal in how to interrogate research, as well as some great clinical topics. Having listened to a number of episodes in which Dr Anthony Crocco was a Guest Skeptic, I was thrilled to here that Antony had launched his own website focusing on Evidence Based Medicine and research appraisal.

Sketchy

 

Sketchy EBM has a growing gallery of videos that will undoubtedly enhance your thought processes around critically appraising medical research publications. As well as the EBM videos, Anthony’s (in)famous Ranthony videos are hosted – these are soapbox style rants in which Anthony tackles dogma and bugbears in Paediatric Medicine.

By way of introduction, I would strongly recommend spending the four minutes viewing ‘How I Read a Paper’. Jump in and get Sketchy and as the tagline says:

Always draw your own Conclusions