Tuesday, June 24, 2014

Media Evaluation Techniques (Part 2)

In Part 1 of this discussion on media evaluation techniques there was a look at the processes involved in collecting the cuttings, creating a media list and using packages like MS Excel and MS Access to collate and sample the media coverage. 

Although this 2-parter is an exploration of media evaluation techniques, deciding on a strategy is no less important. To be clear on this it is necessary to set clear campaign or period PR objectives. These objectives need to be measurable; thereby enabling the evaluator to hang the relevant techniques off of these, and thereby addressing these goals.

One of the points to emphasise is the non-prescriptive nature of this advice. As is often said in media evaluation circles, there is no one-size-fits-all solution. To paraphrase Philip Sheldrake 'Every organisation is unique, generating unique exposure, demanding a unique set of evaluation metrics'.

Once the media list has been collated and the chosen fields queried the most interesting stage to an evaluation project can start. My chosen query tool is an Excel Pivot Table, in my experience one of the most flexible and little used tools in the entire MS Office suite.


After extracting a listing featuring all the needed fields from MS Access, (see Part 1) paste into a blank Excel sheet. Add an extra column at the far left titled 'Count' and fill it with '1's down to the last entry. This will be used to calculate volume figures. 



Then from the Insert tab on the top line select Pivot Table which will then create a new tab. 



The list of fields on the right should match the column on the your listing. The first thing to do is select a metric with which to generate results, which if you want to use volume is what the 'Count' column is for. Drag it to that box unless you want to use another metric like cumulative audience. Be aware that if selecting another metric make sure it says 'Sum of....'. If it says 'Count of...' click the drop down to the right of it and select Value Field Settings and select Sum.

Pivot tables are a very flexible tool and its worth spending time dragging the fields around between the boxes and seeing what the impact is on the grid on the left. It is quite hard to break a pivot table. When you start dragging values around and doing cross-references, it is tempting to add a column or alter the source listing in some way. That is fine but delete the pivot table tab and insert a new one as it won't recognise the changes made to the listing. 



The sort of useful results which can be generated using this tool is multiple cross-referencing of fields. For example, finding out how many proactive clips there are for different subject or product areas or tracked competitors. As long as it is on the listing you should be able to query it.

The next stage is to parse these results into some form of report or presentation. Though this might make good content for a later post it is likely you have a very particular application in mind. 

I hope you have found this of interest and I would welcome any comments or thoughts you might wish to share.  

Monday, June 23, 2014

Media Evaluation Techniques (Part 1)

Although everyone has a different way of doing an evaluation of a project or periods media exposure, there are certain common processes worth highlighting. From a strategic perspective the starting point has to be the objectives. on the other hand, if you are more interested in evaluation techniques, getting the media clips seem like the logical beginning point. 

While there is much to be said about PR and campaign objectives at the strategic level, this mini-guide concentrates on the execution techniques. More often than not these media clips are electronically circulated press and internet cuttings from the various specialist monitoring outfits like Gorkana or Kantar. Personally I can't speak for the latter in any detail but for the former there is also the option to create an Excel media list, which we'll talk about in a bit.

Having worked with these sources for a number of years it is my experience of the UK media that most of the coverage found tends to be either printed or online coverage. It may be a symptom of my clients interests or the keywords checked but there is seldom coverage from broadcast sources.



These Excel media lists can be filtered to only the coverage for a set period; helpful if doing monthly evaluation reports. Often these listings will also have an indication of circulation or audiences (and the dreaded AVE figures!). Unless my own  master media listing file is missing a title I tend to ignore these figures. Once this Excel listing has been formated I Paste Append into the clients MS Access file.



This is when to find out how well the listing has been formatted. All the Excel columns have to match up with the Access file. Also this will draw on a separate master media data file, and so the media titles have to match. This is often a problem as my master media titles do not start with 'The', instead leaving it at the end. For example 'Daily Telegraph, The' is our chosen format but the cuttings agencies don't always do it that way.   Once the data has been added I run a query to highlight those media titles it can't find.

After 'cleaning' the entries the next stage is to work through the cuttings, sampling them for the relevant tracked fields. It is always a good idea to sample for as many fields as you might think could be relevant. This often goes beyond the brief, but you never know what might be of interest or where coverage might take you. 

Core requirements tend to be things like favourability towards the brand and presence of key messages. Then there are additional things like the tracking of subject or product/service areas, corporate/brand descriptors, pro-activity, scale of cutting. Below is an example of a form with fields (excluding messages).



That is the first part of a two-parter. In the next section we will take a look at some techniques on how to interrogate the data, filtering, sorting and sampling. Then there is report writing, mixing graphs and tables with narrative and choosing formats, all to address the original objectives.

If you have thoughts on this or additions it would be great to hear from you.

Wednesday, June 11, 2014

'Just try to become less crap at it'

This was a declaration by Rory Sutherland, leader of Ogilvy Change on their website when talking about marketing. Somewhat self-deprecating, but a useful starting point from a great intellect and very relevant to media measurement.

He makes the point that he is opposed to the concept of surveys and focus groups. The areas of the brain that fire buyer behaviour do not respond well to this type of introspection, often residing at a sub-concious level. I have often wondered about the value of this style of research, seeing it as a way of gauging peoples views on questioning process rather than the actual questions; the Hawthorne Effect, and all that....

So could the media be the touch-point on people forecasting? Yes, but it needs to be one of a selection, a host of pointers based on unique circumstances, fashioned and adapted to reflect the changing circumstances (competitors, technology, expectations, etc). A gauge on the media has its advantages. Whilst it often seems a contrived situation, it is about free expression and combined with social media selective amplification, provides a good measure of weight and velocity.

Friday, January 24, 2014

Thoughts on another media year

There are a number of fast forming trends shaping and reshaping the media. In a few short generation the influence of the printed has diminished dramatically with much of the audience seeking their content online, sometimes behind paywalls. Any well-informed clues on  where these trends are going is gold dust and the CIPR in this video-cast featured David Pemsel, Chief Commercial Officer, The Guardian; a paper advocating free access, for the time being anyway.

He highlighted that newspapers can be more flexible online, using a more diverse a selection of media  sources and formats. The Guardian take the view that they need to have a scale of usage to gather the influence necessary to be a news 'player', and that level of engagement presently only come from the site being without a paywall.

The discussion make a point about news ownership. The Guardian believes that their news content is most trustworthy as a result of their being owned by a Trust...an aspect they believe contrasts with other providers which may be subject to commercial or political pressures.There's also a perspective on newspaper opinion and if they are being dragged into a competition with bloggers for influence. The idea of the media 'melting pot' seemed central with good ideas and perspectives rising to the top.

Monday, September 09, 2013

What is the value of an online newspaper comment?

Much has been said about the value of a Facebook Like and so it seems only logical to think about the value of other online  third-party endorsements and claims. What about comments on online newspapers? Just about every site that could be described as an online newspaper has a mechanism for allowing readers to comment, however in the many years I have been evaluating online content (mostly manually) I have not included or ever been asked to include consideration of the online comments. I would be very interested in any readers experiences?

While it might be suggested that online comments have moved up the agenda following Arianna Huffington's decision to stop all anonymous comments on the Huffington Post, it was actually an article in the Telegraph this weekend that got me thinking. There a few more emotive subjects in the UK than housing and this article waves a red flag with the headline 'House prices spiral up in 'virtuous circle''. In the last 3 days there has been 466 comments, taking for main part, a counter view. 

If I were evaluating just this article (not the comments) I think you would have to ascribe it as positive towards the housing market and the efforts of the Government to support it. If on the other hand you were evaluating just the comments you could seriously only take a contrarian view. Just take a look - see what I mean?


Comments seem to be becoming a bigger part of the online newspaper experience. Today's Guardian home page has on average received 232 individual comments per news article. 


A search of Google on the value of comments has so far been unproductive - if you have anything please get in contact. I strongly believe this area needs more research.


As mentioned, with this volume of feedback there is a need to find ways of analysing and integrating this material. 


There are a couple of options:



  1. Skim-off the top comments for separate or composite analysis. Most online newspapers allow you to sort the comments in order of the time, popularity, recommendation, etc.  
  2. Develop a comparative index to integrate the impact of all the comments across the online newspapers.
But there are also a few hurdles. If you relay on electronic cuttings from a cuttings agency the likelihood is they will not include the comments. Similarly if you use a social media monitoring tool the chances are it will not capture or analyse comments - I just tried to use one to find the comments on this Telegraph cuttings, with no success.

So if these tools won't help you find the stuff and there is little research to suggest it matters, maybe we should carry on ignoring it? I am thinking to the contrary. In our social/connected world there is valuable intelligence and relationships going to waste.

Tuesday, July 30, 2013

Is it ever okay to use AVEs? - Thinking the unthinkable

I was reading this article on the BBC website about the value to sponsors of the Olympics and it led me to think a little about that horrid PR conundrum. Are there ever circumstances when the use of AVE is acceptable?

AVEs have loads of reasons for their non-use. I really don't want to rehash that list, but they are irrefutable and compelling. They are also the default measure for lazy PR folk. They are just complicated enough to allow the lazy types to explain the methodology to the layman and make them look clever, but not so complicated as to loose those same people.

However the world changed and they are now a bit like Babycham, deeply unfashionable, widely regarded as horrid. The transparency around AVEs methodology exposes their flaws making them easy to reject; end of discussion....

But I keep getting this pain...it nags away at me demanding satisfaction. Its a question...why won't AVEs just die?

If they won't go away are there circumstances then they might be considered as okay to use them? Arguably within the sponsorship industry they are still widely used as a measure of velocity (frequency & audience & scale). All sponsors coverage is positive (?) and progress is ink on page. I have major questions about applying same thoughts to the digital world. Also, (& this is where lazy PR can happen) if we say its okay to measure sponsorship coverage by default it is okay to measure other 'good news' stories - the creep begins!


Friday, May 24, 2013

Are you missing something?



Having been around media monitoring and analysis for quite a few years I sometimes wonder if a tipping point has been reached and you can achieve a picture of coverage by considering just the online sources.

However, is there a possibility you are missing something by taking the web exposure as a proxy of all exposure?

For the last couple of months I have been comparing the online and printed output for a couple of trade publications to establish how much is original to each medium. Although this is a limited sample the results seem quite startling.


In essence, if you only consider the online coverage you are missing almost half of the picture. Please note this is quite unscientific and I would hasten to add much dependant on the titles. With some, their online coverage almost mirrors the printed coverage, while others are apparently quite different. If I were to offer a broad observation on which types fit into these, the more frequently published titles (daily & weeklies) will often more closely resemble the online offering, while the less frequent (ie. monthly magazines) tend towards greater unique online and printed content.

That said, I am sure many of us have found it difficult to find a printed article in the online version of a daily press title.

Obviously press is in decline and online is the future, but we would appear to not quite be there yet. I would be very interested in if this matches with your experiences?