Since February our readers must pass a mandatory quiz to comment on our articles. But the world famous quiz has found another use too.
First, some navel–gazing and context.
NRKbeta is the Norwegian Broadcasting Corporation’s (NRK) tech-site and R&D-blog. This February we launched a new experiment.
Readers wanting to comment on a story had to pass a mandatory test about the articles content.
Despite other websites abandoning their comments section, ours has mainly remained a positive space the last ten years.
Comments from our audiences have solved difficult problems for us. It has gathered fantastic and crazy ideas from our audience that we have tried out. It’s even helped us hire talented people! In short: a valuable arena we dedicate a lot of time and effort to maintain and improve.
But as some of our most read stories have ended up with discussions below average, we suspect some readers just skimmed the headline before heading for the comments section. That is not what it is for.
That is how the idea of a speed bump in the form of a quiz originated.
We hoped this would contribute to eager commentators cooling down. It could give them the opportunity to think things through before punching away at the keyboard.
Around the world
A short while after we went live, Harvard’s Nieman Lab picked up on it and wrote about it. Joseph Lichterman’s story became one of their most read and started a tsunami of international attention.
The experiment recieved praise from prominent personalities in the media industry world wide. Soon CNN, France24, BBC and Gizmodo picked up the story as well. Then we received a ton of interview requests from radio, TV, newspapers and news sites from all over the world.
This project from @NRKbeta and @NRKno is quite possibly one of the best new journalism ideas I've seen, ever.https://t.co/DCT3s9pDty
— Amy Webb (@amywebb) 1. mars 2017
Our small newsroom took turns answering questions about how horrible our comments section must have had been (no, it really was not), if the quality has improved (almost impossible to say) or if the experiment had been a success (we do not know, it has just been live a few days).
We kept note to about 300 articles were published about it, and can still feel the shockwaves.
Today in Great Ideas Of Our Age, this Norwegian news site stops readers commenting unless they read the story https://t.co/lOZYkDtvbI
— Dave Lee (@DaveLeeBBC) 2. mars 2017
Because creating MORE friction in engaging increasingly time-poor & promiscuous digital audiences, who can go anywhere else, is very good https://t.co/tqKXA3RSFi
— Raju Narisetti (@raju) 1. mars 2017
Just a few days ago, a link to the Indepentent’s story from early March received over 86 000 upvotes on reddit. A tweet from Wall Street Journal’s tech columnist received over 5 000 retweets and close to 8 0000 likes. Not bad on Twitter.
TIL if you want to comment on the Norwegian news site, NRKbeta you must first take a quiz to test your basic understanding of the article. This is done to prevent ranting and foster positive conversations from todayilearned
The feedback from colleagues in the media industry and most of our readers have – as mentioned – been very positive. But this is just the initial reaction to the concept and us trying it out. Not really to to the results. Results we did not have when the questions started pouring in.
The combination of a media industry desperation for new innovation, poor comments sections and an apparent easy and smart solution proved to constitute a potent viral mix.
So let’s try to summarize what we have learned so far, almost six months after we went live.
In our own comments section the quiz has of course been diligently discussed. The majority has been positive, but at least one of our readers thinks the quiz is a bad idea. Details like if a conference is called F8 or Eight has nothing to do with «if you are qualified to comment what the article actually is is about«, the reader writes.
Another suggests an improvement and adds «Kind regards one writing in a comments section for the first time in ten years with new optimism!«.
We have taken a closer look at the numbers, but there isn’t much logic or clear answers to report. At least not if assume that everyone taking the quiz actually wanted to leave a comment.
On average, there is a lot more attempts – both correct and wrong – than actual comments.
It seems many take the quiz to check how much they remember from the story – and not necessarily to leave a comment. Almost as a fun little game after reading.
Clik here to view.
A story that stand out is our explainer on how to like Facebook statuses with a rainbow in connection to pride. There were over a thousand wrong attempts to answer the quiz. Due to a human error, the right answer to one of the questions was not indicated. It made it impossible to pass the quiz. Hence the many logged wrong attempts.
On average, there is a staggering error rate of 72% on the quiz. We also suspect a lot of wrong answers coming from visitors of faraway lands. Most would have a hard time breaking our encryption made of solid Norwegian language.
The recent wave of visitors from the Independent’s story via Reddit, were met with our most recent article. We did not expect it would have traffic above normal, and did not see the need to enable the quiz.
Several left comments asking about the quiz. Thus we assume most international visitors have not understood the quiz. Likely they wanted to play with the functionality (and lucky for you we open sourced it).
We have not published a large amount of stories with a quiz, making the sample small. It’s one of the reasons why we have not established any set rules for when the quiz should be enabled. The idea was to test it on stories that had potential for a gloomy comments section. It is something we are proud to rarely have here at NRKbeta.
Colleague Anders highlights the fact that a quiz raises the bar to leave small comments. Popping in to say «fun piece» or «hey spitfire, chill out» before leaving the article is now a bit harder.
This favors the most eager with the most time on their hands. From time to time this has lead to a decline in quality and tone, causing him to often abandon the quiz module. If you have tried the quiz module, we would like to hear your experience. Leave a comment below 😉
In other words, it is hard for us to present evidence concluding with success or failure. But the numbers seem to show that the quiz has worked like a little game for many readers. They like to take the quiz, but not to leave a comment.
Being tested on how much they remember from the article seem to be the most popular use of our quiz.
Clik here to view.
Clik here to view.
Clik here to view.
Clik here to view.
Clik here to view.