The first CPN pilot is finished!
/Almost 100 end-users tested the CPN news recommender platform during the first pilot phase. Here are the main takeaways from the pilot, information about the tested prototype, as well as details about the next steps.
Background
The news recommender that the CPN project is developing will be iteratively tested and validated in operational real-life environments throughout the project in the three different pilot countries. The pilot countries are Belgium (coordinated by VRT), Germany (coordinated by Deutsche Welle) and Cyprus (coordinated by DIAS). Three main pilot tests will be organised at different stages, each adding a larger number of users as the developed proof of concept becomes more mature. We just finished the first pilot – read on to learn about the results!
What did we do in the first pilot?
In October and November 2018, we engaged almost 100 end-users to test our first prototype for a period of 10 days. Specifically, the participating end-users tested the web interface of the recommender. In each pilot country, the recommender contained news content from the local media partner (VRT, Deutsche Welle and DIAS).
The first CPN prototype consisted of three news sections, presented in separate tabs:
Most popular – the most-read content among the users of the particular news outlet
Latest news – all articles in chronological order, with the latest on top
Personalised – the most relevant content for the user
All end-users were free to use the recommender and read the three tabs as much as they wanted during the test period.
Screenshot of the recommender’s web interface in Belgium
During the pilot, we tested the personalisation algorithm and evaluated the technical components. Furthermore, in a qualitative research track, we evaluated the user experience of the web application, and assessed the users’ expectations with regard to news personalisation and FOMO (Fear Of Missing Out). 56 end-users tested the prototype in Belgium, 21 in Cyprus and 20 in Germany. Feedback was gathered during and after the pilot test by means of online questionnaires, focus groups, interviews and ad hoc interactions with the research participants.
Setting focus group interview in Belgium
What are the main takeaways of the first pilot?
The “Personalised” tab was the most read stream in all three pilot countries. But while news personalisation itself was evaluated as positive, there was a big fear of missing out (FOMO) when receiving personalised news articles: the participants were afraid that through personalisation they might only get news content based on their interests, which could cause them to miss other relevant or important news. CPN tries to tackle this in two ways:
By providing a ‘Most Popular’ and ‘Latest News’ tabs in combination with the ‘Personalised’ tab, thus giving end-users control over which tab they read first.
By giving end-users a transparent insight and control over how their personalised news offer is created.
Another suggested way to deal with FOMO is to indicate an important or breaking news item in the personalised tab, e.g. by means of a notification or a bar at the bottom of the page. This option will be further explored and integrated in the second pilot.
The fact that the recommender is not available as a mobile application was a negative point among the testers. (The pilot phase 2 of CPN will specifically test a mobile application.) On the other hand, the participants generally thought the web interface was straightforward and easy to use, although some thought that more important articles could be made more prominent.
What’s next?
In 2019, two more pilots will be set up. Currently, we are working on a mobile application of the recommender that will be tested in pilot 2. While pilot 1 was a controlled (small) testing phase, for pilot 2 we will move to semi-controlled testing, and selected external participants will be able to try the platform at this point. Finally, pilot 3 will be fully open, so anyone interested in CPN can participate in the last pilot phase.
Do you want be among the first to test the platform? Fill in this form and we’ll contact you when pilot 2 will start!
Read the full report on the first CPN pilot here.
