Media evaluator can be helpful for video spot tests – ads, TV spots, radio spots, tutorials, or any kind of video content you might have. You can also test audio files.

The media evaluator tool shows a video or an audio spot to your respondents and collects their feedback in real-time. They use rating buttons to evaluate any moments in the spot on a scale of 1-7. They can submit the text comments, too.

Video evaluator

The feedback is recorded in 1-second intervals and the respondent can see his rating on the horizontal bar at the bottom of the spot. The respondent only sees his ratings; not the ratings of other participants. Additionally, it’s possible for the respondent to pause the spot or to go back and change any of his evaluations.

The moderator can see all results in real-time at the bottom of the whiteboard:

Video evaluator - results

Set up the stimuli

Before your session add a new video or audio file and turn the rating feature on:

Go to the whiteboard settings (the Whiteboard tab in the secondary menu) and add a new file by clicking the ‘Add file or activity’ button.

Name the new activity so you can easily recognize it on the list, select the type ‘video MP4 (whiteboard)’ or ‘audio MP3’ depending on what media you wish to test. Choose a file from your computer to upload…

To activate the evaluation feature, switch the ‘Show media rating’ option to Yes.

Save the form and wait until the file is uploaded – it can take several minutes for large files.

Video evaluator settings

Start the media evaluator during the focus group

Like other Quallie whiteboard activities, the moderator only needs to select the evaluator activity in the dropdown menu from the control panel and click the arrow button to send it to respondents’ whiteboards.

Respondents can see the media evaluator on their whiteboards and can enlarge it by clicking the Maximize icon.

Focus group whiteboard - video evaluator

Results and outputs of the media evaluation tool

The results are instantly available to the moderator during the session. Just scroll down in the whiteboard panel and you will see result timelines below the spot. The detailed results can be found in the Export section > Video evaluation.

You can display the results of all media at once or select only the results of a specific media file.

Timelines show the evaluation on a visual timeline for all respondents:

Video evaluator - results

You can display the source data for each file (or for all tested files) – a table that shows all events from all respondents:

Video evaluator - data table

Text comments are also available in the export section and the list can be downloaded, too.

Frequently Asked Questions

Do participants (moderators and respondents) need any special software to install on their devices?

No. Provided they use the latest internet browsers, no additional software is required.

What media format should I use?

MP4 for video files and MP3 for audio files. Please use a good compression setting and reasonable video resolution to make the file size smaller.

What is the maximum file size?

500 MB at the moment. We plan to increase the size in the future.

Can I use the media evaluator outside the video focus group – in the bulletin board page or blog/diary?

Yes. You can use the whiteboard activities in other modules too – go to your bulletin board or blog page settings and select the requested whiteboard activity in the dropdown menu. The dropdown is located below the intro text editor. A new icon and activity name will appear below the introduction text to respondents. After they click it, the activity will be opened in a lightbox.

Interested or need more information?