Our evaluation plans allow you to evaluate sessions by type, track, or any other criterium, which you can learn about it in the following article: Evaluations: From start to finish. However, certain event organizers, such as those from the academic community, sometimes want to have specific evaluators focus on particular sessions, such as those that directly fall within their area of expertise. Another reason to limit the number of sessions per evaluator could be to spare them from a huge amount of work or to distribute the workload between multiple evaluators, which can be particularly useful for events with many submissions.
For such situations, we developed the assigned evaluators feature. Here's how to use it.
Set your evaluation plan to allow assigned evaluators
When creating a new evaluation plan or editing an existing one (assuming it's open), check the Only assigned evaluators option to enable a manual selection of evaluators for every single submitted session. This can only be done when using the Stars rating and Yes/No evaluation modes. Due to the nature of the algorithm used, the assigned evaluators feature isn't available for the Comparison evaluation mode.
To find out about the differences in our evaluation modes and get help deciding on the best one to use, please refer to the following article: Evaluation modes explained.
Assign evaluators for each session manually
To manually assign evaluators for each submitted session, go to your event's Sessions page and select the Assigned evaluators option from either of the two dropdown menus.
Then, simply click on the pencil icon and select the evaluator(s) you want to assign to each nominated session. Your evaluators will only get to evaluate the sessions you assigned them to; they won't have access to other sessions when doing evaluations.
Don't see a certain team member in the list of available evaluators? Visit your event's Team page and check if the team roles have been set properly. To learn about the differences in team roles and ways of editing them, please read the following article: Team roles explained.
Assign evaluators in bulk
A faster way to assign evaluators to sessions is to use our wizard. The wizard has several nifty features that give you a lot of freedom to quickly and accurately assign evaluators to desired sessions.
Assigning evaluators randomly
The most basic way of using the wizard is to simply select the evaluators you want to assign sessions to and choose the desired number of evaluators per session. Our system will then randomly assign the selected number of evaluators to all sessions in scope.
Assigning evaluators with filters
A more advanced method of using the wizard is to combine it with filters. You can filter the sessions by their status, an existing evaluation plan, or any other criteria defined through the custom fields which you added on the Edit event page.
Here's a real-world example. Let's say you have a couple of evaluators that are German native speakers, and you want to assign them to all nominated sessions that are in German.
- Filter the sessions by selecting Nominated in the Status field and German in the Language field
- Click the refresh button to apply the selected filters
- Open the wizard and select your German-speaking evaluators
- Select how many of them you want to assign per each session
- Click the green Assign evaluators to X sessions button to confirm
The same principle applies to all other such situations. You can use the identical method to assign evaluators to particular session formats or levels of expertise, specific tracks, or anything else you find necessary.
Keeping track of the number of sessions assigned to particular evaluators could become difficult, especially when using filters, which is why we added a quick statistical overview of the exact number of sessions assigned to each evaluator. Access it by clicking the View statistics button, found next to the Assigned evaluators field.
The stats show both the total number of assigned sessions per evaluator, as well as the number of assigned sessions with the currently active filter. You can observe the active filter below the statistics.
Always filter the nominated sessions
As evaluations are supposed to be done exclusively on nominated sessions, it is a good idea always to filter the sessions by their nominated status. That way, you're effectively removing the chance to accidentally assign evaluators to sessions that have already been accepted or declined.
Quickly remove assigned evaluators
To quickly remove one or more evaluators assigned to a particular session, simply click on their names and remove them from the Assigned evaluators list on the Sessions page.
Clear all assigned evaluators
If you, for whatever reason, want to reassign all of your evaluators, open the wizard and click the Clear all assigned evaluators link at the bottom. Confirm by clicking OK. Bear in mind this cannot be undone.