Our evaluation plans allow you to evaluate sessions by type, track, or any other criterium, which you can learn about it in the following article: Evaluations: From start to finish. However, certain event organizers, such as those from the academic community, sometimes want to have specific evaluators focus on particular sessions, such as those that directly fall within their area of expertise. Another reason to limit the number of sessions per evaluator could be to spare them from a huge amount of work or to distribute the workload between multiple evaluators, which can be particularly useful for events with many submissions.
For such situations, we developed the assigned evaluators feature. Here's how to use it.
Set your evaluation plan to allow assigned evaluators
When creating a new evaluation plan or editing an existing one (assuming it's open), check the Only assigned evaluators option to enable a manual selection of evaluators for every single submitted session. This can only be done when using the Stars rating and Yes/No evaluation modes. Due to the nature of the algorithm used, the assigned evaluators feature isn't available for the Comparison evaluation mode.
To find out about the differences in our evaluation modes and get help deciding on the best one to use, please refer to the following article: Evaluation modes explained.
Assign evaluators for each session manually
To manually assign evaluators for each submitted session, go to your event's Sessions page and select the Assigned evaluators option from either of the two dropdown menus.
Then, simply click on the pencil icon and select the evaluator(s) you want to assign to each nominated session. Your evaluators will only get to evaluate the sessions you assigned them to; they won't have access to other sessions when doing evaluations.
Don't see a certain team member in the list of available evaluators? Visit your event's Team page and check if the team roles have been set properly. To learn about the differences in team roles and ways of editing them, please read the following article: Team roles explained.
Assign evaluators in bulk
To assign evaluators in bulk, go to the Sessions page and click the checkboxes next to the sessions to which you want to assign evaluators. Then click the Assign evaluators button at the bottom of the page.
A window will pop up, containing all relevant options to quickly assign evaluators to selected sessions. After you select the evaluators you want to assign sessions to and choose the desired number of evaluators per session, our system will randomly assign the selected number of evaluators to all sessions in scope.
Once you're happy with your selection, click the green Assign X evaluators to Y sessions. Wait for a short countdown, and finish by clicking Apply changes.
Assigning evaluators with filters
A more advanced way of assigning evaluators is to use filters. You can filter the sessions by their status, an existing evaluation plan, or any other criteria defined through the custom fields that you added on the Edit event page.
Here's a real-world example. Let's say you have a couple of evaluators who are German native speakers, and you want to assign them to all nominated sessions that are in German.
- Filter the sessions by selecting Nominated in the Status field and German in the Language field
- Click the refresh button to apply the selected filters
- Select the sessions you want to assign evaluators to and click the Assign evaluators button
- Select your German-speaking evaluators
- Select how many of them you want to assign per each session
- Click the green Assign X evaluators to Y sessions button
- Confirm by clicking Apply changes
The same principle applies to all other such situations. You can use the identical method to assign evaluators to particular session formats or levels of expertise, specific tracks, or anything else you find necessary.
View statistics
Keeping track of the number of sessions assigned to particular evaluators could become difficult, especially when using filters, which is why we added a quick statistical overview of the exact number of sessions assigned to each evaluator. Access it by clicking the View statistics button, found next to the Assigned evaluators field.
The stats show both the total number of assigned sessions per evaluator, as well as the number of assigned sessions with the currently active filter. You can observe the active filter below the statistics.
Bonus tips
Always filter the nominated sessions
As evaluations are supposed to be done exclusively on nominated sessions, it is a good idea always to filter the sessions by their nominated status. That way, you're effectively removing the chance to accidentally assign evaluators to sessions that have already been accepted or declined.
Quickly remove assigned evaluators
To quickly remove one or more evaluators assigned to a particular session, simply click on their names and remove them from the Assigned evaluators list on the Sessions page.
Clear all assigned evaluators
If you, for whatever reason, want to clear all of your evaluators from specific (or all) sessions, select the appropriate sessions, click the blue Assign evaluators button, and then check the Clear assigned evaluators option. Click the green Clear assigned evaluators from X sessions button, wait for the three-second countdown, and confirm by clicking Apply changes. Bear in mind this cannot be undone.