ABS500 - Assign Reviewers to Submissions

This batch report does the initial assignment of reviewers to submissions.  The process selects reviewers from the reviewers assigned to the call submission type for which reviewer assignments are being made. 

Only submissions that have a defined minimum number of required reviewers will be selected for processing by this batch process.  If the organization wishes to use the ABS500 batch process, when call submission type(s) are defined, a “Default Minimum Number of Required Reviewers” should be defined, which defaults into each submission, where it can be changed. 

The process:

·            Identifies the maximum number of open assignments and total assignments a reviewer is allowed to have as defined for the call submission type

·            Selects all submissions belonging to the call submission type being processed by the batch process where the submission external status = ‘UNDER_REVIEW’ and the submission internal status = ‘NEED_REVIEWER_INVITE’ and the number of invited reviewers for the submission is less than the minimum number of required reviewers defined for the submission. 

The process then identifies the assignment mode based on Assignment Mode parameter.

When to Run ABS500

The ABS500 batch process can be run anytime there are submissions to which reviewers need to be assigned. Reviewers need to have first been assigned to the call submission type. The process provides an option to match reviewer’s area of expertise with the primary topic defined for each submission.  The process will first try to assign reviewers who have not been assigned to any submissions, and then it will select reviewers with lower number of review assignments. The process will not select any reviewers who have reached the maximum number of review assignments defined for the call submission type. 

Assigning Reviewers to Submissions Randomly

If submissions are to be assigned to reviewers randomly (assignment mode is ‘RANDOM’), then the process simply selects the first reviewer and assigns that reviewer to the first submission, following the rules described above, in the “General Reviewer Assignment Logic” section.  If the number of reviewers to be assigned to a submission has been defined as more than 1 (Abs500_Submission. MIN_NUMBER_OF_REQUIRED_REVIEWERS > 1), the process repeats the steps, selecting the second reviewer and so on until the context submission has been assigned the minimum number of required reviewers defined for the submission. The process continues until all submissions have been assigned  as many reviewers as required, or until all reviewers have the maximum number of open assignments or total assignments.

Assigning Reviewers Based on Reviewer Expertise and Submission Topic

If the assignment mode is ‘EXPERTISE_AND_TOPIC’, the process allows the user to define a “Minimum Percentage Match”, which identifies how high of a match must exist between a reviewer’s areas of expertise and topic codes defined for a submission.   The lower the percentage, the more likely it is that a reviewer will be able to be assigned.

A 100% match means that the reviewer has an area of expertise at the highest level of expertise for each topic assigned to a submission.  A 0% match means that the reviewer has no areas of expertise that match any topics assigned to the submission.   The formula that calculates the percentage match is detailed in the section entitled, “Calculating the Reviewer/Submission Expertise/Topic Match Percentage”.

First, reviewers are evaluated against the selection criteria detailed in the “General Reviewer Assignment Logic” section above.  Then, the “Minimum Percentage Match Required” and “Requires 100% Match to Primary Topic” parameters work together. 

·            If “Minimum Percentage Match” parameter is set to 0 and “Requires 100% Match to Primary Topic” parameter = ‘N’, the process will assign the reviewer with the highest percentage match between a reviewer’s areas of expertise and all of the topics defined for the submission.  If more than one reviewer has the highest percentage match, then the reviewer is assigned based on an alphabetic sort of last name, first name, Customer ID.

·            If “Minimum Percentage Match” parameter is set to 0 and “Requires 100% Match to Primary Topic” parameter = ‘Y’, the process will assign the reviewer that has a 100% match between the reviewer’s areas of expertise and the submission’s primary topic, and then has the highest percentage match between a reviewer’s areas of expertise and all of the topics defined for the submission. If more than one reviewer meets this condition, then the reviewer is assigned based on an alphabetic sort of last name, first name, Customer ID.

·            If “Minimum Percentage Match” parameter is > 0 and “Requires 100% Match to Primary Topic” parameter = ‘N’, the process will assign the reviewer with the highest percentage match between a reviewer’s areas of expertise and topics defined for the submission, as long as the percentage match is >= the value defined for the “Minimum Percentage Match” parameter.  If more than one reviewer has the highest percentage match, then the reviewer is assigned based on an alphabetic sort of last name, first name, Customer ID.

·            If “Minimum Percentage Match” parameter is > 0 and “Requires 100% Match to Primary Topic” parameter = ‘Y’, the process will assign the reviewer that has a 100% match between the reviewer’s areas of expertise and the submission’s primary topic, and then has the highest percentage match between a reviewer’s areas of expertise and all of the topics defined for the submission, as long as the percentage match is >= the value defined for “Minimum Percentage Match” parameter.

Calculating the Reviewer/Submission Expertise/Topic Match Percentage

The calculation of the percentage match between a submission and a reviewer is fairly straight forward.  For each match to a topic, the reviewer is assigned a percentage match value based on their level of expertise in the topic. 

The formula is: 100 divided by the maximum expertise level value times the expertise level value of the reviewer in the topic area.

If a reviewer does not have an expertise in a topic assigned to the submission, the reviewer gets a 0 for that topic.  The calculated percentage match for each topic is then summed and divided by the total number of topics assigned to the submission.  This is the reviewer’s percentage match.

For example, assume an organization supports three levels of expertise for reviewers (SOME_KNOWLEDGE (1), KNOWLEDGEABLE (2), INDUSTRY_EXPERT (3)), so the maximum expertise level value is 3.  Assume a submission with 5 topic codes, TOPIC1 through TOPIC5.  Assume a reviewer who has areas of expertise as follows:

There is a 73.34% match between the reviewer and the submission ((100 + 0 + 66.67 + 100 + 100) divided by 5).

If ABS500 cannot find a 100% match, it will assign the reviewer that has the highest percentage match while not exceeding the maximum number of reviewer assignments defined for the call submission type, assuming that the reviewer’s match percentage is greater than or equal to the value defined for the “Minimum Percentage Match Required” parameter.  If more than one reviewer has the highest percentage match, it assigns the reviewer who has the lowest number of assigned submissions.  If more than one reviewer qualifies, ABS500 picks the first reviewer in alphabetical order by last name, first name, reviewer master customer ID.

Reviewer Expertise Level

ABS500 tries to match reviewers with submissions based on topic(s) defined for the submission and areas of expertise defined for the reviewer. When selecting a reviewer based on an area of expertise, ABS500 is looking for reviewers with the highest level of expertise in a topic. 

To identify the expertise level that is the highest, users need to update App_Type.OPTION_1 with the expertise level code that represents the highest level of expertise, and App_Type.OPTION_2 with the numeric value for that expertise level.  The OPTION_1 column of ABS EXPERTISE_LEVEL App_Code records should be updated with the numeric value that represents the expertise level, with 1 identifying the lowest level of expertise and the highest number identifying the highest level of expertise.  For example:

Parameters

Logic Flow

See Also:

·            Overview: Abstracts and Submissions

·            ABS1000 - Submissions Requiring Staff Action by Call and Submission Type

·            ABS1001 - Submissions Where Reviewer Scores are Overdue

·            ABS1002 - Index of Submission by Primary Author

·            ABS1003 - Index of Submissions by Submission Title