Configure Grading App

After creating your app, you will be redirected to a page where you can adjust the settings and design your app.

Assessment study details

In this tab, you can change the name of your app and add a description. Don’t forget to click “Done” to save the changes.

Assessors

Assessors are people who you invite to participate in your photo grading study. You can invite as many assessors as you want. For a participant to have access to your study, you need to create an assessor ID for them first. To do that, go to the “Assessors” tab and type an ID for each of the assessors in the text bar.

Please note that if your dataset contains photos of the assessors and you don’t want them to rate themselves, their assessor ID should be the same as their subject ID in the dataset. Keep that in mind so you don’t accidentally create an assessor ID identical to a Subject ID from the dataset.

Note that every participant’s ID starts on a new line. Their ID cannot contain spaces; otherwise, the system won’t be able to count the number of assessors correctly. As you type, you can check the number of assessors that will be created under the text bar.

Click “Save” once you’re done. A unique link to the study will be generated for each of the assessors. Along with the link, each of them will be assigned a PIN code to access the grading study.

Be sure to download a CSV file with the credentials and communicate them to your participants. To do that, press the “Export CSV” button in the upper-right corner of the screen.

Question

During the photo grading study, the assessors will be shown pictures from your dataset and asked to rate them based on a question you create. To create the question, go to the “Question” tab and type it into the text bar. There can only be one question per study.

Here, you can also set up your grading scale and the names of minimal/maximum values. For example, for the question “How irritated is this person’s skin?”, you can name the minimum value “Not irritated at all” and the maximum one “Very irritated”. You can adjust the number of grades as well, but please note that it can be no greater than 7.

Don’t forget to click the “Done” button in the upper-right corner of the screen to save the changes.