As well as fixing the bug, I also rewrote the test to use
data providers, which should lead to more useful failure messages.
And, I moved the magic number we used as the float tolerence to
be a named constant.
For behaviours that allow multiple tries, and when the teacher had chosen to
generate statistics for the first tries only, and when a student had not
attempted one question, then an error occurred. It should have been classified
as no response.
If the validation failed, so the manual grading page was re-displayed
with a validation error, then any comment that had just been typed in
was getting lost. This fixes that.
Certainty -1 has never been used in standard Moodle, but is
used in Tony-Gardiner Medwin's patches to mean 'No idea' which
we intend to implement: MDL-42077. In the mean time, these changes
avoid errors for people who have used TGM's patches.
We now compute the average CBM score, accuracy, CBM bonus and enhanced
accuracy, both for the entire quiz, and for just the questions answered.
Note that these calculations must work correctly in the presence of
descriptions, ungraded questions, and manually graded questions. For
example, imagine a essay added at the end of the quiz "Summarise what
you learned attempting this exercise." This might have max mark zero or
non-zero. The CBM statistics just ignores questions like that.
We now change so that minfraction is -6 and maxfraction is 3, so getting
the question right a low certainty gives maxmark marks, and you get a
bonus for being more confident (rather than being penalised for being
unconfident). Mathematically it is the same, but the difference is
importnat psychologically.
We also change how partially correct scores are handled.
It is too harsh to penalise a partially correct score with full
certainty by doing a linear interpolation between -6 and +3. Instead,
any partially correct score (e.g. 0.5) becomes that fraction of the
correct score (e.g. 0.5 * 3 = 1.5). Also, any incorrect score is treated
as 0, so if you have a multiple choice question that normally gives a
negative score for a wrong choice, this will now never give a score of
less than -6.
Finally we change how this is displayed to students beside the question.
Rather than saying "Marked out of 1.00", we say "Base mark 1.00", and
then later we say "CBM mark 3.00" (or whatever it is).
This parallels question_attempt->minfraction, which allows the
fractional mark to go below zere.
This is needed to allow the certainty-base marking behaviours to work
better.
It was always a bit of a hack to use static methods on the
qbehaviour_whatever classes to return metadata about the behaviour. It
is better design to have real qbehaviour_whatever_type classes to report
that metadata, particularly now that we are planning to add more such.
For example, inheritance works better with real classes. See, for
example, the improvements in
question_engine::get_behaviour_unused_display_options().
This change has been implemented in a backwards-compatbile way. Old
behaviours will continue to work. There will just be some developer debug
output to prompt people to upgrade their code properly.
At the moment, when attempt is built on the last one, "not yet answered"
message is shown, which confuses many people. This patch modifies the state to
"complete" for attempt based on previous and modifies the output string.
Many thanks to Tim Hunt for guiding me through quiz infrastructure and some code
suggestions.
1. Autosave works in some ways just like a normal save. We ultimately
call $behaviour->process_save() to do the work, and create a new step to
hold the data.
2. However, we come in through a completely different route through the
API, starting with separate auto-save methods. This keeps the auto-save
changes mostly separate, and so reduced the chance of breaking existing
working code.
3. When the time comes to store the auto-save step in the database, we
save it using a negative sequence number.
This is a clever trick that not only distinguises these steps, but also
avoids unique key errors when an auto-save and a real action happen
simultaneously. (There are unit tests for these tricky edge cases.)
4. When we load the data back from the database, most of the time the
auto-save steps are loaded back as if they were a real save, and so the
auto-saved data is used when the question is then rendered.
5. However, before we process another action, we remove the auto-saved
step, so it does not appear in the final history.
While doing this, I found various bugs in the manages question types admin page, and so fixed them, and updated the code
there to use $OUTPUT and html_writer.
AMOS BEGIN
MOV [cannotdeletemissingqtype,admin],[cannotdeletemissingqtype,question]
MOV [cannotdeleteqtypeinuse,admin],[cannotdeleteqtypeinuse,question]
MOV [cannotdeleteqtypeneeded,admin],[cannotdeleteqtypeneeded,question]
MOV [deleteqtypeareyousure,admin],[deleteqtypeareyousure,question]
MOV [deleteqtypeareyousuremessage,admin],[deleteqtypeareyousuremessage,question]
MOV [deletingqtype,admin],[deletingqtype,question]
MOV [numquestions,admin],[numquestions,question]
MOV [numquestionsandhidden,admin],[numquestionsandhidden,question]
MOV [qtypedeletefiles,admin],[qtypedeletefiles,question]
MOV [uninstallqtype,admin],[uninstallqtype,question]
AMOS END
