This commit contains three changes in the three files:
1. A bug in the backup process meant that anything logged after a
certain point did not appear in the on-screen display of the
backup log, because the logger was serialised and deserialised
but display code referred to the old version. Changed so that
code retrieves new object.
2. Add more information to backup log when there is a missing file.
3. Add more information to restore log when there is a missing file
(and remove existing code duplication of the current message).
The 'missing file' situation is one that generally shouldn't occur in
normal usage, but when it does happen, it is useful to have full
information about the file.
Files bigger than 4GB is not supported at this point. Zip archiver doesn't know about it and always returns true. We need to verify if it is a valid file or not and delete if not
Also delete the backup file immediately if anything goes wrong anywhere in backup. We don't want to use up space
If a backup contains a large number of questions, the
create_question_files function can take a very long time. This
change adds progress reporting during that function.
When restoring a backup that contains a large number of users
to a different server where those users don't already exist,
creating users can take a significant time. This change adds
progress reporting so that it doesn't time out and shows
activity in the user interface while doing the creation.
In the case of a very large backup with many activities, simply loading
the restore controller (specifically, building the plan) can take a long
time and cause timeouts. This change adds progress reporting while the
restore controller is being loaded.
(This also reduces the time between first going to the page, and the
progress bar appearing, making a slightly better user experience.)
Conflicts:
backup/restore.php
Deleting directories at the end of backup can take a long time if
there are many directories (potentially up to 65536 file
directories). This change reports progress while deleting
directories to avoid timeout.
The new experimental setting enabletgzbackups allows backups to be
created so that the internal format for .mbz files is .tar.gz.
Restore transparently supports .mbz files with either internal
formats (.zip or .tar.gz).
The .tar.gz format has the following benefits for backup:
- Supports larger files (no limit on total size, 8GB on single file
vs. 4GB limit on total size)
- Compresses text better, resulting in smaller .mbz files.
- Reports progress regularly during compression of single files,
reducing the chance of timeouts during backups that include a
very large file.
Time performance may also be improved although I haven't done a
direct comparison.
When doing a backup or restore operation with the normal user interface,
if there is anything in the backup/restore log, it will now be displayed
at the end of the process below the Continue button.
If you have developer debugging enabled, it sets the display level to
LOG_DEBUG and there will always be content in the log. Otherwise, it
uses LOG_INFO which generally means the log is empty, and nothing
displays (no visible change for users).
This parallels question_attempt->minfraction, which allows the
fractional mark to go below zere.
This is needed to allow the certainty-base marking behaviours to work
better.
Adds 'Preparing page display' progress bars for user interface pages
if they take a long time to display.
Also adds changes where other parts of the backup progress timed out on
long backups. After this change, and MDL-41838, it is finally possible on
my dev server to successfully back up the 'XL' test course.
Now users can choose to exclude the question bank from the backup,
this will in turn prevent any modules which use the question bank
from being included in the backup.
This patch includes:
* version column removed from modules table, now using standard config, this allows decimal version for modules
* version column removed from block table, now using standard config, this allows decimal version for blocks
* module version.php can safely use $plugins instead of module
* new plugin_manager bulk caching, this should help with MUC performance when logged in as admin
* all missing plugins are now in plugin overview (previously only blocks and modules)
* simplified code and improved coding style
* reworked plugin_manager unit tests - now using real plugins instead of mocks
* unit tests now fail if any plugin does not contain proper version.php file
* allow uninstall of deleted filters
1. Changes progress bar code to allow headings for progress bar (so users have
some clue what's going on if a page has more than one progress bar).
2. Changes restore code so that a progress bar can display during pre-checks if
they take longer than 5 seconds.
3. Changes pre-check and restore code so that, in various points where the system
can take a long time within an individual step, intederminate progress is
indicated and it won't time out.
Adds calls to the new backup progress tracking API within various steps of the
backup system - previously it only tracked progress between steps, but some steps
can themselves be slow. This ensures the system displays progress (either by
moving the progress bar if possible, or by making the wibbler below it pulsate)
during nearly all of the backup process.
To constrain memory we only cache a single question's answers.
A quiz must display this many answers, so it will not result
in undue memory pressure.
We now match on the full answer text rather than the first 255
characters. This removes the risk of matching questions with the
same start.
encode_backup_temp_info() and decode_backup_temp_info() have been
introduced to keep the info field encoding in one place.
Many locations used get_backup_ids_record() to obtain info, that
makes lots of calls to get_backup_ids_record() which can be slow.
We now complete all those inline by adding the info field to the query.
To reduce memory usage, all queries of that nature have been changed
to use get_recordset_*. gzcompress was introduced if available to minimize
traffic to/from the database and to decrease the memory required for caching.
The compression time is saved by the benefits in other places of having smaller data.
