Implements a mechanism by which search engines can provide different
result orderings, and implements a 'by location' ordering within the
Solr search engine (available whenever the user starts their search
from within a course or activity).
Adds group support to the core search API and the Solr search engine.
This allows for:
* User searching by group (in the API only, no interface yet)
* Automatically restrict search results by group (in some cases like
separate-groups forums)
Adds a new 'Gradual reindex' link to the search areas page for each
area. When clicked, this takes you to a confirm prompt, and then
adds each context from that search area to the indexing queue.
The search areas page now displays the 'Additional indexing queue'
(if it is non-empty). The table shows the first 10 items in the
queue, and it also indicates the total number in case there are
more. (I don't think people really need to see the entire
contents of it, so I didn't implement paging.)
Adds indexpriority field to the database table which holds a queue of
indexing requests. This allows for potentially large area reindexes
to have a lower priority, so as not to halt the special indexes that
run after a course restore.
This new API returns a list of contexts for each search area. This
allows the areas to be reindexed in a sensible order (roughly
speaking, newest first) and also allows this to be controlled by
each area.
An implementation in the forum module means that forums are ordered
by the date of the most recent discussion, so that active forums
will be reindexed early even if they were created a long time ago.
Without this change its possible that the unit tests will fail at any time.
Before this change the indexing time is measured by real-time, not fake time,
making all index timings 0.
This happens as PHP offers no guarantee around the sort-order of an array for
any given two members that equate as equal. It just happens to pass for the
current array of search areas in vanilla Moodle.
The recordsets used for search indexing sometimes return results
which are invalid (e.g. cannot be found in database). When this
happens, the result in the iterator for the recordset will be
false. Due to a bug, the iterator used to stop when it encountered
a false value, which prevented indexing from getting past the
problematic record.
In addition, the iterator that skips future data resulted in the
current() function of its parent indicator being called twice per
entry, which meant that search indexing called get_document()
twice as many times.
We were previously testing tha the parent is valid, which it was, and
then fetching the current record, before fetching data from it.
However, the way in which the recordset walk works, the valid function
checks whether the _record_ itself is valid, whilst the current allows
for a callback to be applied.
In this instance, the data-entry was failing because the count of
indexfields was < 2. The recordset data itself was valid, but the view
was not, and as a result, the current() function returned false.
This false was not previously handled.
I've changed the logic so that we handle this case, and have removed a
double-negative in the process.
The search area API now includes a new function get_document_recordset
which should be implemented in preference to the older
get_recordset_by_timestamp. (It's also possible to implement both in
plugin search areas which need to work against older Moodle versions.)
Existing search areas without the new function will continue to work as
before (obviously without the new functionality).
New API \core_search\manager::request_index($context, $areaid = '')
adds the given context to a list which is intended to be indexed
later by the scheduled task.
New function \core_search\manager::is_indexing_enabled(), analagous
to existing is_global_search_enabled().
This replaces existing duplicated code, ready for more use in
following commits.
If an indexed file has been modified or deleted the search index
will still reference the old file. Remove the file from search
results until the change has been indexed.
Increase the scope of the files that are detected and
indexed by Moodle's Global Search. This includes intro
and content areas of search areas. The use case here is
in a description for an activity there is a file added
that contains more information about the activity.
It would an advantage to be able to search this content
as well. It would also allow search engine plugins to be
able to index non text files such as images.
Allow Global Search 3rd party engines to be much faster by supporting batch processing.
Refactor the iterator loop for the documents to add to the search index into its own method.
Move this new method from the manger class to the base engine class.
The move to the base engine class will allow search engine plugins to override this and determine
how they implment it. It will not break the existing interface contract with existing plugins.
Finally, add an elasped time indication to the trace output.
