127 Commits

Author SHA1 Message Date
Meirza
f0aa52f75e MDL-77350 search: Added class properties that are not declared
In PHP 8.2 and later, setting a value to an undeclared class property is
deprecated and emits a deprecation notice.
So we need to add missing class properties that still need to be declared.
2023-05-16 00:33:19 +07:00
Jun Pataleta
9095bb0ef9 MDL-78000 upgrade: add the 4.2.0 separation line to all upgrade scripts 2023-04-22 23:17:50 +08:00
Eloy Lafuente (stronk7)
061c9d7ba0 MDL-77940 versions: bump all versions and requires near release
version = 2023042400 release version
requires= 2023041800 current rc1 version
2023-04-18 18:08:09 +02:00
Marina Glancy
8fc1486d36 MDL-77164 various: fix incorrect phpdocs 2023-04-13 11:35:06 +01:00
Ilya Tregubov
05535e38df Merge branch 'MDL-76497-master' of https://github.com/junpataleta/moodle 2022-12-01 08:56:09 +03:00
Marina Glancy
a800e7e62f MDL-74853 various: add second parameter to htmlentities functions
Default value of the $flag argument changed in PHP 8.1 from ENT_COMPAT to ENT_QUOTES | ENT_SUBSTITUTE
To ensure consistent behavior across different PHP version the second parameter is now required for the functions:
htmlspecialchars(), htmlentities(), htmlspecialchars_decode(), html_entity_decode() and get_html_translation_table()
2022-11-28 16:12:20 +01:00
Jun Pataleta
c6ab792ddf MDL-76497 upgrade: add the 4.1.0 separation line to all upgrade scripts 2022-11-28 14:43:04 +08:00
Jun Pataleta
00bae8b8a4 MDL-76403 versions: bump all versions and requires near release 2022-11-23 09:45:23 +08:00
Jun Pataleta
7a87871c25 MDL-70870 search_simpledb: Sort search queries via doc ID 2022-06-02 20:26:23 +08:00
Renaud Lemaire
1ecce127be MDL-70870 core_search: db indexes optimization 2022-05-30 09:09:59 +02:00
Jun Pataleta
70f6b1605b MDL-74471 versions: bump all versions and requires near release 2022-04-13 10:37:36 +08:00
Sara Arjona
f2ff93b3bc MDL-74419 search_simpledb: Search with quotes not working on Oracle
Global search with quotes is not working on DB engines without
full-text search support (i.e. Oracle).
2022-04-06 11:32:50 +02:00
Eloy Lafuente (stronk7)
4003e31983 MDL-73785 phpunit: Move more tests to use correct names and namespaces
Applied the following changes to various testcase classes:

- Namespaced with component[\level2-API]
- Moved to level2-API subdirectory when required.
- Fixed incorrect use statements with leading backslash.
- Remove file phpdoc block
- Remove MOODLE_INTERNAL if not needed.
- Changed code to point to global scope when needed.
- Fix some relative paths and comments here and there.
- All them passing individually.
- Complete runs passing too.

Special mention to:

- When belonging to other components and being valid api:
  - analytics related tests have been moved to tests/analytics subdir.
  - backup & restore related tests have been moved to tests/backup subdir.
  - events related tests have been moved to tests/event subdir.
  - privacy related tests have been moved to tests/privacy.
  - task related tests have been moved to tests/task subdir.
- Some simple renames, not including the component part anymore (not
  needed now that they are namespaced):
  - some xxxlib_test.php have been renamed lib_test.php
    (when they where testing the corresponding lib.php).
  - cache stores tests have been all renamed store_test, originally
    each one had its own name (file_test, apcu_test, redis_test...)
  - assign feedback tests have been all renamed feedback_test, originally
    each one had its own name (file_test, editpdf_test...)
2022-03-03 12:19:30 +01:00
Eloy Lafuente (stronk7)
3bb4a4009c MDL-73764 tool_xmldb: Regenerate all the files detected by the report
Note that, instead of going one by one to them from the XMLDB Editor
(load, makes some tiny change, save, revert the change, save), the
report includes a commented line that, once uncommented, does regenerate
the file while reporting.

I've left it there as a commodity for the developer.
2022-02-08 19:25:03 +01:00
Eloy Lafuente (stronk7)
e4a2d9c1d4 MDL-73348 phpunit: normalize all privacy provider tests
As far as now all them have correct privacy level2 namespace:
- Move them to "privacy" subdir.
- Rename the files to "provider_test.php", this includes old
  privacy_test.php and privacy_provider_test.php files
- Rename the testcase to provider_test too (to match file name)

Also, change some relative paths and comments to point to new
locations.
2021-12-17 14:21:02 +01:00
Eloy Lafuente (stronk7)
57e6fb7ad9 MDL-73348 phpunit: privacy and privacy_provider testcase names
All privacy_test and privacy_provider_test classes:

- Namespaced with component\privacy.
- Fixed incorrect use statements with leading backslash.
- Changed code to point to global scope when needed.
- Renamed a few files to make all be privacy_test or privacy_provider_test.php
- All them passing individually.
- Complete runs passing too.
2021-12-17 14:21:02 +01:00
Jake Dallimore
6f5719ff29 MDL-72118 search: fix unit test dependant on HTML block pluginname
Change this to use "Block: Text" instead.
2021-09-16 11:51:35 +08:00
Eloy Lafuente (stronk7)
515fc0447e MDL-71731 phpunit: isolate tests requiring lots of peak mem
It's mostly addDocuments(), used by test_add_document_batch_large()
with 100 big documents what requires a lot of ram
although apparently, it's freed (partially) once ended.
Just the peak usage remains high. In any case, isolating that
test to avoid the non-freed side of it to consume too
much memory for the rest. (We allow "only" 2GB).

Still, I think that there is a good work about to detect which
tests are leaking / consuming too much, I'd bet there are
a bunch running completely out of control.
2021-05-27 19:42:22 +02:00
sam marshall
cca005b48a MDL-70446 search_solr: File indexing can fail due to multipart upload
Due to Solr bug SOLR-15039, uploading files for indexing can fail if
it uses multipart upload. This changes it to use direct binary upload.

Unfortunately, the direct binary version in PHP curl only supports a
string, so we have to load the file into memory. I added extra code to
restrict the size of files indexed to (memory limit - 100MB), which is
usually 284MB unless configured differently because cron runs under
MEMORY_EXTRA.
2021-04-01 11:18:33 +01:00
Eloy Lafuente (stronk7)
ba5b6089d5 MDL-71036 phpunit: Renamed various regexp-related assertions
In PHPUnit 9.1, the following regexp-related assertions
have been deprecated and there are new alternatives for
all them:
    - assertRegExp()     -> assertMatchesRegularExpression()
    - assertNotRegExp()  -> assertDoesNotMatchRegularExpression()

This is about to, simply, move all cases to the new alternatives.

Source: https://github.com/sebastianbergmann/phpunit/blob/9.1.0/ChangeLog-9.1.md

Regexp to find all them:

    ag 'assertRegExp|assertNotRegExp' -li
2021-03-11 19:22:24 +01:00
Eloy Lafuente (stronk7)
40de097e65 MDL-67673 phpunit: Remove deprecated assertContains() uses on strings
Both assertContains() and assertNotContains() are deprecated in PHPUnit 8
for operations on strings. Also the optional case parameter is. All uses
must be changed to one of:

- assertStringContainsString()
- assertStringContainsStringIgnoringCase()
- assertStringNotContainsString()
- assertStringNotContainsStringIgnoringCase()

More info: https://github.com/sebastianbergmann/phpunit/issues/3422

Regexp to find all uses:

ag 'assert(Not)?Contains\('
2020-10-21 12:46:00 +02:00
Eloy Lafuente (stronk7)
f6711bb394 MDL-67673 phpunit: Fix the return type of template methods
All the setup/teardown/pre/post/conditions template methods
now are required to return void. This was warned with phpunit 7
and now is enforced.

At the same time, fix a few wrong function names,
provider data and param types, return statements...
2020-10-21 12:45:59 +02:00
Eloy Lafuente (stronk7)
fbb0767536 MDL-69475 versions: bump all versions and requires in master
version = 2021052500 release version
requires= 2021052500 same than version

Why 20210525? (25th May 2021) ?

Because master is going to be Moodle 4.0, to be released
on November 2021. And, until then, we are going to have
a couple of "intermediate" releases:

- Moodle 3.10 to be released 9th November 2020. (2020110900)

  This version will be using versions from today to 2020110900
  (once it's released the YYYYMMDD part stops advancing).

- Moodle 3.11 to be released 10th May 2021. (2021051000)

  This version will be using versions from 3.10 release to 2021051000
  (once it's released the YYYYMMDD part stops advancing).

That means that all versions from today to 2021051000 are going
to be used by those 2 "intermediate" releases (3.10 and 3.11).

And we cannot use them in master, because it's forbidden to have
any overlapping of versions between branches (or different upgrade
paths will fail).

So, get that 2021051000, let's add it a couple of weeks to cover
the on-sync period (or a 2 weeks delay max!) and, the first version
that master can "own" in exclusive (without any overlap) is, exactly,
25th May 2021, hence our 20210525.
2020-08-18 00:47:15 +02:00
sam marshall
679e8d8b66 MDL-68729 Search: Allow query on one server while indexing another
To support transitions from one search engine to a different one, or
to a different installation of the same kind, this feature allows for
queries to use a different search engine from indexing. So you can
reindex (and do all other search operation) on one server, while
user queries are unaffected on a different server.

This feature supports changing between search engine types, and also
between two Solr installations.
2020-08-06 12:12:01 +01:00
sam marshall
9b3646417e MDL-68726 Search: Stop Solr 'optimize' behaviour
The optimize feature in Solr is usually considered harmful, especially
prior to Solr 7.5.

This change simply removes the optimize implementation from the Solr
engine.
2020-07-17 17:11:57 +01:00
sam marshall
0deb19468d MDL-68690 Search: Allow Solr to add documents in batches
Adding documents in batches instead of one at a time can make
indexing using Solr significantly faster.

This adds new API functions for search engines, including
add_document_batch() to add a batch of documents,
supports_add_document_batch(), get_batch_max_documents() and
get_batch_max_content().
2020-07-10 11:32:46 +01:00
Eloy Lafuente (stronk7)
115cc0214f MDL-68973 versions: bump all versions and requires near release
version = 2020061500 release version
requires= 2020060900 current rc1 (week7roll1) version
2020-06-09 16:23:09 +02:00
Eloy Lafuente (stronk7)
d5be859063 MDL-67199 versions: bump all versions and requires near release
version = 2019111800 release version
requires= 2019111200 current beta (week7roll1) version
2019-11-12 17:57:08 +01:00
Mikhail Golenkov
17fed0d1cc MDL-66973 search: Reset Content writer after test. 2019-10-21 17:55:30 +11:00
Eloy Lafuente (stronk7)
78168d7c9b MDL-66326 Global search: Unit test fix for Oracle 2019-08-30 00:17:12 +02:00
sam marshall
7ba2a20166 MDL-66326 Global search: Delete from index when courses are deleted
Adds new API support within search engines for optional methods to
delete data for courses and contexts, and implements this for the
two core search plugins (simpledb and solr).

The new API is automatically called when courses or contexts are
deleted. When a whole course is deleted, it only sends the course
delete rather than sending 1,000 separate context deletions as
each activity/block is deleted.
2019-08-23 11:51:50 +01:00
Matteo Scaramuccia
463421144f MDL-66140 search: Mitigate PECL solr extension regression
Refs:
- https://bugs.php.net/bug.php?id=72740
- https://bugs.php.net/bug.php?id=75631
- https://github.com/php/pecl-search_engine-solr/pull/16
2019-08-11 20:03:25 +02:00
sam marshall
c207289127 MDL-65806 Search: Solr does not find words in italic 2019-06-11 16:40:15 +01:00
Eloy Lafuente (stronk7)
c9a1a6341c MDL-65571 versions: bump all versions and requires near release
version = 2019052000 release version
requires= 2019051100 current beta (week6roll2) version
2019-05-12 23:46:34 +02:00
Dmitrii Metelkin
2085e8603a MDL-62869 search: implement searching for all courses 2019-03-01 07:18:37 +11:00
Paul Holden
1a36b8b5aa MDL-64190 search_solr: use password unmask config elements. 2019-01-11 15:11:24 +00:00
Eloy Lafuente (stronk7)
085353b315 MDL-64282 versions: bump all versions and requires near release
version = 2018120300 release version
requires= 2018112800 current rc2 (week7roll1) version
2018-11-29 18:24:08 +01:00
Matteo Scaramuccia
13719f2ebb MDL-63789 search_solr: use proxy creds, if any
Also use proxy port, IF any.
2018-11-02 23:13:15 +01:00
Michael Hawkins
fc087fd29e MDL-63552 search_solr: Add support for removal of context users
This issue is a part of the MDL-62560 Epic.
2018-10-25 11:41:08 +08:00
Michael Hawkins
4cef924482 MDL-63552 search_simpledb: Add support for removal of context users
This issue is a part of the MDL-62560 Epic.
Also added test_get_contexts_for_userid unit test.
2018-10-25 11:41:08 +08:00
Helen Foster
9722f6a25e MDL-62763 lang: Merge English strings from the en_fix language pack
Significant string changes:

* various strings in tool_dataprivacy and tool_policy - 'Data Protection
  Officer' changed to 'privacy officer'
* idnumbermod_help,core - mentioning resource and custom reporting
2018-07-02 11:15:26 +02:00
David Monllao
99a080f12a Merge branch 'MDL-62361' of git://github.com/stronk7/moodle 2018-05-10 09:17:25 +08:00
David Monllao
94522f8dd0 MDL-62217 search_solr: Plugin provider fake implementation 2018-05-09 10:57:14 +02:00
David Monllao
74fc7d3120 MDL-62217 search: Privacy providers 2018-05-09 10:56:02 +02:00
Eloy Lafuente (stronk7)
57ea6753d3 MDL-62361 versions: bump all versions and requires near release
version = 2018051400 release version
requires= 2018050800 current beta+ (week4roll1) version
2018-05-09 02:46:30 +02:00
Eloy Lafuente (stronk7)
57565dc3b2 Merge branch 'MDL-62183_master' of git://github.com/dmonllao/moodle 2018-04-24 15:49:16 +02:00
David Monllao
b137e02973 MDL-62183 search_solr: Set solr as search engine for solr unit tests 2018-04-24 13:12:48 +02:00
David Monllao
590c774d37 Merge branch 'MDL-62042-master' of https://github.com/sammarshallou/moodle 2018-04-23 16:07:37 +02:00
David Monllao
c5a8065258 Merge branch 'MDL-61028-master' of https://github.com/sammarshallou/moodle 2018-04-23 13:09:53 +02:00
sam marshall
222a97ce76 MDL-61028 core_search: Allow filtering search by user (back-end, Solr)
Adds support in the Solr engine for a new 'userids' parameter to
restrict search results to those created by a specific user or users.
2018-04-20 13:36:55 +01:00