109 Commits

Author SHA1 Message Date
sam marshall
a3b1e3b4ee MDL-76784 core_cache: versioned cache (modinfo) can fail in install
When cache is disabled but temporary in-memory caches are allowed,
these were direct instances of cachestore_static with no loader, which
meant that the get_versioned and set_versioned functions did not work.
2023-02-21 10:40:23 +00:00
sam marshall
3353aecc8b MDL-76791 core_cache: Locking breaks for multiple-identifiers store
Cache locking fails if the cache store supports multiple identifiers
(in core, the only two which do are cachestore_static and
cachestore_mongodb, so this is unlikely to cause severe problems).
2023-02-06 15:46:17 +00:00
sam marshall
0b2c2a1f95 MDL-76129 upgrade: Allow caching in specific functions
During install/upgrade, caching is disabled. This change provides a
way to temporarily enable caching (using in-memory cache storage only)
within a specific function; caches are deleted afterwards.

Adding this to two locations improves install performance quite a lot.

Caching is not enabled during the parts of plugin installation that
can vary for individual plugins (install.php/upgrade.php) as these
might be relying on its absence, for example by making direct database
changes.
2022-12-22 11:05:45 +00:00
Mark Johnson
67c7ed25e3 MDL-76076 cache: Add locking before write for parent caches 2022-10-31 13:55:54 +00:00
Mark Johnson
045ee091da MDL-67020 Cache: Make locking work for local file caches
* Add new requirelockingbeforewrite flag for cache definitions
* Implement native locking in cachestore_file (leveraging
file_lock_factory).
2022-10-20 11:31:33 +01:00
Eloy Lafuente (stronk7)
9f53b0e965 MDL-73971 phpunit: Move more tests to use correct names and namespaces
Applied the following changes to various testcase classes:

- Namespaced with component[\level2-API]
- Moved to level2-API subdirectory when required.
- Fixed incorrect use statements with leading backslash.
- Remove file phpdoc block
- Remove MOODLE_INTERNAL if not needed.
- Changed code to point to global scope when needed.
- Fix some relative paths and comments here and there.
- All them passing individually.
- Complete runs passing too.

Special mention to:

- Some fixtures, initially defined in the test files have been
  moved to new files in fixtures subdirectory, leaving the unit
  test files clearer:
  - moodle2_course_format_test.php
- Rename wrong named test:
  - baseoptiogroup_test = baseoptigroup_test
2022-05-06 18:29:11 +02:00
Sujith Haridasan
1bb972ae0a MDL-73848 administration: Add tests for breadcrumbs and nav highlights
This change includes addition of tests for verifying the secondary
and tertiary nav highlights.
It also includes the tests to verify the breadcrumbs for the pages.
2022-03-25 17:44:58 +08:00
Eloy Lafuente (stronk7)
4003e31983 MDL-73785 phpunit: Move more tests to use correct names and namespaces
Applied the following changes to various testcase classes:

- Namespaced with component[\level2-API]
- Moved to level2-API subdirectory when required.
- Fixed incorrect use statements with leading backslash.
- Remove file phpdoc block
- Remove MOODLE_INTERNAL if not needed.
- Changed code to point to global scope when needed.
- Fix some relative paths and comments here and there.
- All them passing individually.
- Complete runs passing too.

Special mention to:

- When belonging to other components and being valid api:
  - analytics related tests have been moved to tests/analytics subdir.
  - backup & restore related tests have been moved to tests/backup subdir.
  - events related tests have been moved to tests/event subdir.
  - privacy related tests have been moved to tests/privacy.
  - task related tests have been moved to tests/task subdir.
- Some simple renames, not including the component part anymore (not
  needed now that they are namespaced):
  - some xxxlib_test.php have been renamed lib_test.php
    (when they where testing the corresponding lib.php).
  - cache stores tests have been all renamed store_test, originally
    each one had its own name (file_test, apcu_test, redis_test...)
  - assign feedback tests have been all renamed feedback_test, originally
    each one had its own name (file_test, editpdf_test...)
2022-03-03 12:19:30 +01:00
sam marshall
8a0f706033 MDL-72837 core_cache: Add versioned cache support
Adds new set_versioned and get_versioned APIs to cache, which means you can
request a specific version from cache and it will not return an outdated
version.

This is important when using multi-layer-caches where a local cache might have
an outdated version of the cache, but the shared cache has a current version.
With this feature, the content of the cache does not have to be rebuilt, as
it will automatically retrieve it from the shared cache if necessary.
2022-02-23 16:58:38 +00:00
sam marshall
691c5b8326 MDL-72837 core_cache: Rename unit test classes to new standard
For the three unit tests that are going to be modified in future
commit, I first changed them to use a namespace and class name
matching filename, as per current standard.
2022-02-23 16:36:40 +00:00
Eloy Lafuente (stronk7)
c11096576a Merge branch 'MDL-72619-master' of https://github.com/sammarshallou/moodle 2021-11-03 18:15:28 +01:00
sam marshall
9c29979b8b MDL-72596 core_cache: Track cache I/O size in perfdebug
For cache types which mean this information can be obtained without a
significant performance cost (i.e. just by calling strlen and not
having to serialize something that wasn't serialized already),
this change calculates the size of data read from or written to cache
in each request and includes it in the perfdebug table at bottom of
output (when that is turned on).

This supports the following cache types:

* File store
* Redis (only if caching is enabled)
2021-10-19 17:00:48 +01:00
sam marshall
1b94bb8c20 MDL-72619 core_cache: Provide admin page to view cache size estimates 2021-10-11 16:49:21 +01:00
Eloy Lafuente (stronk7)
309a65a6f7 MDL-71036 phpunit: Renamed various file-related assertions
In PHPUnit 9.1, the following file-related assertions
have been deprecated and there are new alternatives for
all them:
- assertNotIsReadable()         -> assertIsNotReadable()
- assertNotIsWritable()         -> assertIsNotWritable()
- assertDirectoryNotExists()    -> assertDirectoryDoesNotExist()
- assertDirectoryNotIsReadable()-> assertDirectoryIsNotReadable()
- assertDirectoryNotIsWritable()-> assertDirectoryIsNotWritable()
- assertFileNotExists()         -> assertFileDoesNotExist()
- assertFileNotIsReadable()     -> assertFileIsNotReadable()
- assertFileNotIsWritable()     -> assertFileIsNotWritable()

This is about to, simply, move all cases to the new alternatives.

Source: https://github.com/sebastianbergmann/phpunit/blob/9.1.0/ChangeLog-9.1.md

Regexp to find all them:

ag 'assertNotIsReadable|assertNotIsWritable|assertDirectoryNotExists|\
assertDirectoryNotIsReadable|assertDirectoryNotIsWritable|\
assertFileNotExists|assertFileNotIsReadable|assertFileNotIsWritable'
2021-03-11 19:22:23 +01:00
Eloy Lafuente (stronk7)
3a5641cb74 MDL-67673 phpunit: Remove deprecated assertEquals() params
The optional parameters of assertEquals() and assertNotEquals()
are deprecated in PHPUnit 8 (to be removed in PHPUnit 9):

- delta => use assertEqualsWithDelta()
- canonicalize => use assertEqualsCanonicalizing()
- ignoreCase => use assertEqualsIgnoringCase
- maxDepth => removed without replacement.

More info @ https://github.com/sebastianbergmann/phpunit/issues/3341

Initial search done with:

ag 'assert(Not)?Equals\(.*,.*,' --php

Then, running tests and fixing remaining cases.
2020-10-21 12:46:00 +02:00
Eloy Lafuente (stronk7)
40de097e65 MDL-67673 phpunit: Remove deprecated assertContains() uses on strings
Both assertContains() and assertNotContains() are deprecated in PHPUnit 8
for operations on strings. Also the optional case parameter is. All uses
must be changed to one of:

- assertStringContainsString()
- assertStringContainsStringIgnoringCase()
- assertStringNotContainsString()
- assertStringNotContainsStringIgnoringCase()

More info: https://github.com/sebastianbergmann/phpunit/issues/3422

Regexp to find all uses:

ag 'assert(Not)?Contains\('
2020-10-21 12:46:00 +02:00
Eloy Lafuente (stronk7)
f94195c320 MDL-67673 phpunit: Remove deprecated assertInternalType()
While this is not strictly required, because removal will
happen in PHPUnit 9.0, we are already getting rid of all
uses in core.

From release notes:https://phpunit.de/announcements/phpunit-8.html

assertInternalType() is deprecated and will be removed in
PHPUnit 9. Refactor your test to use assertIsArray(), assertIsBool(),
assertIsFloat(), assertIsInt(), assertIsNumeric(), assertIsObject(),
assertIsResource(), assertIsString(), assertIsScalar(),
assertIsCallable(), or assertIsIterable() instead.
2020-10-21 12:45:59 +02:00
Eloy Lafuente (stronk7)
f6711bb394 MDL-67673 phpunit: Fix the return type of template methods
All the setup/teardown/pre/post/conditions template methods
now are required to return void. This was warned with phpunit 7
and now is enforced.

At the same time, fix a few wrong function names,
provider data and param types, return statements...
2020-10-21 12:45:59 +02:00
Peter Burnett
3d49421a21 MDL-68440 cache: Added counting of default mappings 2020-10-12 11:41:00 +10:00
Peter Burnett
ba9fc5ed57 MDL-41492 cache: Added ability for alternate cache config 2020-09-28 15:12:33 +10:00
Andrew Nicols
7f7df727c2 MDL-42012 cache: Valid definitions are required even when disabled 2020-09-09 14:04:17 +08:00
Andrew Nicols
063c9ee94f MDL-42012 cache: Allow data sources to work when caching is disabled 2020-09-09 13:59:41 +08:00
Paul Holden
2d70187ade MDL-69005 cache: fix private $store access in loader. 2020-06-10 10:31:47 +01:00
Brendan Heywood
9f4bb48e40 MDL-68329 cache: Improve cache performance footer info 2020-04-28 10:08:28 +10:00
David Mudrák
ea43f6c1c8 MDL-64452 cache: Add tests for key prefix based on session id
Unfortunately, we can't simply use session_id() to regenerate the
session id in unit tests. Starting from PHP 7.2, it would trigger
"session_id(): Cannot change session id when headers already sent",
refer to MDL-60978 and PHP bug #75628 for more details.

As a workaround, we use a static property allowing us to inject the
value that we then use as a session identifier. This is reasonably
enough to make sure that the identifier is used as a part of the key
prefix.
2019-02-15 09:36:43 +01:00
Andrew Nicols
0a197875df MDL-63101 cache: Improve accuracy of cache event invalidation 2018-08-27 14:43:42 +08:00
sam marshall
4998d3a5be MDL-59427 MUC: cache_helper::purge_all should reset adhoc caches 2017-06-30 11:15:30 +01:00
Skylar Kelty
761ac5a0aa
MDL-58658 caching: Add static acceleration unit test for empty arrays 2017-05-11 10:56:04 +01:00
Eloy Lafuente (stronk7)
4d505af519 Merge branch 'MDL-55267-master-deprecation' of https://github.com/marcusgreen/moodle 2017-03-21 10:18:56 +01:00
Rajesh Taneja
dba39da081 MDL-58169 phpunit: Clear opcode cache before checking file
In unit test, we write cache file often and
opcache might fail to detect changes. It will
be nice to invalidate opcode cache to ensure
we have the correct file_exists status
2017-03-14 13:30:24 +08:00
Marcus Green
f2894f322d MDL-55267 cache: Remove deprecated field datasourceaggregate 2017-03-13 16:31:45 +00:00
Russell Smith
f3789f2fb3 MDL-45584 cache: Make identifiers part of the cache creation.
It is now safe to cache a reference to a cache and expect consistent results.

Changing identifiers altered cache results where a reference was
held to the cache. Identifiers have been set to be cached with
identifiers included so the caches are separate.

As a consequence of this it was identified that invalidation events
and identifiers don't easily work together as an event can't determine
which identifiers should be used for cache invalidation.  So invalidation
events have been made incompatible with identifiers being set.  No core
code used this combination as it's not possible to understand any expected
behaviour.

Event invalidation for application and session caches was centralised to the same
location.  The only difference was the name of the lastinvalidation variable. This
improves support and consistency of invalidation code.
2017-03-02 11:06:17 +08:00
Jonathan Champ
d3a6b91aec MDL-57427 cache: Use perfdebug only when enabled 2017-01-09 10:18:13 -05:00
Rajesh Taneja
96813571b1
MDL-57147 phpunit: Check if time is greater than or equal to
Some windows only have 1/100 microtime resolution
so a lot of times it does not advance enough.
2016-11-28 09:40:21 +08:00
Andrew Nicols
8fa816e0b0 Merge branch 'wip-mdl-56273' of https://github.com/rajeshtaneja/moodle 2016-11-02 14:30:08 +08:00
Rajesh Taneja
4a910b9aa6
MDL-56273 cache: Fixed initialise call
After store is created, it should be checked if
it's ready before calling initialise
2016-11-02 14:26:50 +08:00
Andrew Nicols
c6ae02c184 Merge branch 'MDL-48468' of https://github.com/mr-russ/moodle 2016-11-02 10:13:42 +08:00
Russell Smith
a330e83ade MDL-56273 cache: Use is_ready() to determine if cache is really ready. 2016-10-31 19:44:05 +11:00
Russell Smith
a169739d5d MDL-56273 cache: Use cache initialise always.
purge_all() and purge_by_definition() look in the configuration
for which caches are available and then creates them to purge them.
The configuration stores the values used by initialise(), not
initialise_unit_test_instance() and would therefore fail to purge
all caches if they were not purged by another means.

In the case of filestore, it's purged by unit tests, in the case
of memcache(d), it purges the whole store when a single definition
is requested.

Therefore all configuration was moved into the configuration file
during unit tests and does not have any special override codes in
the unit test infrastructure.
2016-10-30 13:42:43 +11:00
Russell Smith
68b1e13094 MDL-48468 cache: Import Redis cachestore
This code was originally taken from; https://github.com/durana/moodle-cachestore_redis
Special thanks to Mark Nielsen and Adam Durana for their work on this.

Since the original import;
- Mode session support has been added.
- Some cleanup of names has been made.
- Data guarantee and references support was added.
2016-10-30 13:34:43 +11:00
David Monllao
ab9ca549e7 Merge branch 'wip-mdl-56347' of https://github.com/rajeshtaneja/moodle 2016-10-25 15:20:06 +08:00
Russell Smith
ae9e735c77
MDL-56347 caching: Better class variables for notices. 2016-10-21 09:21:11 +08:00
Rajesh Taneja
502b66f53d
MDL-56347 phpunit: Set class for alternative cache store
If alternative cache store is sued, then set the class
and warn user about this.
2016-10-21 09:20:37 +08:00
Russell Smith
ec4b83b92a MDL-55922 cache: Improve static cache performance
Static cache has a very flexible storage capability
and does not need all the checks performed on it.
Enabling dereferencing handling and multiple identifiers
removes a lot of unneeded overhead for request caches that
have many get calls.

The changes were modelled on the static acceleration changes
done in MDL-53208.
2016-09-12 11:58:55 +10:00
David Monllao
4b5e0c542a MDL-55604 cache: Updating comment according to code changes 2016-08-24 09:39:15 +08:00
Russell Smith
7ff43e1acf MDL-55604 cache: Fix identifier and cacheable_object
When a cachable object is store in the static cache from
the backing store, it was incorrect serialised rather than
using the wake function.  This has been resolved and tests added.

During the investigation into cacheable_object, it was discovered
that set_identifiers never removes identifiers when you call it,
so set_identifiers(array('a')) and set_identifiers(array('b')) really
resulted in array('a','b') as the identifiers rather than 'b'.

The fix for this issue depends on the set_identifiers fix and
they have been coupled together as a result.
2016-08-24 11:26:20 +10:00
Jun Pataleta
d6806d8eaf MDL-54984 cache: Added todo tags to items for 3.3 deprecations 2016-07-26 00:46:46 +08:00
Russell Smith
98d580888b MDL-53208 caching: parse_key not required for static cache.
static cache operates on a single cache instance, it can't share
with others and can use the supplied key immediately.
2016-02-28 17:01:50 +11:00
Russell Smith
5007975d1e MDL-53206 caching: Ensure get_many and set_many dereference objects. 2016-02-28 17:01:49 +11:00
Matt Petro
2351ed55e8 MDL-51482 caching: Fix unnecessary updates to lastinvalidation time 2015-10-08 11:48:17 -04:00