132 Commits

Author SHA1 Message Date
David Monllao
dd13fc222c MDL-59694 analytics: Track processed analysables 2017-10-09 07:47:28 +02:00
Andrew Nicols
1afd3c72aa Merge branch 'MDL-59630_master' of git://github.com/dmonllao/moodle 2017-10-05 14:56:57 +08:00
David Monllao
abafbc8405 MDL-59630 analytics: mlbackend model directories deletion 2017-10-03 10:34:47 +02:00
David Monllao
f9222c4953 MDL-59630 analytics: New clean up task 2017-10-03 10:34:43 +02:00
David Monllao
99b84a26f9 MDL-59630 analytics: Purge everything when the model is deleted 2017-10-03 10:07:32 +02:00
David Monllao
aa1b04dc81 MDL-60003 analytics: Skip samples with only null values 2017-09-28 16:56:33 +02:00
David Monllao
6faa982986 MDL-58835 analytics: Unit tests for prediction actions 2017-09-07 14:45:34 +02:00
Eloy Lafuente (stronk7)
9533cf3419 Merge branch 'MDL-59779_master' of git://github.com/dmonllao/moodle 2017-08-30 01:09:20 +02:00
Eloy Lafuente (stronk7)
a7417383fa Merge branch 'MDL-59067_master' of git://github.com/dmonllao/moodle 2017-08-23 02:55:20 +02:00
David Monllao
0690a271c3 MDL-59067 analytics: Store indicator calculations
This was supposed to be split into multiple commits to make it easier to understand
but I failed to do it properly. So this is the list of changes:

- New analytics_indicator_calc db table to store indicators calculations
- Reuse previous calculations during prediction/training; other models
  previous calculations should also be reused as long as they belong to
  the same sample (sampleid depends on sampleorigin), time range and indicator
- Allow bulk inserting of these calculations as this can hurt database performance
- Block the same analysable to be analysed for training and for prediction
- Use a new instance of the target and use it for is_valid_* functions
  as using ::is_valid_sample can lead to problems if people
  uses it to cache stuff
2017-08-22 22:28:04 +02:00
David Monllao
9955e0fe06 MDL-59661 analytics: New test for the datasets manager 2017-08-22 13:04:49 +02:00
Mark Nelson
7e00aa725a MDL-59204 analytics: changed get_name so that it returns a lang_string
This was done for indicators, targets and time splitting methods so that we
can get the string identifier and component in order to display a help_icon.

The functions were also made abstract, removing the default implementation.
Indicators, targets and time splitting methods should define this function.
2017-08-18 18:02:38 +08:00
David Monllao
cab7abec70 MDL-59779 analytics: Bulk-insert predictions 2017-08-17 20:55:38 +02:00
David Monllao
00da1e6010 MDL-59116 analytics: Multiple predictions for each analysable time range
Now we only predict using the most recent range available, this means
that if someone upgrades to moodle 3.4 at three quarters of a course
we will only calculate the latest range, previous ranges were not
displayed anyway once more recent predictions were available.

This commit deletes all previous predictions :) this shouldn't be a
problem in master as we don't provide any guarantee, the alternative
(retrive sampleids from mdl_files) would have been slow and a waste of
time as well as require horrible code in an upgrade step (text fields
do not accept defaults nor we can use NOTNULL).
2017-08-07 06:56:11 +02:00
David Monllao
466f683aa6 MDL-57791 analytics: Compatibility with oracle
Also moved predictions test to analytics/tests/prediction_test.php
2017-07-24 17:49:09 +02:00
David Monllao
382a089052 MDL-57791 tests: Restore logstores pre-test state 2017-07-24 10:43:04 +02:00
David Monllao
690ad8759d MDL-57791 analytics: Fixes during integration review
This commit includes the following changes:
- cibot complains fixes
- removed randomly failing test
- fixed course_dropout return
- other minor fixes
2017-07-24 08:37:03 +02:00
David Monllao
e709e54435 MDL-57791 analytics: Add builtin models during upgrade 2017-07-24 08:37:01 +02:00
David Monllao
2db6e9811b MDL-59212 analytics: Core indicators tests
Part of MDL-57791 epic.
2017-07-24 08:37:01 +02:00
David Monllao
206d7aa9ad MDL-59212 analytics: Move API uses out of analytics dir
Part of MDL-57791 epic.
2017-07-24 08:37:00 +02:00
David Monllao
f67bd6af70 MDL-59212 analytics: Expand unit test coverage
Part of MDL-57791 epic.
2017-07-24 08:36:59 +02:00
David Monllao
b3d6879489 MDL-59212 analytics: Analyser and static methods tests
Part of MDL-57791 epic.
2017-07-24 08:36:59 +02:00
David Monllao
1cc2b4bac3 MDL-57791 analytics: Second review round 2017-07-24 08:36:50 +02:00
David Monllao
413f19bc49 MDL-59211 analytics: Make cibot happy
Part of MDL-57791 epic.
2017-07-24 08:36:49 +02:00
David Monllao
1611308b58 MDL-57791 analytics: Changes after review
- Split model::predict in parts
- JS promises updated according to eslint-plugin-promise
- New API methods replacing direct DB queries
- Reduce insights nav link display cost
- Increase time limit as well as memory for big processes
- Move prediction action event to core
- Dataset write locking and others
- Refine last time range end time
- Removed dodgy splitting method id to int
- Replace admin_setting_predictor output_html overwrite for write_setting overwrite
- New APIs for access control
- Discard invalid samples also during prediction
2017-07-24 08:36:49 +02:00
David Monllao
584ffa4ffc MDL-57791 analytics: Replace sql queries for API calls 2017-07-24 08:36:49 +02:00
David Monllao
3a217fc394 MDL-57791 analytics: Replace settings by get_archetype_roles 2017-07-24 08:36:48 +02:00
David Monllao
cad362520f MDL-59010 analytics: Logstores initialised during tests
Part of MDL-57791 epic.
2017-07-24 08:36:47 +02:00
David Monllao
bd164083eb MDL-59057 analytics: Unit test fixes
Updating tests according to courses - site_courses analysers change.

Part of MDL-57791 epic.
2017-07-24 08:36:45 +02:00
David Monllao
e499074f39 MDL-59057 analytics: Adapt tests to static models API changes
Part of MDL-57791 epic.
2017-07-24 08:36:43 +02:00
David Monllao
8de0d1da61 MDL-57791 testing: Switch to analytics API namespace 2017-07-24 08:36:19 +02:00
David Monllao
ff656baeab MDL-58859 analytics: Add tests to core
Part of MDL-57791 epic.
2017-07-24 07:53:10 +02:00