284 Commits

Author SHA1 Message Date
David Mudrák
6109e8b272 MDL-70631 files: Replace hard-coded value with FL_ENC_RAW constant
This is not related to the issue but I noticed it while trying to debug
it.

Back when this line was introduced in 9c140a681e9, the ZipArchive had
not yet exposed this flag as a constant. It was added later with PHP
7.0.8 and we can switch to using it now.
2021-02-03 16:06:39 +01:00
David Mudrák
b227b6ea45 MDL-70631 files: Fix performance of zip_packer::extract_to_pathname()
The original implementation was based on ZipArchive::getStream() which
turns out to be very slow and if the archive contains many files, the
unzipping performance is very slow.

The patch changes the implementation to use ZipArchive::extractTo()
unless the extracted entry path contains a folder name ending with dot
(such as some/path./to/file.txt). There is a known upstream bug in the
PHP ZIP extension #77214 (also #74619 and #69477) so that we fall back
to keep using the stream in those cases.
2021-02-03 16:06:39 +01:00
Eloy Lafuente (stronk7)
40de097e65 MDL-67673 phpunit: Remove deprecated assertContains() uses on strings
Both assertContains() and assertNotContains() are deprecated in PHPUnit 8
for operations on strings. Also the optional case parameter is. All uses
must be changed to one of:

- assertStringContainsString()
- assertStringContainsStringIgnoringCase()
- assertStringNotContainsString()
- assertStringNotContainsStringIgnoringCase()

More info: https://github.com/sebastianbergmann/phpunit/issues/3422

Regexp to find all uses:

ag 'assert(Not)?Contains\('
2020-10-21 12:46:00 +02:00
Eloy Lafuente (stronk7)
d95c378771 MDL-67673 phpunit: Remove expectedException annotations
While this is not strictly required, because removal will
happen in PHPUnit 9.0, we are already getting rid of all
uses in core.

From release notes:https://phpunit.de/announcements/phpunit-8.html

The annotations `@expectedException`, `@expectedExceptionCode`,
`@expectedExceptionMessage`, and `@expectedExceptionMessageRegExp`
are now deprecated.
Using these annotations will trigger a deprecation warning
in PHPUnit 8 and in PHPUnit 9 these annotations will be removed.

Also, all uses of expectExceptionMessageRegExp() has been moved
to expectExceptionMessageMatches(). See https://github.com/sebastianbergmann/phpunit/issues/3957

TODO: Various weirdness found while doing the changes with these tests:
- vendor/bin/phpunit lib/tests/exporter_test.php (created MDL-69700)
- vendor/bin/phpunit competency/tests/external_test.php (same issue than prev one)
- vendor/bin/phpunit question/engine/tests/questionengine_test.php (created MDL-69624)
- vendor/bin/phpunit lib/tests/event_test.php (created MDL-69688)
2020-10-21 12:46:00 +02:00
Eloy Lafuente (stronk7)
f6711bb394 MDL-67673 phpunit: Fix the return type of template methods
All the setup/teardown/pre/post/conditions template methods
now are required to return void. This was warned with phpunit 7
and now is enforced.

At the same time, fix a few wrong function names,
provider data and param types, return statements...
2020-10-21 12:45:59 +02:00
Paul Holden
582bb534c9 MDL-37802 repository: return error if unzipping files fails.
For instance, when a user tried to unzip a password protected zip
archive using the file manager.
2020-09-25 09:33:44 +01:00
Mihail Geshoski
09a55f952e MDL-65115 core_files: Add get_total_content_size() in stored_file class
New function get_total_content_size() is introduced in the stored_file
class. The puprose of this function is to calculate and return the
total size (in bytes) of the content of an archive file.
2020-09-08 08:54:26 +02:00
Andrew Nicols
cc29e04f27 Merge branch '68137-master-prevent-removing-all-dots-in-filename' of https://github.com/DSI-Universite-Rennes2/moodle 2020-05-22 11:53:22 +08:00
Julien Boulen
7d918bc693 MDL-68137 core_files: replace consecutive dots in filename by single dot 2020-04-28 11:29:35 +02:00
Brendan Heywood
5bf5a7aaeb MDL-68342 files: Add a faster xsendfile method to avoid db lookups 2020-04-27 16:11:34 +10:00
Sara Arjona
799c3955e4 Merge branch 'MDL-67635-filesystem-cleanup' of https://github.com/brendanheywood/moodle 2020-02-11 13:22:54 +01:00
Brendan Heywood
2d98ebb9de MDL-67635 file_storage: Speedup file storage cleanup cron 2020-01-08 14:00:48 +11:00
Andrew Nicols
9d0192e9ce Merge branch 'MDL-58281-file-missing-500' of https://github.com/brendanheywood/moodle 2020-01-07 11:30:30 +08:00
Eloy Lafuente (stronk7)
4621917c62 MDL-67114 core: php74 fix. Fix use of scalar as array in core
There are various places where it's not guaranteed that the
variable being used is array, and instead, can be null, bool, int...

We need to check that because php74 warns about it.

Where possible we have used the coalesce operator as
replacement for isset() ternary operations.
2020-01-03 11:33:15 +01:00
Brendan Heywood
1fef1de592 MDL-58281 files: Correctly handle missing files in readfile_accel 2019-12-30 17:18:49 +11:00
Sara Arjona
5b91deed2c Merge branch 'MDL-63349-Master' of github.com:tuanngocnguyen/moodle 2019-10-14 13:58:09 +02:00
Eloy Lafuente (stronk7)
25494613f4 Merge branch 'MDL-66304' of https://github.com/golenkovm/moodle 2019-10-14 10:53:24 +02:00
Nathan Nguyen
4a6edc5767 MDL-63349 assignfeedback_editpdf: Rotate submitted image automatically 2019-10-11 15:26:53 +11:00
Andrew Nicols
fe0944987e MDL-66011 test: remove magic @covers lines 2019-10-04 09:40:34 +08:00
Mikhail Golenkov
1a15925240 MDL-66304 files: Allow xsendfile support in alternative file system. 2019-09-30 09:00:57 +10:00
Mark Nelson
94c71056a3 MDL-35773 core_backup: use existing API to recover files 2019-07-26 11:59:56 +08:00
Mark Nelson
88031d35de MDL-35773 core: revert introduction of new recover_file() method
Also -

1. Reverted recover_file() in file_system_filedir back to its
   original state.
2. Removed no longer needed method create_recovery_record().
2019-07-26 11:59:56 +08:00
Matt Porritt
d7e4481e98 MDL-35773 Backup: API should have option to not backup files
Allow both UI and automated backups to be created without
including files.  Instead include only file references.
This is essentially implementing "SAMESITE" to backup files
instead of only for import and export functionality.
A new backup setting to include files (defaults to yes)
has been included.

The restore process will also look for and attempt to
restore files from the trashdir as part of restoring
backups.  Additionally to support this process the
ammount of time files are kept in trashdir before they
are cleaned up via cron is also adjustable via admin
setting.
2019-07-26 11:59:56 +08:00
Adrian Greeve
df4d6b0696 Merge branch 'MDL-65130-master' of git://github.com/andrewnicols/moodle 2019-04-03 13:39:28 +08:00
Ryan Wyllie
cba111b376 MDL-64820 filestorage: add multi item support to get_area_files 2019-03-22 09:24:04 +08:00
Andrew Nicols
51114517be MDL-65130 files: Add @covers Annotation 2019-03-21 09:00:41 +08:00
Jake Dallimore
28c3f6e4d8 MDL-60819 core_files: fix exception thrown from generate_image_thumbnail
This function now requires gdlib.php.
2018-06-11 10:44:51 +08:00
Tim Hunt
27d625a863 MDL-62482 file_storage: may have many files with same content hash 2018-05-17 11:24:33 +01:00
Andrew Nicols
51d31432c8 MDL-61253 repository: Add referenced files to archives 2018-01-23 10:38:05 +08:00
Damyon Wiese
8c2a7f9c38 Merge branch 'wip-MDL-49995-master' of git://github.com/marinaglancy/moodle 2017-12-18 11:35:29 +08:00
Marina Glancy
80364b7b1e MDL-49995 files: delete old file when overwriting 2017-11-28 14:50:20 +08:00
sam marshall
a938e4096c MDL-60174 core_dml: fix miscellaneous incorrect recordset usage
The new recordset support for Postgres requires transactions and
will cause errors if recordsets are not closed correctly. This
commit fixes problems that were identified during unit tests, and
via some basic code analysis, across all core code. Most of these
are incorrect usage of recordset (forgetting to close them).
2017-11-27 11:10:33 +00:00
Eloy Lafuente (stronk7)
796e48a58b MDL-60611 phpunit: remove deprecated stuff and other minor fixes
Thanks @scara for MDL-60611 phpunit: improved redis skipping reasons
2017-11-05 01:45:04 +01:00
Eloy Lafuente (stronk7)
801a372dad MDL-60611 phpunit: switch to namespaced phunit classes 2017-11-02 23:45:34 +01:00
Damyon Wiese
022dccf2a7 Merge branch 'MDL-60001_gzopen' of https://github.com/mrmark/moodle 2017-10-19 13:00:13 +08:00
Eloy Lafuente (stronk7)
4c54ac22d3 Merge branch 'm34_MDL-60318_Undefined_method_file_storage-move_to_trash' of https://github.com/scara/moodle 2017-10-09 23:11:08 +02:00
Matteo Scaramuccia
5ca85a1efe MDL-60318 filestorage: Fixed undefined method move_to_trash() 2017-10-02 22:49:42 +02:00
Matt Clarkson
b1eb6dd7f2 MDL-33174 lib/filestorage: Add filename to curl request
While the lack of the filename in curl POST does not
directly affect Moodle, some external API's expect to get
the filename of the original file, which affects plugins
using the CURL class.
2017-10-02 09:27:51 +13:00
Dan Poltawski
a1b8b9c9b5 Merge branch 'MDL-59960_mimetype' of https://github.com/mrmark/moodle 2017-09-19 09:10:24 +01:00
Mark Nielsen
879827d6fa MDL-60001 core_files: Use local file for gzopen
The gzopen method does not appear to support
file streams.
2017-08-31 15:47:00 -07:00
Mark Nielsen
b757c648f3 MDL-59960 core_files: prevent mimetype update on every file update
Every time a file is updated, the mimetype is re-processed.
But, some mimetypes can only be detected if the file is present
on local disk.  With external file systems, this could cause
the file to be downloaded unnecessarily.  Instead, only update
the mimetype when the file content or the file name has changed.
2017-08-28 11:36:15 -07:00
Mark Nielsen
ee8bce2d3b MDL-59960 core_files: improve mimetype detection
Improve mimetype detection for remote files that
have no file extension.  The mimetype detection
that makes use of the file, only works with local
files, so do not use the remote path which can be
a URL or stream.
2017-08-28 11:36:15 -07:00
Mark Nielsen
9eb1a2c31a MDL-59961 core_files: make content hash validation reusable
Other file systems could make use of this code as they
all need to validate the content has and be able to read
the file from disk.
2017-08-28 11:15:12 -07:00
Cameron Ball
03756f0bdb
MDL-59700 filestorage: Rework repositories to avoid add_file_to_pool
For historical reasons repositories need to call add_file_to_pool
to sync file records. However now that a before_file_created hook
has been added additional information is needed by add_file_to_pool.

Ideally add_file_to_pool and friends will become private/protected,
so we need to remove all uses of it in core.

This patch adds some new methods to the file class to allow syncing
to be managed internally by the file and the file_storage class.
2017-08-07 15:50:15 +08:00
Cameron Ball
8243706542
MDL-59700 filestorage: Add before_file_created hook 2017-08-07 12:27:04 +08:00
gthomas2
023037dc9e MDL-57476 file: Add CUD callbacks 2017-07-14 13:51:51 +01:00
Vladimir Zhirov
ff2dab4566 MDL-59005 resource: fixed zip extraction for filenames in CP-1251/CP-866 2017-06-19 08:50:23 +08:00
Jake Dallimore
e927581a50 MDL-58684 files: Ensure stored_file never serializes the file_storage
Magic methods __sleep and __wakeup used to properly store only the
file_record property and to reinit using get_file_storage().
2017-05-08 11:12:39 +08:00
Andrew Nicols
9020212522 MDL-58718 filestorage: Normalise file paths for Windows 2017-04-26 09:36:10 +08:00
Jake Dallimore
84fb93a77e Merge branch 'MDL-58297-master' of git://github.com/andrewnicols/moodle 2017-04-11 10:49:07 +08:00