145 Commits

Author SHA1 Message Date
Marina Glancy
5117dee664 MDL-80821 dml: declare arguments for sql_concat() 2024-02-19 12:27:51 +00:00
Srdjan
bb1ecd8ea4 MDL-54704 dml: support for SSL with MySQL / MariaDB and PostgreSQL 2023-06-30 15:41:39 +10:00
Marina Glancy
8fc1486d36 MDL-77164 various: fix incorrect phpdocs 2023-04-13 11:35:06 +01:00
Marina Glancy
b0a83aa7bd MDL-76362 various: Avoid passing nulls to functions that don't allow nulls
PHP 8.1 is more strict on the parameter type. Functions such as trim(), strlen(), str_replace(), etc
show notice when null is passed as an argument
2023-01-23 09:15:54 +08:00
Eloy Lafuente (stronk7)
eeaaf131ac MDL-75977 xmldb: Avoid using null params on built-in DB functions
This changes a couple of cases, in postgres and oracle where we
were trying to calculate strlen() on null values (that causes a
PHP 8.1 warning).

Also, at the same time, fixing another case in xmldb_field, it
was detected that it had a bug around precision (being set to the
value of type), being used as object property, when it's not, so
it was also fixed by moving the code logic a little bit.

Note that the bug has been there since inception because there
isn't any code in Moodle using that ->precision property. It
was just detected thanks to PHP 8.2, but that's another story. :-)

Verified that with the patch:
- All DBs can be installed (phpunit install)
- Both DDL and DML pass without any ddl/dml warning (in all DBs).
2022-11-26 16:16:20 +01:00
Alex Morris
70dfee9e23
MDL-75170 dml: add sql_order_by_null method
Standardises return patterns of null values across database types
to keep with sorting nullsto the top when ascending, and
to the bottom when descending
2022-09-30 11:27:26 +13:00
Sara Arjona
eef0a430be Merge branch 'MDL-75548' of https://github.com/timhunt/moodle 2022-09-05 13:09:09 +02:00
Srdjan
5248067884 MDL-71000 lib/dml: New query type SQL_QUERY_AUX_READONLY
For SQL_QUERY_AUX queries that can go to readonly, conditions permitting.
2022-08-23 10:39:43 +10:00
Tim Hunt
797b9fbd62 MDL-75548 misc: final Required parameter follows optional notices 2022-08-22 15:20:13 +01:00
Srdjan
bab70a42a0 MDL-74681 lib/dml: moodle_read_slave_trait: written table timestamping
Moved written table timestamping from query_start() to query_end():
We are adjusting table last written times at the end of transaction.
That does not apply to immediate database writes that are not performed
within transaction. This change is to set last written time after the query
has finished for such writes, rather than before it started. That way
long write operations cannot spill over the latency parameter.
2022-07-08 12:27:33 +10:00
Paul Holden
27f9929b52 MDL-72184 dml: implement driver methods for casting to char. 2022-06-12 19:31:34 +01:00
Brendan Heywood
66a923fe77 MDL-68164 dml: Cache postgres field type metadata in MUC 2021-09-07 19:29:31 +10:00
Srdjan
fa0eecd6bf MDL-69451 dml: use same temptables for both rw and ro database handle
moodle_read_slave_trait: when creating another handle, restore temptables
property that is clobbered by raw_connect().

Also a better condition for temptable related queries detection in
pgsql_native_moodle_database.

dml_pgsql_read_slave_test::test_temp_table(): use real db connection
if possible, otherwise skip the test.
2021-08-06 10:00:20 +10:00
Marina Glancy
badf442ec7 MDL-70965 core_dml: php8 related changes:
1. pg_connection_status(false) fails on PHP8 because the argument is not a stream
2. skipeed the unittest that expected that pg_query() does not show an error if the resource is invalid
see also MDL-71482
2021-04-28 12:42:07 +02:00
Paul Holden
b2d19cc29f MDL-52817 dml: implement driver methods for SQL group concatenation. 2021-02-19 15:31:00 +00:00
Ruslan Kabalin
536fdb0c2f MDL-70055 pgsql DML: Support large number of SQL-IN parameters.
This patch adds a solution for Postgres that allows overriding IN
statement limit of 16bit (65535 values) by wrapping items in VALUES
list. This only takes place in number of items exceeds 65535, otherwise
parent class get_in_or_equal() function is used.
2020-12-24 14:39:21 +00:00
Tim Hunt
dc72bc6ac5 MDL-54907 dml: Correct incorrect PHPdoc 2020-11-27 14:57:36 +00:00
Srdjan
46cfde3d95 MDL-19711 dml: Enable use of readonly slave database handles
Implemented with moodle_read_slave_trait

Functionality is triggered by supplying config dboption['readonly'].
See config-dist.php for more info on supported dboptions.

pgsql and mysqli drivers are using this feature. Also added support for
connection timeout for these two drivers.
2020-05-19 11:50:42 +10:00
Shamim Rezaie
b4d062c6be MDL-58584 core: remove duplications from get_columns functions 2020-02-03 15:07:15 +11:00
Matteo Scaramuccia
a8a4eac163 MDL-67414 database: Fixed PostgreSQL 12.x support 2020-01-26 09:37:17 +01:00
sam marshall
46875866c6 MDL-63315 DB: Postgres - Clear cursors in persistent connection
If there is a PHP fatal error and destructors do not run (this can
happen in out-of-memory errors, and maybe if there is an error in a
previous destructor) then Postgres cursors may be left open.

Usually this does not cause a problem because the connection is
closed anyway, but if using persistent connections, a future
request may reuse the connection with a cursor open. It then gets
errors when it tries to create a new cursor with the same name.

This change closes all cursors at the start of a persistent
connection.
2019-09-02 14:08:57 +01:00
Andrew Nicols
66fa73d84e MDL-66327 pgsql DML: Update get_records_sql to consume less memory
It's more memory efficient to use `pg_fetch_assoc` for each row than to
call `pg_fetch_all` and release memory immediately. This is because we
can treat the assoc fetch like an iterator and it only fetches the
current record into memory one at a time, whilst the all fetch fetches
all records and never unsets them. Attempting to unset them is extremely
time consuming.
2019-08-08 15:48:12 +08:00
David Monllao
c2e970779d MDL-53226 search_simpledb: Refine the patch
- Clumsy fallback only when there is no full-text search support
- Mimic solr tests
- pgsql tokenization using simple configuration
- workaround for mysql '*' search issue
- total results proper calculation
- SQL server FTS support
- Standarize dml full-text search checkings
- Upgrade note about the new dml method
- Set search_simpledb as default engine if no solr config
2018-03-20 18:49:38 +01:00
Matteo Scaramuccia
bccbd3b4eb MDL-61626 core_dml: pgsql and schema qualification. 2018-03-11 22:56:36 +01:00
sam marshall
ed00d67c99 MDL-60174 core_dml: get_recordset on Postgres eats all the RAM
On Postgres, get_recordset_sql loads all the results into memory
(within the Postgres library, which doesn't count towards the PHP
memory limit, but does count towards making your server run out of
memory) as soon as the query completes.

This commit changes the code to use cursors, which in Postgres
allow the results to be returned in smaller chunks (by default
100,000 rows).
2017-11-27 11:10:29 +00:00
Marina Glancy
109fbd5a15 MDL-46269 dml: Add casesensitive argument to sql_regex() 2017-10-16 16:53:46 +08:00
Russell Smith
d59f1d3150 MDL-55124 database: Mode connection test to after connection.
To verify the client encoding and other data is correct then
the connection must have been made first.
2016-09-22 08:40:03 +10:00
Russell Smith
f834a8f5eb MDL-55124 database: Allow option to not send options.
pgbouncer doesn't accept the PostgreSQL options command as it can't
process it.  If you are using pgBouncer you need to make sure your
database is configured to set client encoding and standard_conforming_strings
correctly on each connection.  pgBouncer can do this, as can ALTER
commands.

Also using sockets without a filename did not allow different ports to work.
This is because you either specify a filename which includes a port or
you just specify a port and libpq works out the correct socket location.
2016-09-20 19:53:01 +10:00
Russell Smith
e868574f02 MDL-48856 dmllib: Adjust performance improvements. 2016-07-22 11:52:41 +10:00
Tony Levi
73d37b20ed MDL-48856 pgsql: Performance tuning 2016-07-21 10:31:41 +10:00
Russell Smith
1b0b082a08 MDL-54947 database: Update PostgreSQL binary handling.
PostgreSQL 9.1 allows hex formating for binary which is handled better
by pg_query_params().

Getting bytea isn't required on connection, it can be used as pg_field_type()
when binary needs to be checked.
2016-06-30 12:58:53 +10:00
Simey Lameze
87a3e50192 MDL-51374 core_dml: change database layer to pass table name as parameter 2016-03-15 14:43:45 +08:00
Russell Smith
d22a01fd99 MDL-53213 caching: Faster databasemeta caching for all databases. 2016-02-28 17:03:14 +11:00
Petr Skoda
267ac1b9eb MDL-52511 dml: add support for PostgreSQL 9.5
The default definition behaviour has changed for some int values.
2016-02-10 09:45:07 +00:00
Tim Hunt
5bcfd504df MDL-45618 Fix @since PHPdoc tags. 2014-05-19 17:03:04 +01:00
Petr Škoda
96255f472c MDL-44862 always add dbport to dbsocket in pg driver
This should hopefully resolve problems when using sockets connection
to pg servers running on non-standard ports.
2014-03-30 10:06:39 +08:00
Sam Hemelryk
b2ca12e904 Merge branch 'MDL-43884-master' of git://github.com/ankitagarwal/moodle 2014-02-04 09:11:27 +13:00
Eloy Lafuente (stronk7)
6dcea23522 Merge branch 'w04_MDL-34055_m27_insertrecords' of https://github.com/skodak/moodle 2014-01-28 10:43:51 +01:00
Ankit Agarwal
aa611d661c MDL-43884 database: Fix get_tables() api for external stores 2014-01-28 16:28:57 +08:00
Petr Škoda
cc5dba8e54 MDL-34055 add bulk DB->insert_records() method
This patch was inspired by patch by Simon Coggins from Totara.
2014-01-28 10:41:43 +08:00
Petr Škoda
b4b03d38b7 MDL-43761 execute bulk sql when modifying db structure
In case of postgresql this uses transactions to allow easier recovery from broken installs.
2014-01-24 13:47:41 +08:00
Petr Skoda
3f17d7091b MDL-43761 refactor db manager code to send arrays of SQL to the dml driver 2014-01-24 13:45:07 +08:00
Dan Poltawski
c582565a06 MDL-43395 dml: warn about invalid limit params
Help developers to find bugs by throwing a DEBUG_DEVELOPER message
rather than ignoring this case completely.
2014-01-13 08:42:07 +08:00
Sam Hemelryk
37f59a68bd Merge branch 'MDL-41359-master' of https://github.com/mr-russ/moodle 2013-12-18 10:08:33 +13:00
Russell Smith
9c7dfbe6ed MDL-41359 database: no table columns is a dml_excpeiton
When insert_record is called and it doesn't find any columns
to insert data into, that is a dml_exception as you need at least
one column in the database to insert into it.  This can happen
when you have created a zero column database table, or the table
does not exist.
2013-12-07 08:56:24 +11:00
Petr Škoda
6c3ae510f1 MDL-31625 fix multiple global search-replace issues
Includes following fixes:
* support for MS SQL Server
* optional trimming of of oversized VARCHAR fields
* conversion to forms library
* full localisation
* other cleanup
2013-12-06 09:51:49 +08:00
Petr Škoda
302506e03a MDL-42493 revert nasty DB meta caching hack that was trying to work around shutdown issues
The original problem was resolved via new core shutdown handler support.
2013-10-25 21:20:06 +02:00
Petr Škoda
d79d5ac276 MDL-31501 rework user session architecture
List of changes:
 * New OOP API using PHP namespace \core\session\.
 * All handlers now update the sessions table consistently.
 * Experimental DB session support in Oracle.
 * Full support for session file handler (filesystem locking required).
 * New option for alternative session directory.
 * Official memcached session handler support.
 * Workaround for memcached version with non-functional gc.
 * Improved security - forced session id regeneration.
 * Improved compatibility with recent PHP releases.
 * Fixed borked CSS during install in debug mode.
 * Switched to file based sessions in new installs.
 * DB session setting disappears if DB does not support sessions.
 * DB session setting disappears if session handler specified in config.php.
 * Fast purging of sessions used in request only.
 * No legacy distinction -  file, database and memcached support the same functionality.
 * Session handler name included in performance info.
 * Fixed user_loggedin and user_loggedout event triggering.
 * Other minor bugfixing and improvements.
 * Fixed database session segfault if MUC disposed before $DB.

Limitations:
 * Session access time is now updated right after session start.
 * Support for $CFG->sessionlockloggedinonly was removed.
 * First request does not update userid in sessions table.
 * The timeouts may break badly if server hosting forces PHP.ini session settings.
 * The session GC is a lot slower, we do not rely on external session timeouts.
 * There cannot be any hooks triggered at the session write time.
 * File and memcached handlers do not support session lock acquire timeouts.
 * Some low level PHP session functions can not be used directly in Moodle code.
2013-09-21 13:11:56 +02:00
Petr Škoda
d11b8d0c63 MDL-40808 remove unused DML strings 2013-07-25 12:02:29 +02:00
Petr Škoda
92b00c32a7 MDL-37420 rework pg schema testing 2013-01-11 02:12:38 +01:00