Commit Graph

125 Commits

Author SHA1 Message Date
sam marshall
46875866c6 MDL-63315 DB: Postgres - Clear cursors in persistent connection
If there is a PHP fatal error and destructors do not run (this can
happen in out-of-memory errors, and maybe if there is an error in a
previous destructor) then Postgres cursors may be left open.

Usually this does not cause a problem because the connection is
closed anyway, but if using persistent connections, a future
request may reuse the connection with a cursor open. It then gets
errors when it tries to create a new cursor with the same name.

This change closes all cursors at the start of a persistent
connection.
2019-09-02 14:08:57 +01:00
Andrew Nicols
66fa73d84e MDL-66327 pgsql DML: Update get_records_sql to consume less memory
It's more memory efficient to use `pg_fetch_assoc` for each row than to
call `pg_fetch_all` and release memory immediately. This is because we
can treat the assoc fetch like an iterator and it only fetches the
current record into memory one at a time, whilst the all fetch fetches
all records and never unsets them. Attempting to unset them is extremely
time consuming.
2019-08-08 15:48:12 +08:00
David Monllao
c2e970779d MDL-53226 search_simpledb: Refine the patch
- Clumsy fallback only when there is no full-text search support
- Mimic solr tests
- pgsql tokenization using simple configuration
- workaround for mysql '*' search issue
- total results proper calculation
- SQL server FTS support
- Standarize dml full-text search checkings
- Upgrade note about the new dml method
- Set search_simpledb as default engine if no solr config
2018-03-20 18:49:38 +01:00
Matteo Scaramuccia
bccbd3b4eb MDL-61626 core_dml: pgsql and schema qualification. 2018-03-11 22:56:36 +01:00
sam marshall
ed00d67c99 MDL-60174 core_dml: get_recordset on Postgres eats all the RAM
On Postgres, get_recordset_sql loads all the results into memory
(within the Postgres library, which doesn't count towards the PHP
memory limit, but does count towards making your server run out of
memory) as soon as the query completes.

This commit changes the code to use cursors, which in Postgres
allow the results to be returned in smaller chunks (by default
100,000 rows).
2017-11-27 11:10:29 +00:00
Marina Glancy
109fbd5a15 MDL-46269 dml: Add casesensitive argument to sql_regex() 2017-10-16 16:53:46 +08:00
Russell Smith
d59f1d3150 MDL-55124 database: Mode connection test to after connection.
To verify the client encoding and other data is correct then
the connection must have been made first.
2016-09-22 08:40:03 +10:00
Russell Smith
f834a8f5eb MDL-55124 database: Allow option to not send options.
pgbouncer doesn't accept the PostgreSQL options command as it can't
process it.  If you are using pgBouncer you need to make sure your
database is configured to set client encoding and standard_conforming_strings
correctly on each connection.  pgBouncer can do this, as can ALTER
commands.

Also using sockets without a filename did not allow different ports to work.
This is because you either specify a filename which includes a port or
you just specify a port and libpq works out the correct socket location.
2016-09-20 19:53:01 +10:00
Russell Smith
e868574f02 MDL-48856 dmllib: Adjust performance improvements. 2016-07-22 11:52:41 +10:00
Tony Levi
73d37b20ed MDL-48856 pgsql: Performance tuning 2016-07-21 10:31:41 +10:00
Russell Smith
1b0b082a08 MDL-54947 database: Update PostgreSQL binary handling.
PostgreSQL 9.1 allows hex formating for binary which is handled better
by pg_query_params().

Getting bytea isn't required on connection, it can be used as pg_field_type()
when binary needs to be checked.
2016-06-30 12:58:53 +10:00
Simey Lameze
87a3e50192 MDL-51374 core_dml: change database layer to pass table name as parameter 2016-03-15 14:43:45 +08:00
Russell Smith
d22a01fd99 MDL-53213 caching: Faster databasemeta caching for all databases. 2016-02-28 17:03:14 +11:00
Petr Skoda
267ac1b9eb MDL-52511 dml: add support for PostgreSQL 9.5
The default definition behaviour has changed for some int values.
2016-02-10 09:45:07 +00:00
Tim Hunt
5bcfd504df MDL-45618 Fix @since PHPdoc tags. 2014-05-19 17:03:04 +01:00
Petr Škoda
96255f472c MDL-44862 always add dbport to dbsocket in pg driver
This should hopefully resolve problems when using sockets connection
to pg servers running on non-standard ports.
2014-03-30 10:06:39 +08:00
Sam Hemelryk
b2ca12e904 Merge branch 'MDL-43884-master' of git://github.com/ankitagarwal/moodle 2014-02-04 09:11:27 +13:00
Eloy Lafuente (stronk7)
6dcea23522 Merge branch 'w04_MDL-34055_m27_insertrecords' of https://github.com/skodak/moodle 2014-01-28 10:43:51 +01:00
Ankit Agarwal
aa611d661c MDL-43884 database: Fix get_tables() api for external stores 2014-01-28 16:28:57 +08:00
Petr Škoda
cc5dba8e54 MDL-34055 add bulk DB->insert_records() method
This patch was inspired by patch by Simon Coggins from Totara.
2014-01-28 10:41:43 +08:00
Petr Škoda
b4b03d38b7 MDL-43761 execute bulk sql when modifying db structure
In case of postgresql this uses transactions to allow easier recovery from broken installs.
2014-01-24 13:47:41 +08:00
Petr Skoda
3f17d7091b MDL-43761 refactor db manager code to send arrays of SQL to the dml driver 2014-01-24 13:45:07 +08:00
Dan Poltawski
c582565a06 MDL-43395 dml: warn about invalid limit params
Help developers to find bugs by throwing a DEBUG_DEVELOPER message
rather than ignoring this case completely.
2014-01-13 08:42:07 +08:00
Sam Hemelryk
37f59a68bd Merge branch 'MDL-41359-master' of https://github.com/mr-russ/moodle 2013-12-18 10:08:33 +13:00
Russell Smith
9c7dfbe6ed MDL-41359 database: no table columns is a dml_excpeiton
When insert_record is called and it doesn't find any columns
to insert data into, that is a dml_exception as you need at least
one column in the database to insert into it.  This can happen
when you have created a zero column database table, or the table
does not exist.
2013-12-07 08:56:24 +11:00
Petr Škoda
6c3ae510f1 MDL-31625 fix multiple global search-replace issues
Includes following fixes:
* support for MS SQL Server
* optional trimming of of oversized VARCHAR fields
* conversion to forms library
* full localisation
* other cleanup
2013-12-06 09:51:49 +08:00
Petr Škoda
302506e03a MDL-42493 revert nasty DB meta caching hack that was trying to work around shutdown issues
The original problem was resolved via new core shutdown handler support.
2013-10-25 21:20:06 +02:00
Petr Škoda
d79d5ac276 MDL-31501 rework user session architecture
List of changes:
 * New OOP API using PHP namespace \core\session\.
 * All handlers now update the sessions table consistently.
 * Experimental DB session support in Oracle.
 * Full support for session file handler (filesystem locking required).
 * New option for alternative session directory.
 * Official memcached session handler support.
 * Workaround for memcached version with non-functional gc.
 * Improved security - forced session id regeneration.
 * Improved compatibility with recent PHP releases.
 * Fixed borked CSS during install in debug mode.
 * Switched to file based sessions in new installs.
 * DB session setting disappears if DB does not support sessions.
 * DB session setting disappears if session handler specified in config.php.
 * Fast purging of sessions used in request only.
 * No legacy distinction -  file, database and memcached support the same functionality.
 * Session handler name included in performance info.
 * Fixed user_loggedin and user_loggedout event triggering.
 * Other minor bugfixing and improvements.
 * Fixed database session segfault if MUC disposed before $DB.

Limitations:
 * Session access time is now updated right after session start.
 * Support for $CFG->sessionlockloggedinonly was removed.
 * First request does not update userid in sessions table.
 * The timeouts may break badly if server hosting forces PHP.ini session settings.
 * The session GC is a lot slower, we do not rely on external session timeouts.
 * There cannot be any hooks triggered at the session write time.
 * File and memcached handlers do not support session lock acquire timeouts.
 * Some low level PHP session functions can not be used directly in Moodle code.
2013-09-21 13:11:56 +02:00
Petr Škoda
d11b8d0c63 MDL-40808 remove unused DML strings 2013-07-25 12:02:29 +02:00
Petr Škoda
92b00c32a7 MDL-37420 rework pg schema testing 2013-01-11 02:12:38 +01:00
Petr Škoda
6ad8d762da MDL-37420 add support for more PostgreSQL schemas 2013-01-08 17:12:31 +01:00
Petr Škoda
46a86dbbc2 MDL-36211 do not lock sessions for guests and not-logged-in users 2012-11-17 10:42:47 +01:00
Sam Hemelryk
9381983e5b MDL-25290 conversion: Added db column cache to pgsql + mysql 2012-10-19 08:08:59 +08:00
Petr Škoda
3c17c85ae6 MDL-35691 fix max length calculation in pg, mysql and our bigint checker 2012-10-06 10:07:48 +02:00
Petr Škoda
bedd0982e8 MDL-35506 improve pg transaction snapshotting perf 2012-09-26 10:39:28 +02:00
Petr Škoda
8005d4eedf MDL-35506 add pgsql workaround for automatic transaction rollback on any SQL error 2012-09-21 10:52:53 +02:00
Petr Skoda
bd991d03cf MDL-33018 add general index type hints and use PostgreSQL varchar_pattern_ops index type for context.path
This significantly improves performance of accesslib queries,
credit for the discovery of this solution goes to Andrew Masterton from OU.
2012-07-06 08:19:39 +02:00
Tim Hunt
5415d91e20 MDL-34145 postgres driver: fix get_tables perf on <= 9.0
The query of the information_schema view seems to be painfully slow on
older versions of postgres (2000ms instead of 20ms). Therefore, it is
worth detecting that case, and using a more postgres specific feedback.
This is particularly important for 2.3+ unit tests.

Thanks to Petr Skoda for his help with this.
2012-07-02 16:19:00 +01:00
Petr Skoda
00902cd974 MDL-32003 fix phpdocs and use __DIR__ for includes in dml layer
DML layer is not supposed to use $CFG,
it is better to rely on __DIR__ in self-contained libs…
2012-06-17 22:56:28 +02:00
Petr Skoda
ed63f7f883 MDL-32907 prevent sql_like() backslash quoting problems in PostgreSQL 2012-05-12 16:40:11 +02:00
Petr Skoda
54c5b1ad35 MDL32392 delete unused is_min_version() in some DB drivers 2012-04-15 19:40:07 +02:00
Eloy Lafuente (stronk7)
97d00c9f51 MDL-32368 pgsql: bytea_output only available in 9.0 and upwards 2012-04-10 21:32:56 +02:00
Eloy Lafuente (stronk7)
7b74fb7071 MDL-32392 cleanup: adding @todo about unused is_min_version() method 2012-04-10 21:26:00 +02:00
Petr Skoda
d2b357b923 MDL-32368 fix pg to always use old bytea encoding 2012-04-10 15:25:20 +02:00
Petr Skoda
62cd8cd9f5 MDL-32029 fix get_tables() when prefix empty in external database 2012-03-24 11:36:01 +01:00
Dan Poltawski
d68c62fe12 Merge branch 'w12_MDL-29515_m23_deprdb' of git://github.com/skodak/moodle 2012-03-21 09:25:47 +08:00
Petr Skoda
de640a2d3f MDL-29515 remove deprecated $DB->sql_ilike() and fix debug messages 2012-03-17 19:20:39 +01:00
Petr Skoda
e618cdf3f6 MDL-29894 forbid objects in DML parameters
Objects with __toString we never fully supported as parameters in DML layer, this finally brings consistent behaviour.
2012-03-17 18:44:12 +01:00
Petr Skoda
a681b6c0a9 MDL-31689 fix moodle_database:execute() description 2012-03-07 12:36:40 +01:00
Aparup Banerjee
dafa20e85c MDL-30972 Documentation : improved @see as well as added appropriate uses of inline @link. 2012-02-16 10:29:45 +08:00