Skip to content

6 Architecture

Source chapter 3.4 for PostgreSQL 11.
GASource pages: PDF 36-43Original PDF on GitHub

PDF page 36

3.4.1 Modified catalogs

The following catalogs have been changed.

Catalog nameAdded columnData Type Description
pg_aggregateaggfinalmodifycharWhether the aggfinalfn function changes the value
aggmfinalmodifycharWhether the aggmfinalfn function changes the value
pg_attributeatthasmissingboolHave a default value that has not updated the page
attmissingvalanyarrayDefault value not updating page
pg_classrelrewriteoidOID when the new relation is created during DDL execution
pg_constraintconparentidoidParent partition constraint OID
pg_indexindnkeyattssmallintNumber of key columns
pg_partitioned_tablepartdefidoidOID of the default partition
pg_procprokindcharObject kind f: function p: procedure a: aggregate function w: window function
pg_publicationpubtruncatebooleanTRUNCATE propagation
pg_stat_wal_receiversender_hosttextConnection destination hostname
sender_portintegerConnection destination port number
information_schema.table_constraintsenforcedyes_or_noReserved for future use
Table 11 System catalogs with columns added

PDF page 37

Catalog nameDeleted columnDescription
pg_classrelhaspkeyHave primary key
pg_procproisaggIs aggregate function
proiswindowIs Window function
Table 12 Catalogs with columns deleted
Catalog nameColumn nameDescription
triggersaction_orderTrigger execution order
action reference new table___“NEW” transition table name
action reference old table___“OLD” transition table name
tablestable_typeFOREIGN stored for the foreign table (previously FOREIGN TABLE stored)
Table 13 Value is now stored information_schema catalog of the schema
pg_stat_activity catalog

Backend_type column and process name are now synchronized. The value of the backend_type column of the replication launcher process was “background worker” in PostgreSQL 10, but in PostgreSQL 11 it is “logical replication launcher”.

Example 42 Refer pg_stat_activity catalog
postgres=# SELECT pid,wait_event, backend_type FROM pg_stat_activity ;
  pid  |     wait_event      |         backend_type
-------+---------------------+------------------------------
 17099 | LogicalLauncherMain | logical replication launcher
 17097 | AutoVacuumMain      | autovacuum launcher
 17101 |                     | client backend
 17095 | BgWriterHibernate   | background writer
 17094 | CheckpointerMain    | checkpointer
 17096 | WalWriterMain       | walwriter
(6 rows)
pg_attribute catalog

In PostgreSQL 11, columns with the DEFAULT value and NOT NULL constraint can now be added without updating the actual tuples. Along with this, the columns for this feature have been added to the pg_attribute catalog.

PDF page 38

Example 43 Refer pg_attribute catalog
postgres=> ALTER TABLE cols1 ADD COLUMN c3 INT NOT NULL DEFAULT 10 ;
ALTER TABLE
postgres=> SELECT atthasmissing, attmissingval FROM pg_attribute
WHERE attname='c3' ;
 atthasmissing | attmissingval
---------------+---------------
 t             | {10}
(1 row)

3.4.2 Added ROLE

The following roles were added. These roles are mainly used to allow general users to execute COPY statements and execute the file_fdw Contrib module.

Role nameDescription
pg_execute_server_programExecute programs on the server
pg_read_server_filesRead files on the server
pg_write_server_filesWrite files on the server
Table 14 Added ROLE
Example 44 pg_read_server_files role
postgres=# GRANT pg_read_server_files TO user1 ;
GRANT ROLE
postgres(user1)=> COPY copy1 FROM '/tmp/copy1.csv' CSV ;
COPY 2000

3.4.3 LLVM integration

PostgreSQL 11 supports JIT compilation using LLVM (https://llvm.org/) to speed up long-running SQL statements caused by processor bottlenecks. SQL statements estimated to exceed more than a certain amount of cost are compiled beforehand and then executed.

Installation

In order to use LLVM, it is necessary to specify the option –with-llvm of “configure” command at

PDF page 39

the time of installation. When executing “configure” command, “llvm-config” command and “clang” command must be included in command execution path (or specified in environment variable LLVM_CONFIG and environment variable CLANG).

The behavior of JIT compilation

For SQL statements whose total execution cost exceeds the parameter jit_above_cost (default value 100000), the JIT compilation feature by LLVM runs. If this parameter is specified as “-1” or the parameter jit is set to “off”, the JIT feature will be disabled. The behavior of JIT compilation processing is changed by inlining (parameter jit_inline_above_cost) and optimization (parameter jit_optimize_above_cost).

Execution plan

By executing EXPLAIN statements for SQL statement using the JIT compilation feature, information starting from “JIT:” is displayed.

Example 45 Execution plan of SQL using JIT compilation features
postgres=> EXPLAIN ANALYZE SELECT COUNT(*) FROM jit1 ;
                                    QUERY PLAN
------------------------------------------------------------------------
Aggregate  (cost=179053.25..179053.26 rows=1 width=8)
(actual time=2680.558..2680.559 rows=1 loops=1)
   ->  Seq Scan on jit1  (cost=0.00..154053.60 rows=9999860 width=0)
(actual time=0.022..1424.095 rows=10000000 loops=1)
 Planning Time: 0.024 ms
 JIT:
   Functions: 2
   Generation Time: 1.505 ms
   Inlining: false
   Inlining Time: 0.000 ms
   Optimization: false
   Optimization Time: 0.594 ms
   Emission Time: 8.688 ms
 Execution Time: 2682.166 ms
(12 rows)
In order to enable the JIT compile feature, it is necessary to set the parameter "jit" to "on" in advance.

PDF page 40

3.4.4 Predicate locking for GIN / GiST / HASH index

Predicate locks are now available for GIN index, GiST index, HASH index. This feature reduces the lock range, improving the concurrency of SQL statements with multiple sessions. The following example is the result of verification using the HASH index. In PostgreSQL 10, the lock range is the relation, but in PostgreSQL 11 it is found to be page.

Example 46 Test case
postgres=> CREATE TABLE lock1(c1 INT, c2 VARCHAR(10)) ;
CREATE TABLE
postgres=> CREATE INDEX idx1_lock1 ON lock1 USING hash(c1) ;
CREATE INDEX
postgres=> INSERT INTO lock1 VALUES (generate_series(1, 100000),
'data1') ;
INSERT 0 100000
postgres=> BEGIN ISOLATION LEVEL SERIALIZABLE ;
BEGIN
postgres=> SELECT * FROM lock1 WHERE c1=10000 FOR UPDATE ;
  c1   |  c2
-------+-------
 10000 | data1
(1 row)
Example 47 Lock status on PostgreSQL 10
postgres=> SELECT locktype, relation::regclass, mode FROM pg_locks ;
   locktype    |  relation  |      mode
---------------+------------+-----------------
 relation      | pg_locks   | AccessShareLock
 relation      | idx1_lock1 | AccessShareLock
 relation      | lock1      | RowShareLock
 virtualxid    |            | ExclusiveLock
 transactionid |            | ExclusiveLock
 tuple         | lock1      | SIReadLock
 relation      | idx1_lock1 | SIReadLock
(7 rows)

PDF page 41

Example 48 Lock status on PostgreSQL 11
postgres=> SELECT locktype, relation::regclass, mode FROM pg_locks ;
   locktype    |  relation  |      mode
---------------+------------+-----------------
 relation      | pg_locks   | AccessShareLock
 relation      | idx1_lock1 | AccessShareLock
 relation      | lock1      | RowShareLock
 virtualxid    |            | ExclusiveLock
 transactionid |            | ExclusiveLock
 page          | idx1_lock1 | SIReadLock
 tuple         | lock1      | SIReadLock
(7 rows)

3.4.5 Enhanced LDAP authentication

“ldapsearchfilter” attribute has been added to the LDAP authentication parameter described in the pg_hba.conf file. LDAP server can be searched more flexibe than “ldapsearchattribute” attribute.

3.4.6 Extended Query timeout

The timeout of conventional Extended Queries was determined by the time until SYNC messages were sent after sending multiple SQL statements. PostgreSQL 11 now takes into account the execution time of individual SQL statements. The example below is the definition file of pgproto (https://github.com/tatsuo-ishii/pgproto). In PostgreSQL 10 timeout occurred even if “SET statement_timeout = 4s” was set.

PDF page 42

Example 49 pgproto definition file
# Test case for statement timeout patch.
'Q'
"SET statement_timeout = '4s'"
'Y'
# Execute statement which takes 3 seconds.
'P'
"S1"
"SELECT pg_sleep(3)" 0
'B'
""
"S1"
0
0
0
PostgreSQL 11
'E'
""
0
Timeout check
'C'
'S'
"S1"
# Execute statement which takes 2 seconds.
'P'
"S2"
"SELECT pg_sleep(2)" 0
PostgreSQL 10
'B'
""
"S2"
0
0
0
PostgreSQL 11
Timeout check
'E'
""
0
Timeout check
'C'
'S'
"S2"
# Issue Sync message and terminate
'S'
'Y'
'X'

3.4.7 Change of backup label file

A timeline ID is added to the backup_label file created when online backup is executed.

PDF page 43

Example 50 backup_label file
postgres=# SELECT pg_start_backup(now()::text) ;
pg_start_backup
-----------------
 0/2000060
(1 row)
postgres-# \! cat data/backup_label
START WAL LOCATION: 0/6A000028 (file 00000001000000000000006A)
CHECKPOINT LOCATION: 0/6A000098
BACKUP METHOD: pg_start_backup
BACKUP FROM: master
START TIME: 2018-10-22 08:59:10 JST
LABEL: 2018-10-22 08:59:10.132746+09
START TIMELINE: 1
<- Added line

3.4.8 Using Huge Pages under Windows environment

The “Lock Pages In Memory” setting is now available in the Microsoft Windows environment. When parameter huge_pages is set to “try” or “on”, shared memory will be reserved in a continuous area. Internally, PAGE_READWRITE, SEC_LARGE_PAGES, and SEC_COMMIT are specified in Windows API CreateFileMapping. Previously only PAGE_READWRITE was specified.

3.4.9 Remove secondary checkpoint information

PostgreSQL 10 saved the last two checkpoint information, but now only the latest checkpoint information is saved.

3.4.10 Error code list

  The {INSTALL_DIR}/share/errcodes.txt file has been added. This file contains error codes, levels,

macro names etc. of PostgreSQL, PL/pgSQL, and PL/Tcl.