PostgreSQL 12 New Features
PostgreSQL 12 New Features
WITH EXAMPLES · ENGLISH SOURCE EDITION
Explore the features, SQL examples, and compatibility changes documented by Noriyoshi Shinoda.
Contents
Reference indexes
The opening chapter includes the first numbered table, Table 1. The indexes below collect the book’s other native numbered components.
Tables
- Table 1 — Version
- Table 2 — Examples notation
- Table 3 — Syntax rules
- Table 4 — Added system catalogs
- Table 5 — Added information_schema catalog
- Table 6 — System catalogs with columns added
- Table 7 — System catalogs with columns deleted
- Table 8 — System catalogs with column modified
- Table 9 — pg_stat_progress_cluster catalog
- Table 10 — pg_stat_progress_create_index catalog
- Table 11 — pg_stat_gssapi catalog
- Table 12 — pg_statistic_ext_data catalog
- Table 13 — Modified parameters
- Table 14 — Modified wait events
- Table 15 — Environment variable PG_COLORS settings
- Table 16 — JSON/SQL functions
- Table 17 — Hyperbolic functions
- Table 18 — Added parameters
- Table 19 — shared_memory_type parameter value
- Table 20 — Changed parameters
- Table 21 — Parameters that default value has been changed
- Table 22 — Added options
- Table 23 — Added long-name option
Examples
- Example 1 — CREATE TABLE statement in PostgreSQL 12
- Example 2 — pg_tablespace catalog in PostgreSQL 12
- Example 3 — FOREIGN KEY constraint name in PostgreSQL 12
- Example 4 — to_date function in PostgreSQL 11
- Example 5 — to_date function in PostgreSQL 12
- Example 6 — substring function in PostgreSQL 11
- Example 7 — substring function in PostgreSQL 12
- Example 8 — Refer pg_stat_progress_create_index catalog
- Example 9 — Refer pg_stat_replication catalog
- Example 10 — Refer pg_stat_database catalog
- Example 11 — Deploy recovery.conf file and start instance
- Example 12 — Instance startup log
- Example 13 — Output of pg_controldata command
- Example 14 — Execute query
- Example 15 — Execution plan on PostgreSQL 11
- Example 16 — Execution plan of PostgreSQL 12
- Example 17 — jsonb type and GIN index
- Example 18 — DECLARE bytea
- Example 19 — PREPARE AS statement
- Example 20 — Refer pg_am catalog
- Example 21 — CREATE TABLE statement
- Example 22 — \d+ command output
- Example 23 — \d+ command output
- Example 24 — \dA command output
- Example 25 — Text search configurations
- Example 26 — Alter system catalog
- Example 27 — ALTER TYPE ADD VALUE statement in the transaction block
- Example 28 — COMMIT AND CHAIN
- Example 29 — Transaction attribute
- Example 30 — COPY FROM WHERE statement
- Example 31 — \copy WHERE command
- Example 32 — COPY FREEZE statement on PostgreSQL 12
- Example 33 — Covering index for GiST index
- Example 34 — Create MCV statistics
- Example 35 — GENERATED column definition
- Example 36 — Update directly for GENERATED column
- Example 37 — Stored the GENERATED column value
- Example 38 — GENERATED column information
- Example 39 — Restriction of GENERATED column
- Example 40 — Partitioned Table Creation and TABLESPACE Clause
- Example 41 — FOR VALUES clause of partition table
- Example 42 — Refer to partition table as reference table
- Example 43 — Suppressing VACUUM on indexes
- Example 44 — Truncate empty block at the end of the table
- Example 45 — EXPLAIN (SETTINGS ON)
- Example 46 — REINDEX CONCURRENTLY statement
- Example 47 — reindexdb command
- Example 48 — strict_multi_assignment setting
- Example 49 — too_many_rows setting
- Example 50 — Lock table
- Example 51 — Skip VACUUM for locked table
- Example 52 — Operation specification by ON/OFF
- Example 53 — Suppress VACUUM for index
- Example 54 — Suppress truncation processing
- Example 55 — WITH NOT MATERIALIZED
- Example 56 — WITH MATERIALIZED
- Example 57 — jsonb_path_query_array function
- Example 58 — pg_partition_tree function
- Example 59 — pg_partition_root function
- Example 60 — pg_partition_ancestors function
- Example 61 — pg_promote function
- Example 62 — pg_ls_tmpdir function
- Example 63 — pg_ls_archive_statusdir function
- Example 64 — date_trunc function with timezone
- Example 65 — Hyperbolic functions
- Example 66 — Copy replication slot
- Example 67 — ssl_library parameter
- Example 68 — Execution plan changes with each execution of EXECUTE statement
- Example 69 — Execution plan is cached
- Example 70 — Setting value force_generic_plan
- Example 71 — default_with_oids parameter
- Example 72 — wal_sender_timeout parameter
- Example 74 — PostgreSQL 11 log (part)
- Example 75 — PostgreSQL 12 log (part)
- Example 76 — wal_level value check
- Example 77 — -R parameter of pg_basebackup command
- Example 78 — Usage of pg_checksums command
- Example 79 — Integrity check
- Example 80 — Enable integrity check
- Example 81 — Abend an instance and enable integrity
- Example 82 — Display command execution status
- Example 83 — pg_ctl logrotate command
- Example 84 — --on-conflict-do-nothing option
- Example 85 — -rows-per-insert option
- Example 86 — --exclude-database option
- Example 87 — Additional comments
- Example 88 — pg_rewind command
- Example 89 — pg_upgrade command
- Example 90 — \pset format csv command
- Example 91 — \d command
- Example 92 — \d command
- Example 93 — \z command
- Example 94 — \conninfo command
- Example 95 — \set VERBOSITY command
- Example 96 — \dP command
- Example 97 — \h command
- Example 98 — Execute against old version
- Example 99 — log_level parameter
- Example 100 — log_settings parameter
- Example 101 — JIT compilation information log
- Example 102 — citext_hash_extended function
- Example 103 — hstore_hash_extended function
- Example 104 — pg_stat_statements_reset function
- Example 105 — options parameter
Figures
Source edition
The source chapter order, wording, examples, and publication credits are retained. PDF page markers link the reading edition to the original document. Source identity and coverage · About the collection
1 About This Document
PDF page 5
1.1 Purpose
The purpose of this document is to provide information about the major new features of PostgreSQL 12.
1.2 Audience
This document is written for engineers who already have knowledge of PostgreSQL, such as installation, basic management, etc.
1.3 Scope
This document describes the major difference between PostgreSQL 11 (11.5) and PostgreSQL 12 (12.0). As a general rule, this document examines the features of behavior change. This document does not describe and verify all new features. In particular, the following new features are not included.
Bug fix
Performance improvement by changing internal behavior
Improvement of regression test
Operability improvement by psql command tab input
Improvement of pgbench command
Improvement of documentation, modify typo in the source code
Refactoring without a change in behavior
1.4 Software Version
The contents of this document have been verified for the following versions and platforms.
| Software | Version |
|---|---|
| PostgreSQL | PostgreSQL 11.5 (for comparison) PostgreSQL 12 (12.0) (Sep 30, 2019 20:10:49) |
| Operating System | Red Hat Enterprise Linux 7 Update 5 (x86-64) |
| Configure option | –with-llvm –with-openssl –with-perl |
PDF page 6
1.5 Question, comment and Responsibility
The contents of this document are not an official opinion of Hewlett Packard Enterprise Japan Co, Ltd. The author and affiliation company do not take any responsibility for the problem caused by the mistake of contents. If you have any comments for this document, please contact Noriyoshi Shinoda (noriyoshi.shinoda@hpe.com) Hewlett Packard Enterprise Japan Co, Ltd.
1.6 Notation
This document contains examples of the execution of the command or SQL statement. Execution examples are described according to the following rules:
| Notation | Description |
|---|---|
| # | Shell prompt for Linux root user. |
| $ | Shell prompt for Linux general user. |
| Bold | The user input string. |
| postgres=# | psql command prompt for PostgreSQL administrator. |
| postgres=> | psql command prompt for PostgreSQL general user. |
| Underline | Important output items. |
The syntax is described in the following rules:
| Notation | Description |
|---|---|
| Italic | Replaced by the name of the object which users use, or the other syntax. |
| [ ] | Indicate that it can be omitted. |
| { A | B } | Indicate that it is possible to select A or B. |
| … | General syntax, it is the same as the previous version. |
2 New Features Summary
PDF page 7
More than 180 new features have been added to PostgreSQL 12. Here are some typical new features and benefits. This version focuses to enhance various new features added in PostgreSQL 11.
2.1 Improve analytic query performance
The following features have been added that can be applied to large scale environments:
Enhancement of Parallel Query
The scope to which parallel queries are applied has been expanded. Even when the transaction isolation level is SERIALIZABLE, parallel queries may be performed.
Enhancement of Partition Table
It is now possible to specify computed values instead of fixed values as partition key values. Partition tables can now be used as foreign key references.
2.2 Improve reliability
In PostgreSQL 12 the checking tool of integrity has been expanded.
pg_checksums command
The pg_verify_checksums command added in PostgreSQL 11 has been changed to the pg_checksums command. In addition to checking the integrity, it is now possible to change the enable/disable of the checksum function with the command.
2.3 Improve maintenancibility
The following features that can improve the operability has been added.
Abolition of recovery.conf file
The recovery.conf file, used for streaming replication standby instances and recovery, have been consolidated into the postgresql.conf file.
Enhancement of REINDEX statement
The CONCURRENTLY clause was added to the REINDEX statement, and the lock range when rebuilding the index became very small.
PDF page 8
Enhancement of monitoring features
Catalogs have been added that allow you to check the execution status of CLUSTER statement, VACUUM FULL statement and CREATE INDEX statement in real time.
Increase in waiting for event type
Several wait events have been added. It can be checked in the pg_stat_activity catalog.
pg_promote function
A function pg_promote has been provided to perform promotion from the standby instance to the primary instance.
2.4 Preparing for future new features
PostgreSQL 12 is preparing for the features to be provided in future versions.
Multiple storage engine
The basic specification of PLUGGABLE STORAGE ENGINE has been decided that can use multiple storage engines simultaneously. It is expected that the corresponding such as zheap is carried out in the future.
64-bit transaction ID
APIs that obtain 64-bit transaction IDs can now be used.
PDF page 9
2.5 Incompatibility
PostgreSQL 12 has the following specifications changed from PostgreSQL 11.
2.5.1 recovery.conf file
Recovery.conf files created in database recovery and standby instances in a streaming replication environment have been discontinued. Various parameters in the recovery.conf file have been integrated into the postgresql.conf file. Details are explained in “3.1.2. Deprecation of recovery.conf file”.
2.5.2 Pg_checksums command
The pg_verify_checksums command added in PostgreSQL 11 has been renamed to pg_checksums and the new feature has been added.
2.5.3 WITH OIDS clause deleted
The WITH OIDS clause of the CREATE TABLE statement is prohibited, and a table with OID cannot be created. Along with this change, the oid column in the system catalog is no longer a hidden column. The WITHOUT OIDS clause can continue to be used. Along with this change, the default_with_oids parameter cannot be changed to “on”. With this specification change, the –oids option (-o option) has been removed from the pg_dump command.
PDF page 10
2.5.4 Timetravel Contrib module
The Contrib module timetravel has been removed.
2.5.5 Removed data type
The datatypes abstime, reltime, and tinterval have been removed. As a result, the data type of column valutil in the system catalog pg_shadow has been changed to “timestamp with time zone”.
2.5.6 COPY FREEZE for the partitioned table
The COPY FREEZE statement for partitioned tables has been prohibited. This specification has also been backported to PostgreSQL 11.2 and later.
2.5.7 Change foreign key constraint name
Auto-generated foreign key names are now generated including all the columns contained in the foreign key.
PDF page 11
2.5.8 Functions
The specifications of the following functions have been changed.
Current_schema function and current_schemas function
Current_schema and current_schemas functions were set to be unsafe for the parallel query (Parallel Unsafe).
To_timestamp function and to_date function
Unnecessary space in the template is now ignored.
PDF page 12
Substring function
The pattern search specification in the escape string has been changed.
PDF page 13
3 Architecture
PDF page 13
3.1.1 Modified catalogs
The following catalogs have been changed.
| Catalog name | Description |
|---|---|
| pg_stat_progress_cluster | Traces the execution status of CLUSTER. |
| pg_stat_progress_create_index | Traces the execution status of CREATE INEDX. |
| pg_stat_gssapi | Showing information about GSSAPI usage. |
| pg_statistic_ext_data | Holds data for extended planner statistic |
| Catalog name | Description |
|---|---|
| column_column_usage | Column information that depends on a specific column (stores information on a table with generated columns). |
| Catalog name | Added column | Data type | Description |
|---|---|---|---|
| *1 | oid | oid | Change attribute to regular column. |
| pg_attribute | attgenerated | char | ’s’ for generated column. |
| pg_collation | collisdeterministic | boolean | Is the collation deterministic? |
| pg_statistic | stacol[1-5] | oid | Collation information. |
| pg_stat_database | checksum_failures | bigint | Number of blocks for which a checksum error was detected. |
| checksum_last_failure | timestamp with time zone | Last timestamp for which checksum error was detected. | |
| pg_stat_replication | reply_time | timestamp with time zone | Message time from standby instance. |
| pg_stat_ssl | client_serial | numeric | Client certificate serial number. |
| issuer_dn | text | Client certificate issuer DN. |
PDF page 14
*1 There are many corresponding catalogs.
| Catalog name | Deleted column | Description |
|---|---|---|
| pg_attrdef | adsrc | A human-readable representation of the default value. |
| pg_class | relhasoids | Whether the table is WITH OIDS specified. |
| pg_constraint | consrc | A human-readable representation of the expression. |
| pg_statistic_ext | stxndistinct | Move to pg_statistics_ext_data |
| stxdependencies | Move to pg_statistics_ext_data |
| Catalog name | Modified Column | Description |
|---|---|---|
| pg_shadow | valuntil | Data type changed to timestamp with time zone. |
| pg_stat_ssl | client_dn | Column name changed from ’lientdn'. |
Among the modified system catalogs, the details of the major catalogs are described below.
Pg_stat_progress_cluster catalog
The pg_stat_progress_cluster catalog can trace the execution status of CLUSTER statements or VACUUM FULL statements.
| Column name | Data type | Description |
|---|---|---|
| pid | integer | Process ID of backend. |
| datid | oid | OID of the database to which this backend is connected. |
| datname | name | Name of the database to which this backend is connected. |
| relid | oid | OID of the table being clustered. |
| command | text | The command that is running. |
| phase | text | Current processing phase. |
| cluster_index_relid | oid | OID of the index being used. |
| heap_tuples_scanned | bigint | Number of heap tuples scanned. |
| heap_tuples_written | bigint | Number of heap tuples written. |
| heap_blks_total | bigint | Total number of heap blocks in the table. |
| heap_blks_scanned | bigint | Number of heap blocks scanned. |
| index_rebuild_count | bigint | Number of indexes rebuilt. |
PDF page 15
Pg_stat_progress_create_index catalog
The pg_stat_progress_create_index catalog can trace the execution status of CREATE INDEX statement. The status is also reflected when the REINDEX statement is executed.
| Column name | Data type | Description |
|---|---|---|
| pid | integer | Process ID of backend. |
| datid | oid | OID of the database to which this backend is connected. |
| datname | name | Name of the database to which this backend is connected. |
| relid | oid | OID of the table on which the index is being created. |
| index_relid | oid | OID of the index. |
| command | text | Executed SQL statement name. |
| phase | text | Current processing phase of index creation. |
| lockers_total | bigint | Total number of lockers to wait for. |
| lockers_done | bigint | Total number of lockers already waited for. |
| current_locker_pid | bigint | Process ID of the locker currently being waited for. |
| blocks_total | bigint | Total number of the blocks to be processed. |
| blocks_done | bigint | Total number of the blocks already processed. |
| tuples_total | bigint | Total number of tuples to be processed. |
| tuples_done | bigint | Number of tuples already processed. |
| partitions_total | bigint | Total number of partitions. |
| partitions_done | bigint | Number of partitions on which the index has been completed. |
PDF page 16
Pg_stat_replication catalog
The reply_time column has been added to the pg_stat_replication catalog. This column stores the sending time of the last reply message received from the standby instance. This column can only be displayed by the SUPERUSER attribute or the holder of the pg_monitor role.
PDF page 17
Pg_indexes catalog
The pg_indexes catalog now includes partitioned indexes. Specifically, the relkind column in the pg_class catalog now includes an index of ‘I’.
Pg_stat_database catalog
The checksum_failures column and checksum_last_failure column in the pg_stat_database catalog are now updated when a block checksum error is detected. The value of checksum_failures column is updated each time when a checksum error is detected, so it does not indicate the actual number of blocks corrupted.
PDF page 18
Pg_stat_gssapi catalog
The pg_stat_gssapi view showing information about GSSAPI usage on this connection. It can be joined to pg_stat_activity or pg_stat_replication on the pid column to get more details about the connection.
| Column name | Data type | Description |
|---|---|---|
| pid | integer | Process ID of backend. |
| gss_authenticated | boolean | True if GSSAPI authentication was used. |
| principal | text | Principal used to authenticate this connection. |
| encrypted | boolean | True if GSSAPI encryption is in use. |
Pg_statistic_ext_data catalog
This catalog holds data for extended planner statistics defined in pg_statistic_ext.
PDF page 19
| Column name | Data type | Description |
|---|---|---|
| stxoid | oid | Reference of pg_statistic_ext catalog |
| stxdndistinct | pg_ndistinct | N-distinct counts |
| stxddependencies | pg_dependencies | Functional dependency statistics |
| stxdmcv | pg_mcv_list | MCV (most-common values) list statistics |
3.1.2 Abolition of recovery.conf file
The recovery.conf file used to build standby instances in a streaming replication environment, and for recovery from backup has been obsolete. Various settings in the recovery.conf file have been integrated into the postgresql.conf file. Some parameters have been changed.
| recovery.conf | postgresql.conf | Note |
|---|---|---|
| standby_modetrigger_file | promote_trigger_file | Discontinued. |
In case of recovery, create a recovery.signal file in the database cluster and start the instance. This file is deleted when the recovery process is complete. In a streaming replication environment, create a standby.signal file in the database cluster on slave instances. This file is deleted when promoted to primary. If the recovery.conf file exists in the database cluster, instance startup fails.
PDF page 20
3.1.3 Instance startup log
The version number is now output to the log at instance startup. If the parameter logging_collector is specified as “on”, it is output to the log file.
PDF page 21
3.1.4 Maximum number of connections
The number of connections for replication is now obtained using the max_wal_senders parameter without depending on the max_connections parameter. As a result of this modification, information of max_wal_senders has been added to the output of the pg_controldata command.
3.1.5 Parallel query enhancement
Parallel queries now work even if the transaction isolation level of the session is SERIALIZABLE. The following example uses the auto_explain module to output the execution plan of the SQL statement executed in the transaction of the SERIALIZABLE isolation level.
PDF page 22
PDF page 23
3.1.6 Jsonb type and GIN index
The “jsonb @@ jsonpath” and “jsonb @? jsonpath” operators have been added to the GIN index created for the jsonb type. The operator class of GIN index can be used either jsonb_ops or json_path_ops. The following is the execution plan of the SELECT statement described in the manual.
3.1.7 Wait Event
The following changes were made to the wait event output to the wait_event column of the pg_stat_activity catalog.
PDF page 24
| Wait event name | Description | Modify |
|---|---|---|
| BackendRandomLock | Waiting to generate a random number. | Deleted |
| GSSOpenServer | Waiting for connect to GSSAPI server. | Added |
| CheckpointDone | Waiting for a checkpoint to complete. | Added |
| CheckpointStart | Waiting for a checkpoint to start. | Added |
| Promote | Waiting for standby promotion. | Added |
| WALSync | Waiting for a WAL file to reach stable storage. | Added |
3.1.8 ECPG
The following features have been added to ECPG.
Bytea data type
The variable of type bytea is defined in the DECLARE SECTION and can be used for data input/output.
PDF page 25
PREPARE AS statement
It is now possible to write SQL statements directly by PREPARE AS statement.
3.1.9 Pluggable Storage Engine
The basic specifications for using multiple storage engines have been decided. The access method for the table specifies the TYPE TABLE clause in the CREATE ACCESS METHOD statement. Syntax
The default storage engine is specified by the default_table_access_method parameter. The default value for this parameter is “heap”. As a result of this modification, it has been added tuple to the pg_am catalog.
PDF page 26
Specify engine when creating table
To specify a storage engine for a table, specify the USING clause in the CREATE TABLE statement. It can also be specified with the CREATE TABLE AS SELECT statement or the CREATE MATERIALIZED VIEW statement.
Display table definition by psql command
When the “\d+ table_name” command is executed, the storage engine name of the table is output.
PDF page 27
Display of access method by psql command
Table access methods are also displayed with the \dA command. Previously, only the index access method was output.
PDF page 28
3.1.10 Pg_hba.conf file
The following new features have been added to the pg_hba.conf file.
New parameter of clientcert item
A new setting “verfy-full” can be specified for the clientcert parameter. This option also ensures that the certificate’s “cn” (common name) matches the username or proper mapping.
GSSAPI authentication encryption option
Generic Security Standard Application Programming Interface (GSSAPI) with encryption option is available for client authentication. An entry of hostgssenc / hostnogssenc can be specified in the pg_hba.conf file. In order to enable GSSAPI authentication, it is necessary to specify –with-gssapi as an option of “configure” command at installation.
3.1.11 Text search
Supported languages for text search increased. It was 16 languages in PostgreSQL 11 but has increased to 22 languages in PostgreSQL 12.
PDF page 29
PDF page 30
3.1.12 Libpq API
The following functions have been added to the C language interface to PostgreSQL.
PQresultMemorySize
“size_t PQresultMemorySize (const PGresult * res)” API has been added. This function returns the amount of memory allocated by PGresult. It can be used to manage application memory.
GetForeignDataWrapperExtended
“ForeignDataWrapper * GetForeignDataWrapperExtended (Oid fwdid, bits16 flags)” API has been added.
GetForeignServerExtended
“ForeignServer * GetForeignServerExtended (Oid fwdid, bits16 flags)” API has been added.
3.1.13 Transaction ID
The 64-bit transaction ID is available. API “GetTopFullTransactionId”, “GetTopFullTransactionIdIfAny”, “GetCurrentFullTransactionId” and “GetCurrentFullTransactionIdIfAny” are provided. However, the current heap does not use them.
3.1.14 Client environment variable
Environment variables PG_COLOR and PG_COLORS have been added. Specify whether to use the color of the diagnostic message in PG_COLOR. Possible values are always, auto or never. PG_COLORS is specified by combining the category of escape sequence and code with an equal sign (=). When specifying multiple categories, separate them with a colon (:).
| Category | Default value | Note |
|---|---|---|
| error | 01;31 | |
| warning | 01;35 | |
| locus | 01 |
PDF page 31
3.1.15 Client connection string
Tcp_user_timeout indicating TCP timeout has been added to the client connection string.
4 SQL Statements
PDF page 32
This section explains new features related to SQL statements.
3.2.1 ALTER TABLE
It is now possible to change the attributes of some system catalogs.
3.2.2 ALTER TYPE ADD VALUE
The ALTER TYPE ADD VALUE statement is now available within a transaction block. However, the added value cannot be used in the transaction block.
3.2.3 COMMIT/ROLLBACK AND CHAIN
It became possible to add CHAIN clause which start a new transaction right after committing or discarding (ROLLBACK) the transaction. Specify the AND CHAIN clause in the COMMIT statement or ROLLBACK statement. Specify “AND NO CHAIN” to explicitly deny the CHAIN clause. These statements can also be used in PROCEDUREs using PL/pgSQL.
PDF page 33
PDF page 34
3.2.4 COPY
The following enhancements have been made to the COPY statement.
COPY FROM statement
It has become possible to store only the table data that matches the conditions specified in the COPY FROM statement.
PDF page 35
It can be executed similarly by \copy command of psql command.
COPY FREEZE statement
Executing a COPY FREEZE statement on partitioned table results in an error. This specification also applies to PostgreSQL 11.2 and later.
3.2.5 CREATE AGGREGATE
It is able to use the OR REPLACE clause in the CREATE AGGREGATE statement. Syntax
PDF page 36
3.2.6 CREATE COLLATION
DETERMINISTIC can be specified as an additional option in the CREATE COLLATION statement. The default value for this parameter is TRUE. Syntax
3.2.7 CREATE INDEX
Creation of GiST index
Covering indexes are now available for GiST indexes.
PDF page 37
GiST Index and VACUUM
Free pages are now reused by VACUUM.
WAL when creating GIN index
The amount of WAL output at indexing time has been significantly reduced.
3.2.8 CREATE STATISTICS
The mcv clause can be used in the CREATE STATISTICS statement. This value indicates Multivariate most-common values (MCV). It expands the regular MCV list and tracks the most frequent value combinations. The acquired statistics are stored in the stxmcv column of the pg_statistic_ext_data catalog.
3.2.9 CREATE TABLE
The following enhancements have been made to the CREATE TABLE statement.
GENERATED column
The generated column is the column defined as the column based on calculation results to the table. At the time of column definition, specify GENERATED ALWAYS AS (calculation expression) STORED clause following data type.
PDF page 38
PDF page 39
In the above example, it is displayed that the c1 column is ‘AB’ (= 0x4142), the c2 column is ‘CD’ (= 0x4344), and the c3 column is ‘ABCD’ (= 0x41424344). Information on generated columns can be found by storing “s” in the attgenerated column added to the pg_attrdef catalog. In addition, information_schema column_column_usage table is newly added.
PDF page 40
TABLESPACE clause of partition table definition
TABLESPACE clause is now enabled when creating the partition table. In previous versions, the TABLESPACE clause was ignored. Also, the value of the TABLESPACE clause of the partitioned table becomes the default tablespace at the time of partition creation.
PDF page 41
FOR VALUES clause of partition table
It is now possible to specify a formula or function instead of a literal in the FOR VALUES clause of a partition. The specified formula is executed only once when the CREATE TABLE statement is executed, and the calculated value is stored in the table definition.
PDF page 42
A foreign key reference to the partition table
The partition table can be referenced as a foreign key.
VACUUM processing for index
By specifying the VACUUM_INDEX_CLEANUP = OFF in WITH clause, it will be able to disable
PDF page 43
the VACUUM process for the index. The default value is ‘ON’, and VACUUM will be performed as before. Specify TOAST.VACUUM_INDEX_CLEANUP = OFF for the TOAST table.
Truncate empty block at the end of the table
VACUUM_TRUNCATE has been added to the attributes of the table. Determines whether the truncate empty block at the end of the table is released when performing VACUUM. The default value is ‘ON’, which truncate empty space as before. When it is specified as ‘OFF’, this operation is not performed.
PDF page 44
3.2.10 EXPLAIN
The SETTINGS ON option can now be specified in the EXPLAIN statement. This option outputs information on parameters related to the execution plan being changed from the default value.
PDF page 45
3.2.11 REINDEX CONCURRENTLY
The CONCURRENTLY option can now be added to the REINDEX statement. By reducing the lock range, application operation can be made to coexist with re-indexing. This feature is realized by temporarily creating a new index ({index_name}_ccnew) and replacing it with the old index.
3.2.12 PL/pgSQL additional check
The following values can now be specified for the parameter plpgsql.extra_warnings: Both can output additional warnings or errors when the function is executed.
Strict_multi_assignment setting
A warning is output when the number of columns output by the SELECT INTO statement does not match the number of input variables. In the example below, two warnings occur in the function.
PDF page 46
Too_many_rows setting
An error is generated when multiple records are returned by the SELECT INTO statement, and execution of the procedure is stopped.
PDF page 47
3.2.13 VACUUM / ANALYZE
The following features have been added to the VACUUM and ANALYZE statements.
SKIP_LOCKED clause
When executing a VACUUM or ANALYZE statement on a locked table, it has been waiting for the lock to be released. In PostgreSQL 12, an option SKIP_LOCKED clause has been added to skip locked tables. When processing is skipped, a log of WARNING level (LOG level for automatic VACUUM) is output. SQLSTATEs are considered successful even if skipped.
PDF page 48
Option specification syntax
In the VACUUM statement and ANALYZE statement, it is now possible to specify the action to be performed with TRUE/FALSE or ON/OFF.
Suppress VACUUM for index option
By specifying the OFF to INDEX_CLEANUP clause in VACUUM statement, it will be able to suppress the VACUUM process for the index. If this specification is omitted, the operation depends on the VACUUM_INDEX_CLEANUP attribute of the table.
PDF page 49
Suppression of free page truncation processing at the end of the table
The TRUNCATE clause has been added to the VACUUM statement. By specifying OFF for this attribute, it is possible to suppress free area deletion processing at the end of the table. If omitted, it depends on the VACUUM_TRUNCATE attribute of the table.
PDF page 50
3.2.14 WITH SELECT
In the past, common table expressions (CTE) specified in the WITH clause were all MATERIALIZED. In PostgreSQL 12, the default behavior has been changed to NOT MATERIALIZED. In order to change these behaviors, it is possible to specify MATERIALIZED or NOT MATERIALIZED in the WITH clause. Specifying the NOT MATERIALIZED clause allows the specification of the WHERE clause to be pushed down into the WITH clause. In the example below, using the NOT MATERIALIZED clause shows that index search is selected and the cost is reduced.
3.2.15 Functions
The following functions have been added/enhanced.
PDF page 51
SQL/JSON
Some functions related to SQL / JSON proposed in SQL 2016 standard are provided.
| Function name | Description |
|---|---|
| jsonb_path_exists | Checks whether the JSON path returns any item for the specified JSON value. |
| jsonb_path_match | Returns JSON path predicate result for the specified JSON value. Only the first result item is taken into account. |
| jsonb_path_query | Gets all JSON items returned by JSON path for the specified JSON value. |
| jsonb_path_query_array | Gets all JSON items returned by JSON path for the specified JSON value and wraps result into an array. |
| jsonb_path_query_first | Gets the first JSON item returned by JSON path for the specified JSON value. |
Pg_partition_tree
Pg_partition_tree is a function to display the tree structure of the partition table. It also supports hierarchical partition structure. Specify a partition table in the parameter. Specifying an object name other than a partition table or partition returns NULL.
PDF page 52
Pg_partition_root
The pg_partition_root is a function that returns the top-level partition table name of a specified partition. In the following example, a sub partition is created and the pg_partition_root function is executed.
PDF page 53
Pg_partition_ancestors
The pg_partition_ancestors function prints the list towards the parent of the partition table that contains the specified partition.
Pg_promote
This function promotes a standby instance to a primary instance. Previously, it was necessary to execute the pg_ctl promote command. Specify the parameter which determines whether to wait (default true) and the number of seconds to wait (default 60 seconds). This function returns false if processing fails or if the promotion is not completed within the waiting time, otherwise, it returns true.
PDF page 54
Pg_ls_tmpdir
The pg_ls_tmpdir function has been added to return a list of file names where temporary data has been saved. Specify the OID of the TABLESPACE in the parameter. If omitted, it is assumed that pg_default is specified. This function requires the SUPERUSER privilege or the pg_monitor role.
Pg_ls_archive_statusdir
The pg_ls_archive_statusdir function has been added to obtain the status of the archive file. This function searches the ${PGDATA}/pg_wal/archive_status directory and outputs the file name, size, and modification date. It does not output the actual information of archived WAL files. This function requires the SUPERUSER privilege or the pg_monitor role.
Date_trunc
This function can now set Timezone.
PDF page 55
Hyperbolic functions
The following hyperbolic functions included in SQL:standards 2016 have been added
| Functionname | Description | Note |
|---|---|---|
| log10 | base 10 logarithm | |
| sinh | hyperbolic sine | |
| cosh | hyperbolic cosine | |
| tanh | hyperbolic tangent | |
| asinh | inverse hyperbolic sine | |
| acosh | inverse hyperbolic cosine | |
| atanh | inverse hyperbolic tangent |
Copy replication slot
The function that performs a copy of an existing replication slot has been provided. The pg_copy_physical_replication_slot function and the pg_copy_logical_replication_slot function are provided according to the type of replication slot. To make a copy, it is necessary to replication slot is being used.
PDF page 56
5 Configuration Parameters
PDF page 57
In PostgreSQL 12 the following parameters have been changed.
3.3.1 Added parameters
The following parameters have been added.
| Parameter name | Description (context) | Default Val |
|---|---|---|
| archive_cleanup_command | Migrate from recovery.conf (sighup) | - |
| data_sync_retry | Behavior when fsync system call fails | off |
| (Also added to 11.2) | (postmaster) | |
| default_table_access_method Default storage engine name (user) | default_table_access_method Default storage engine name (user) | heap |
| log_statement_sample_rate | Rate of output of SQL statement to log (superuser) | 1 |
| plan_cache_mode | Changed the behavior of caching execution plans (user) | auto |
| primary_conninfo | Migration from recovery.conf file (postmaster) | - |
| primary_slot_name | Migrate from recovery.conf (postmaster) | - |
| promote_trigger_file | Migrate from recovery.conf (sighup) | - |
| recovery_end_command | Migrate from recovery.conf (sighup) | - |
| recovery_min_apply_delay | Migrate from recovery.conf (sighup) | 0 |
| recovery_target | Migrate from recovery.conf (postmaster) | - |
| recovery_target_action | Migrate from recovery.conf (postmaster) | pause |
| recovery_target_inclusive | Migrate from recovery.conf (postmaster) | on |
| recovery_target_lsn | Migrate from recovery.conf (postmaster) | - |
| recovery_target_name | Migrate from recovery.conf (postmaster) | - |
| recovery_target_time | Migrate from recovery.conf (postmaster) | - |
| recovery_target_timeline | Migrate from recovery.conf (postmaster) | latest |
| recovery_target_xid | Migrate from recovery.conf (postmaster) | - |
| restore_comand | Migrate from recovery.conf (postmaster) | - |
| shared_memory_type | Type of shared memory (postmaster) | OS depend |
| ssl_library | Name of library providing SSL function (internal) | - |
| ssl_max_protocol_version | Maximum version of SSL protocol to support (sighup) | - |
| ssl_min_protocol_version | Minimum version of SSL protocol to support (sighup) | TLSv1 |
| tcp_user_timeout | TCP timeout specification (user) | 0 |
| wal_init_zero | Fill-in WAL file to zero (superuser) | on |
| wal_recycle | Recycle WAL File (superuser) | on |
PDF page 58
Primary_conninfo parameter
Specifies the connection string for the primary instance. Previously this was specified in the recovery.conf file. The default value for the application_name entry has traditionally been “walreceiver”, but when the cluster_name parameter is specified, the value for cluster_name has been changed to the default value. The value of the application_name column in the pg_stat_replication catalog changes.
Ssl_library parameter
This parameter indicates the name of the library that provides the SSL feature. The parameter value is “OpenSSL” when –with-openssl is specified when executing configure command in Red Hat Enterprise Linux environment.
Shared_memory_type parameter
This parameter specifies the type of shared memory (such as shared_buffers).
PDF page 59
| Setting | Description | System call |
|---|---|---|
| mmap | Use an anonymous memory map. | mmap |
| sysv | Use System V shared memory. | shmget |
| windows | Use Windows shared memory. | CreateFileMapping |
The default value for this parameter on Linux is mmap. This is the same behavior as PostgreSQL
9.3 and later. Very small System V shared memory and most shared memory are configured using
memory map file (mmap). Setting this parameter to sysv allows you to revert to the pre-PostgreSQL
9.2 behavior. In this case, configure all shared memory using System V shared memory.
Plancache_mode parameter
This parameter sets the method for caching the execution plan of a prepared statement (created by the PREPARE statement). The default value is auto, which is the same behavior as previous versions. Normally, an execution plan is generated each time the SQL statement created by the PREPARE statement is executed by the EXECUTE statement. In the following example, it can be seen that the execution plan changes depending on the parameters specified in the EXECUTE statement. It indicates that the data stored in column C2 is uneven, there is little plan0 data (does an index search), and there is a lot of plan1 data (a whole table is searched).
PDF page 60
PDF page 61
The newly added parameter plan_cache_mode changes this behavior. Setting the parameter value to “force_custom_plan” turns off execution plan caching. On the other hand, setting the parameter value to “force_generic_plan” immediately enables execution plan caching.
Data_sync_retry parameter
This parameter determines what happens if the fsync system call issued during a checkpoint fails. In previous versions, the fsync function was re-executed (data_sync_retry = “on”), and the default behavior of the new version (data_sync_retry = “off”) caused an instance stoppage by PANIC if the fsync system call failed. This parameter has been added since PostgreSQL 11.2.
3.3.2 Changed parameters
The setting range and options were changed for the following configuration parameters.
PDF page 62
| dynamic_shared_memory_type | setting value “none” has been deleted. |
|---|---|
| log_autovacuum_min_duration | Log output contents now change according to VACUUM execution status. |
| log_connections | Application_name information has been added to the log. |
| plpgsql.extra_warnings | The following parameter values have been added: too_many_rows strict_multi_assignment |
| trace_sort | The log output message has been changed. |
| wal_level | It is now checked for proper level at startup. |
| wal_sender_timeout | The context has been changed from sighup to user. |
| default_with_oids | It cannot be set to “on”. |
| recovery_target_timeline | “Current” has been added as a setting value, the default value has been changed to “latest”. |
| autovacuum_vacuum_cost_delay Data type changed from integer to real. | autovacuum_vacuum_cost_delay Data type changed from integer to real. |
Parameter name Changes client_min_messages It is no longer possible to set it to a higher level than ERROR.
The default_with_oids parameter cannot be referenced from the pg_settings catalog.
PDF page 63
Wal_sender_timeout parameter
This parameter can now be changed by the user on a per session basis. This allows changing parameters on a per-connection basis from slave instances in a streaming replication environment.
Log_connections parameter
The value of the application_name parameter is added to the log that is output when this parameter is set to ‘on’.
PDF page 64
Connection from psql command
Connection from pg_basebackup command
Connection from Streaming Replication slave instance
Trace_sort parameter
The output log format has been changed when this parameter is set to ‘on’.
Wal_level parameter
When a replication slot is used, it became to be checked whether the wal_level parameter is an appropriate value during the instance startup. If the required level is not set, instance startup will fail.
PDF page 65
In the following example, wal_level is changed to “replica” and the instance is restarted in the Logical Replication environment.
3.3.3 Parameters with default values changed
The default values of the following configuration parameters have been changed.
| Parameter name | PostgreSQL 11 | PostgreSQL 12 | Note |
|---|---|---|---|
| autovacuum_vacuum_cost_delay | 20 | 2 | |
| extra_float_digitsjit | 0 | 1 | |
| off | on | ||
| recovery_target_timelineserver_version | current | latest | |
| 11.5 | 12.0 | ||
| server_version_num | 110005 | 120000 |
6 Utilities
PDF page 66
Describes the major enhancements of utility commands.
3.4.1 configure
The “–disable-strong-random” option has been removed from the “configure” command that is run when installing PostgreSQL from source code.
3.4.2 initdb
The initdb command now refers to the /etc/localtime file when determining the time zone of a database cluster. This file is referenced when the environment variable TZ is not specified.
3.4.3 oid2name
The oid2name command has been reviewed for options, and long-named options are now available.
| Shortoption | Addedlong-namedoption | Description |
|---|---|---|
| -f | –filenode | File node specification |
| -i | –indexes | Include index and sequence |
| -o | –oid | Specify OID |
| -q | –quiet | Omit header |
| -s | –tablespaces | Show tablespace OID |
| -S | –system-objects | Contains system objects |
| -t | –table | Specify table name |
| -x | –extended | Output additional information |
| -d | –dbname | Database to connect to |
| -h | –host | Hostname to connect to (-H option is deprecated) |
| -p | –port | Port number to connect to |
| -U | –username | Database username |
PDF page 67
3.4.4 pg_basebackup
The behavior of the pg_basebackup command has been changed when the –write-recovery-conf parameter (-R) is specified. The standby.signal file is automatically created in the backup destination folder, and the primary_conninfo parameter is added to the postgresql.auto.conf file. This behavior is only performed when connecting to a PostgreSQL 12 or later instance.
3.4.5 pg_checksums
The command pg_verify_checksums, which checks checksum integrity, has been renamed to pg_checksums. In addition to checking the integrity of checksums, it is now possible to change checksum activation/deactivation.
PDF page 68
PDF page 69
PDF page 70
Specify the –progress option (-P option) to display the processing status.
3.4.6 pg_ctl
The following feature has been added to the pg_ctl command.
logrotate option
The parameter logrotate for log rotation has been added to the pg_ctl command. In the past, it was necessary to send a SIGHUP signal to the logger process. To suppress the message, specify the -s parameter at the same time.
3.4.7 pg_dump
The following options have been added to the pg_dump command.
–on-conflict-do-nothing option
This option automatically assigns the ON CONFLICT DO NOTHING clause to the output INSERT statement. Must be specified with the –inserts option or –column-inserts option.
PDF page 71
–extra-float-digits option
If an integer value is specified for this parameter, the “SET extra_float_digits = specified value” statement is executed before data acquisition using the pg_dump command. The dump file does not contain SET statements. The range of values that can be specified is -15 to 3. If a non-numeric value is specified, it is regarded as 0.
–rows-per-insert option
This option is used with the –inserts option. Multiple tuples can be inserted in a single INSERT statement. The range of values is 1 to 2,147,483,647.
PDF page 72
3.4.8 pg_dumpall
The following options have been added to the pg_dumpall command.
–extra-float-digits option
If an integer value is specified for this parameter, the “SET extra_float_digits = specified value” statement is executed before data acquisition using the pg_dumpall command. The dump file does not contain SET statements. The range of values that can be specified is -15 to 3. If a non-numeric value is specified, it is regarded as 0.
–exclude-database option
In PostgreSQL 12, the –exclude-database option has been added. This option specifies which databases to exclude from backup. When specifying multiple databases, use the same pattern as for the psql command. It is also possible to specify the same option multiple times. In the following example, demodb1 and demodb2 are specified as excluded databases.
PDF page 73
–oids
This option has been removed.
Additional comments
Comments have been added to the output file for user settings (ALTER USER SET statement) and database settings.
3.4.9 pg_rewind
The –no-sync option which does not execute sync system calls for storage has been added to the pg_rewind command.
PDF page 74
3.4.10 pg_restore
When specifying standard output as the data output destination, specify “-f -”.
3.4.11 pg_upgrade
The following options have been added to the pg_upgrade command.
–socketdir option
The –socketdir option (or -s option) that specifies the directory for creating a local socket has been added.
–clone option
The –clone option performs fast cloning using the “reflink” feature. This feature is available only on some operating systems and file systems.
PDF page 75
3.4.12 psql
The following features have been added to the psql command.
CSV format output
Output format from psql command can be changed to CSV format. It can be changed in any of the following ways.
Specify psql command parameter –csv
Execute “\pset format csv” command from within the psql command
The default value of column separator is a comma (,), but it can be changed by “\pset csv_fieldsep” command. If the output value contains delimiters, the value is enclosed in double quotation marks ("). Column titles can be suppressed with “\pset tuples_only on” command.
Display of partition table
The partition table is now output explicitly when the \d command is executed.
PDF page 76
Display object privilege
Partitioned tables are now specified in the list of object privileges.
PDF page 77
Display connection information
The TCP / IP address is now output when the \conninfo command is executed.
SQLSTATE with VERBOSITY
SQLSTATE can now be specified in the “\set VERBOSITY” command.
PDF page 78
Display of partition table
The “\dP” command to display only the partition table has been added.
Display of manual URL
Specifying DDL with the \h command now displays the URL of the corresponding manual.
PDF page 79
3.4.13 vacuumdb
The following options have been added to the vacuumdb command.
–disable-page-skipping option
This option is for executing a VACUUM (DISABLE_PAGE_SKIPPING) statement from a command line. This option can be specified for PostgreSQL 9.6 or later instances.
–skip-locked option
This option is for executing a VACUUM (SKIPP_LOCKED) statement from a command line. This option can be specified for PostgreSQL 12 or later instances.
–min-mxid-age option
Only execute the vacuum or analyze commands on tables with a multixact ID age of at least specified age.
PDF page 80
–min-xid-age option
Only execute the vacuum or analyze commands on tables with a transaction ID age of at least specified age.
3.4.14 vacuumlo
The vacuumlo command now has a long name option available.
| Shortoption | Addedlong-namedoption | Description |
|---|---|---|
| -l | –limit | The upper limit of objects to delete |
| -n | –dry-run | Not actually run |
| -v | –verbose | The output of additional information |
| -h | –host | Connect host |
| -p | –port | Connect port number |
| -U | –username | Connect user name |
| -w | –no-password | Do not output password prompt |
| -W | –password | Force password input |
7 Contrib Modules
PDF page 81
Describes new features related to the Contrib module.
3.5.1 auto_explain
Auto_explain.log_level parameter
The parameter auto_explain.log_level (default value LOG) has been added. This parameter can specify the level of log output by the auto_explain module. Previously, it was fixed at log level LOG.
Auto_explain.log_settings parameter
Determines whether changed optimizer-related parameters are output to the log. The default value is ‘off’, so no log is output.
PDF page 82
Auto_explain.sample_rate parameter
Specify the sampling rate for log output. The default value is 1, which outputs all output SQL statements.
Add JIT compilation information
JIT compilation information is now output to the log (backported to PostgreSQL 11.2).
3.5.2 citext
Added citext_hash_extended to calculate 64-bit hash value. The second parameter specifies the SEED.
PDF page 83
3.5.3 hstore
Hstore_hash_extended has been added to calculate 64-bit hash values. The second parameter specifies the SEED.
PDF page 84
3.5.4 pg_stat_statements
The pg_stat_statements_reset function of the pg_stat_statements module has added a parameter that limits the deletion range of statistical information. Statistical information can be deleted by specifying database ID, user ID, and query ID. If these parameters are omitted (the default value is 0), all statistics will be discarded as before.
PDF page 85
3.5.5 postgres_fdw
The postgres_fdw module adds an “options” option that specifies the GUC options that can be run in a remote session. For example, settings of work_mem parameter and geqo parameter can be changed. Set values are described in the “-c parameter name = value” format. When specifying multiple parameters, specify the entire “-c parameter_name = value” separated by a space.
8 URL List
The links in this chapter are preserved from the source PDF’s URL list.
PDF page 86
9 Change History
This chapter preserves the source document’s revision history and publication credits.
PDF page 87
Change history
| Version | Date | Author | Description |
|---|---|---|---|
| 0.1 | Apr 22, 2019 | Noriyoshi Shinoda | Create internal review version. Reviewer: Reviewer: Tomoo Takahashi Akiko Takeshima Takahiro Kitayama (Hewlett Packard Enterprise Japan) |
| 1.0 | May 24, 2019 | Noriyoshi Shinoda | Modification completed according PostgreSQL 12 Beta 1. Reviewer: Satoshi Nagayasu (Uptime Technologies, LCC.) |
| 1.1 | Oct 7, 2019 | Noriyoshi Shinoda | Modification completed according PostgreSQL 12 GA version. |
| Oct 11, 2019 | Noriyoshi Shinoda | Some typo fix | |
| 1.1.1 |