Slow Query Log
source link: https://www.percona.com/doc/percona-server/5.5/diagnostics/slow_extended_55.html
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Slow Query Log¶
This feature adds microsecond time resolution and additional statistics to the slow query log output. It lets you enable or disable the slow query log at runtime, adds logging for the slave SQL thread, and adds fine-grained control over what and how much to log into the slow query log.
The ability to log queries with microsecond precision is essential for measuring the work the MySQL server performs. The standard slow query log in MySQL 5.0 has only 1-second granularity, which is too coarse for all but the slowest queries. MySQL 5.1 has microsecond resolution, but does not have the extra information about query execution that is included in the Percona Server.
You can use Percona-Toolkit‘s pt-query-digest tool to aggregate similar queries together and report on those that consume the most execution time.
Version Specific Information¶
5.5.8-20.0
: Added valuesprofiling
andprofiling_use_getrusage
to variable log_slow_verbosity.5.5.34-32.0
:
- New
slow_query_log_always_write_time
variable introduced.5.5.41-37.0
:
- Implemented improved slow log reporting for queries in stored procedures.
5.5.44-37.3
:
log_slow_sp_statements
now supports skipping the logging of stored procedures into the slow log entirely with newOFF_NO_CALLS
option.
Other Information¶
- Author / Origin: Maciej Dobrzanski, Percona
System Variables¶
variablelog_slow_admin_statements
¶
Command Line: | Yes |
---|---|
Config File: | Yes |
Scope: | Global |
Dynamic: | Yes |
When this variable is enabled, administrative statements will be logged to the slow query log. Upstream version of the MySQL server has implemented command line option with same name. Significant difference is that this feature is implemented as variable in Percona Server, that means it can be enabled/disabled dynamically without restarting the server.
variablelog_slow_filter
¶
Command Line: | Yes |
---|---|
Config File: | Yes |
Scope: | Global, Session |
Dynamic: | Yes |
Filters the slow log by the query’s execution plan. The value is a comma-delimited string, and can contain any combination of the following values:
qc_miss
: The query was not found in the query cache.full_scan
: The query performed a full table scan.full_join
: The query performed a full join (a join without indexes).tmp_table
: The query created an implicit internal temporary table.tmp_table_on_disk
: The query’s temporary table was stored on disk.filesort
: The query used a filesort.filesort_on_disk
: The filesort was performed on disk.
Values are OR’ed together. If the string is empty, then the filter is disabled. If it is not empty, then queries will only be logged to the slow log if their execution plan matches one of the types of plans present in the filter.
For example, to log only queries that perform a full table scan, set the value to full_scan
. To log only queries that use on-disk temporary storage for intermediate results, set the value to tmp_table_on_disk,filesort_on_disk
.
log_slow_rate_type
¶
Command Line: | Yes |
---|---|
Config File: | Yes |
Scope: | Global |
Dynamic: | Yes |
Variable Type: | Enumerated |
Default Value: | session |
Range: | session , query |
Specifies semantic of log_slow_rate_limit
- session
or query
.
log_slow_rate_limit
¶
Command Line: | Yes |
---|---|
Config File: | Yes |
Scope: | Global, session |
Dynamic: | Yes |
Default Value: | 1 |
Range: | 1-1000 |
Behavior of this variable depends from log_slow_rate_type
.
Specifies that only a fraction of session/query
should be logged. Logging is enabled for every nth session/query
. By default, n is 1, so logging is enabled for every session/query
. Please note: when log_slow_rate_type
is session
rate limiting is disabled for the replication thread.
- When
log_slow_rate_type
issession
, this option lets you log full sessions, so you have complete records of sessions for later analysis; but you can rate-limit the number of sessions that are logged. Note that this feature will not work well if your application uses any type of connection pooling or persistent connections. Note that if you changelog_slow_rate_limit
tosession
mode, you should reconnect for the setting to take effect. - When
log_slow_rate_type
isquery
, this option lets you log just some queries for later analysis. For example, if you set the value to 100, then one percent of queries will be logged.
Note that every query has global unique query_id
and every connection can has it own (session) log_slow_rate_limit
.
Decision “log or no” calculated in following manner:
- if
log_slow_rate_limit
is 1 - log every query- If
log_slow_rate_limit
> 1 - randomly log every 1/log_slow_rate_limit
query.
This allows flexible setup logging behavior.
For example, if you set the value to 100, then one percent of sessions/queries
will be logged. In Percona Server 5.5.34-32.0
information about the log_slow_rate_limit
has been added to the slow query log. This means that if the log_slow_rate_limit
is effective it will be reflected in the slow query log for each written query. Example of the output looks like this:
Log_slow_rate_type: query Log_slow_rate_limit: 10
Prior to 5.5.37-35.0
implementation of the log_slow_rate_type
set to query
with log_slow_rate_limit
feature would log every nth query deterministically. With the current implementation each query has a non-deterministic probability of 1/n to get logged.”
log_slow_slave_statements
¶
Command Line: | Yes |
---|---|
Config File: | Yes |
Scope: | Global, session |
Dynamic: | Yes |
Specifies that slow queries replayed by the slave SQL thread on a MySQL slave will be logged. Upstream version of the MySQL server has implemented command line option with same name. Significant difference is that this feature is implemented as variable in Percona Server, that means it can be enabled/disabled dynamically without restarting the server.
To start the logging from the slave thread, you should change the global value: set global log_slow_slave_statements
=ON
; and then execute: STOP SLAVE; START SLAVE;
. This will destroy and recreate the slave SQL thread, so it will see the newly set global value.
To stop the logging from the slave thread, you should just change the global value: set global log_slow_slave_statements
=OFF
; the logging stops immediately.
log_slow_sp_statements
¶
Command Line: | Yes |
---|---|
Config File: | Yes |
Scope: | Global |
Dynamic: | Yes |
Variable Type: | Boolean |
Default Value: | ON |
Range: | ON/OFF/OFF_NO_CALLS |
This variable controls the logging of stored procedures in the slow query log. The options are to log the CALL
statements, to log the individual statements in the stored procedure, or to log neither. These options correspond to values OFF
, ON
, and OFF_NO_CALLS
respectively.
5.5.41-37.0
implementation of logging stored procedures was logging the stored procedure CALLs
themselves along with the queries inside the procedures. This meant that some queries were counted more than once which could make tracking the bad-performing queries harder and it would cause noise in the slow query log. Percona Server 5.5.41-37.0
implemented improvements for logging of stored procedures to the slow query log:
- Each query from a stored procedure is now logged to the slow query log individually
CALL
itself isn’t logged to the slow query log anymore as this would be counting twice for the same query which would lead to incorrect results- Queries that were called inside of stored procedures are annotated in the slow query log with the stored procedure name in which they run.
In 5.5.44-37.3
new OFF_NO_CALLS
option has been implemented that can be used for skipping the logging of stored procedures into the slow log entirely.
Example of the improved stored procedure slow query log entry:
mysql> DELIMITER //
mysql> CREATE PROCEDURE improved_sp_log()
BEGIN
SELECT * FROM City;
SELECT * FROM Country;
END//
mysql> DELIMITER ;
mysql> CALL improved_sp_log();
When we check the slow query log after running the stored procedure ,with variable:log_slow_sp_statements set to TRUE
, it should look like this:
# Time: 150109 11:38:55
# User@Host: root[root] @ localhost []
# Thread_id: 40 Schema: world Last_errno: 0 Killed: 0
# Query_time: 0.012989 Lock_time: 0.000033 Rows_sent: 4079 Rows_examined: 4079 Rows_affected: 0 Rows_read: 4079
# Bytes_sent: 161085
# Stored routine: world.improved_sp_log
SET timestamp=1420803535;
SELECT * FROM City;
# User@Host: root[root] @ localhost []
# Thread_id: 40 Schema: world Last_errno: 0 Killed: 0
# Query_time: 0.001413 Lock_time: 0.000017 Rows_sent: 4318 Rows_examined: 4318 Rows_affected: 0 Rows_read: 4318
# Bytes_sent: 194601
# Stored routine: world.improved_sp_log
SET timestamp=1420803535;
SELECT * FROM Country;
If variable log_slow_sp_statements
is set to FALSE
:
- Entry is added to a slow-log for a
CALL
statement only and not for any of the individual statements run in that stored procedure- Execution time is reported for the
CALL
statement as the total execution time of theCALL
including all its statements
If we run the same stored procedure with the variable log_slow_sp_statements
is set to FALSE
slow query log should look like this:
# Time: 150109 11:51:42
# User@Host: root[root] @ localhost []
# Thread_id: 40 Schema: world Last_errno: 0 Killed: 0
# Query_time: 0.013947 Lock_time: 0.000000 Rows_sent: 4318 Rows_examined: 4318 Rows_affected: 0 Rows_read: 4318
# Bytes_sent: 194612
SET timestamp=1420804302;
CALL improved_sp_log();
Support for logging stored procedures doesn’t involve triggers, so they won’t be logged even if this feature is enabled.
log_slow_verbosity
¶
Version Info: |
|
---|---|
Command Line: | |
Config File: | |
Scope: | Global, session |
Dynamic: |
Specifies how much information to include in your slow log. The value is a comma-delimited string, and can contain any combination of the following values:
microtime
: Log queries with microsecond precision.query_plan
: Log information about the query’s execution plan.innodb
: Log InnoDB statistics.minimal
: Equivalent to enabling justmicrotime
.standard
: Equivalent to enablingmicrotime,innodb
.full
: Equivalent to all other values OR’ed together without theprofiling
andprofiling_use_getrusage
options.profiling
: Enables profiling of all queries in all connections.profiling_use_getrusage
: Enables usage of the getrusage function.
Values are OR’ed together.
For example, to enable microsecond query timing and InnoDB statistics, set this option to microtime,innodb
or standard
. To turn all options on, set the option to full
.
slow_query_log_timestamp_always
¶
Version Info: |
|
---|---|
Command Line: | |
Config File: | |
Scope: | Global |
Dynamic: | |
Variable Type: | Boolean |
Default Value: | FALSE |
Range: | TRUE/FALSE |
If TRUE
, a timestamp is printed on every slow log record. Multiple records may have the same time.
slow_query_log_timestamp_precision
¶
Version Info: |
|
---|---|
Command Line: | |
Config File: | |
Scope: | Global |
Dynamic: | |
Variable Type: | Enumerated |
Default Value: |
|
Range: |
|
Normally, entries to the slow query log are in seconds precision, in this format:
# Time: 090402 9:23:36 # User@Host: XXX @ XXX [10.X.X.X]
If slow_query_log_timestamp_precision
=microsecond
, entries to the slow query log are in microsecond precision, in this format:
# Time: 090402 9:23:36.123456 # User@Host: XXX @ XXX [10.X.X.X]
slow_query_log_use_global_control
¶
Command Line: | |
---|---|
Config File: | |
Scope: | Global |
Dynamic: | |
Default Value: | |
Version Info: |
|
Specifies which variables have global scope instead of local. For such variables, the global variable value is used in the current session, but without copying this value to the session value. Value is a “flag” variable - you can specify multiple values separated by commas:
none
: All variables use local scopelog_slow_filter
: Global variablelog_slow_filter
has effect (instead of local)log_slow_rate_limit
: Global variablelog_slow_rate_limit
has effect (instead of local)log_slow_verbosity
: Global variablelog_slow_verbosity
has effect (instead of local)long_query_time
: Global variablelong_query_time
has effect (instead of local)min_examined_row_limit
: Global variablemin_examined_row_limit
has effect (instead of local)all
Global variables has effect (instead of local)
NOTE: This variable has been renamed from log_slow_timestamp_every
since 5.5.10-20.1.
slow_query_log_always_write_time
¶
Command Line: | Yes |
---|---|
Config File: | Yes |
Scope: | Global |
Dynamic: | Yes |
Default Value: | 10 (seconds) |
This variable can be used to specify the query execution time after which the query will be written to the slow query log. It can be used to specify an additional execution time threshold for the slow query log, that, when exceeded, will cause a query to be logged unconditionally, that is, log_slow_rate_limit
will not apply to it.
Other Information¶
Changes to the Log Format¶
The feature adds more information to the slow log output. Here is a sample log entry:
# User@Host: mailboxer[mailboxer] @ [192.168.10.165]
# Thread_id: 11167745 Schema: board
# Query_time: 1.009400 Lock_time: 0.000190 Rows_sent: 4 Rows_examined: 1543719 Rows_affected: 0 Rows_read: 4
# Bytes_sent: 278 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0
# InnoDB_trx_id: 1500
# QC_Hit: No Full_scan: Yes Full_join: No Tmp_table: No Tmp_table_on_disk: No
# Filesort: No Filesort_on_disk: No Merge_passes: 0
# InnoDB_IO_r_ops: 6415 InnoDB_IO_r_bytes: 105103360 InnoDB_IO_r_wait: 0.001279
# InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000
# InnoDB_pages_distinct: 6430
SET timestamp=1346844943;
SELECT id,title,production_year FROM title WHERE title = 'Bambi';
Another example (log_slow_verbosity
=profiling
):
# Query_time: 0.962742 Lock_time: 0.000202 Rows_sent: 4 Rows_examined: 1543719 Rows_affected: 0 Rows_read: 4
# Bytes_sent: 278 Tmp_tables: 0 Tmp_disk_tables: 0 Tmp_table_sizes: 0
# Profile_starting: 0.000030 Profile_starting_cpu: 0.000028 Profile_Waiting_for_query_cache_lock: 0.000003
Profile_Waiting_for_query_cache_lock_cpu: 0.000003 Profile_Waiting_on_query_cache_mutex: 0.000003
Profile_Waiting_on_query_cache_mutex_cpu: 0.000003 Profile_checking_query_cache_for_query: 0.000076
Profile_checking_query_cache_for_query_cpu: 0.000076 Profile_checking_permissions: 0.000011
Profile_checking_permissions_cpu: 0.000011 Profile_Opening_tables: 0.000078 Profile_Opening_tables_cpu: 0.000078
Profile_System_lock: 0.000022 Profile_System_lock_cpu: 0.000022 Profile_Waiting_for_query_cache_lock: 0.000003
Profile_Waiting_for_query_cache_lock_cpu: 0.000002 Profile_Waiting_on_query_cache_mutex: 0.000054
Profile_Waiting_on_query_cache_mutex_cpu: 0.000054 Profile_init: 0.000039 Profile_init_cpu: 0.000040
Profile_optimizing: 0.000015 Profile_optimizing_cpu: 0.000014 Profile_statistics: 0.000021 Profile_statistics_cpu: 0.000021
Profile_preparing: 0.000020 Profile_preparing_cpu: 0.000020 Profile_executing: 0.000003 Profile_executing_cpu: 0.000003
Profile_Sending_data: 0.962324 Profile_Sending_data_cpu: 0.961526 Profile_end: 0.000006 Profile_end_cpu: 0.000005
Profile_query_end: 0.000004 Profile_query_end_cpu: 0.000004 Profile_closing_tables: 0.000008 Profile_closing_tables_cpu: 0.000008
Profile_freeing_items: 0.000007 Profile_freeing_items_cpu: 0.000007 Profile_Waiting_for_query_cache_lock: 0.000000
Profile_Waiting_for_query_cache_lock_cpu: 0.000001 Profile_Waiting_on_query_cache_mutex: 0.000001
Profile_Waiting_on_query_cache_mutex_cpu: 0.000001 Profile_freeing_items: 0.000017 Profile_freeing_items_cpu: 0.000016
Profile_Waiting_for_query_cache_lock: 0.000001 Profile_Waiting_for_query_cache_lock_cpu: 0.000001
Profile_Waiting_on_query_cache_mutex: 0.000000 Profile_Waiting_on_query_cache_mutex_cpu: 0.000001
Profile_freeing_items: 0.000001 Profile_freeing_items_cpu: 0.000001 Profile_storing_result_in_query_cache: 0.000002
Profile_storing_result_in_query_cache_cpu: 0.000002 Profile_logging_slow_query: 0.000001 Profile_logging_slow_query_cpu: 0.000001
# Profile_total: 0.962751 Profile_total_cpu: 0.961950
# InnoDB_trx_id: 1700
Connection and Schema Identifier¶
Each slow log entry now contains a connection identifier, so you can trace all the queries coming from a single connection. This is the same value that is shown in the Id column in SHOW FULL PROCESSLIST
or returned from the CONNECTION_ID()
function.
Each entry also contains a schema name, so you can trace all the queries whose default database was set to a particular schema.
# Thread_id: 11167745 Schema: board
Microsecond Time Resolution and Extra Row Information¶
This is the original functionality offered by the microslow
feature. Query_time
and Lock_time
are logged with microsecond resolution.
The feature also adds information about how many rows were examined for SELECT
queries, and how many were analyzed and affected for UPDATE
, DELETE
, and INSERT
queries,
# Query_time: 0.962742 Lock_time: 0.000202 Rows_sent: 4 Rows_examined: 1543719 Rows_affected: 0 Rows_read: 4
Values and context:
Rows_examined
: Number of rows scanned -SELECT
Rows_affected
: Number of rows changed -UPDATE
,DELETE
,INSERT
Rows_read
: Number of rows read -UPDATE
,DELETE
,INSERT
Memory Footprint¶
The feature provides information about the amount of bytes sent for the result of the query and the number of temporary tables created for its execution - differentiated by whether they were created on memory or on disk - with the total number of bytes used by them.
# Bytes_sent: 8053 Tmp_tables: 1 Tmp_disk_tables: 0 Tmp_table_sizes: 950528
Values and context:
Bytes_sent
: The amount of bytes sent for the result of the queryTmp_tables
: Number of temporary tables created on memory for the queryTmp_disk_tables
: Number of temporary tables created on disk for the queryTmp_table_sizes
: Total Size in bytes for all temporary tables used in the query
Query Plan Information¶
Each query can be executed in various ways. For example, it may use indexes or do a full table scan, or a temporary table may be needed. These are the things that you can usually see by running EXPLAIN
on the query. The feature will now allow you to see the most important facts about the execution in the log file.
# QC_Hit: No Full_scan: Yes Full_join: No Tmp_table: No Tmp_table_on_disk: No
# Filesort: No Filesort_on_disk: No Merge_passes: 0
The values and their meanings are documented with the log_slow_filter
option.
InnoDB Usage Information¶
The final part of the output is the InnoDB usage statistics. MySQL currently shows many per-session statistics for operations with SHOW SESSION STATUS
, but that does not include those of InnoDB, which are always global and shared by all threads. This feature lets you see those values for a given query.
# InnoDB_IO_r_ops: 6415 InnoDB_IO_r_bytes: 105103360 InnoDB_IO_r_wait: 0.001279
# InnoDB_rec_lock_wait: 0.000000 InnoDB_queue_wait: 0.000000
# InnoDB_pages_distinct: 6430
Values:
innodb_IO_r_ops
: Counts the number of page read operations scheduled. The actual number of read operations may be different, but since this can be done asynchronously, there is no good way to measure it.innodb_IO_r_bytes
: Similar to innodb_IO_r_ops, but the unit is bytes.innodb_IO_r_wait
: Shows how long (in seconds) it took InnoDB to actually read the data from storage.innodb_rec_lock_wait
: Shows how long (in seconds) the query waited for row locks.innodb_queue_wait
: Shows how long (in seconds) the query spent either waiting to enter the InnoDB queue or inside that queue waiting for execution.innodb_pages_distinct
: Counts approximately the number of unique pages the query accessed. The approximation is based on a small hash array representing the entire buffer pool, because it could take a lot of memory to map all the pages. The inaccuracy grows with the number of pages accessed by a query, because there is a higher probability of hash collisions.
If the query did not use InnoDB tables, that information is written into the log instead of the above statistics.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK