Skip to content

Commit

Permalink
OSI - performance information
Browse files Browse the repository at this point in the history
  • Loading branch information
@grooverdan
grooverdan committed Oct 22, 2024
1 parent c169105 commit 8d3d5c4
Showing 1 changed file with 239 additions and 89 deletions.
328 changes: 239 additions & 89 deletions notebooks/2024 Open Source India MariaDB Workshop .ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
{
"cell_type": "markdown",
"id": "8b9099f6-8b19-4a8f-8657-590712e6d006",
"metadata": {},
"metadata": {
"jp-MarkdownHeadingCollapsed": true
},
"source": [
"# Fine Tuning and Optimising MariaDB Query\n",
"\n",
Expand Down Expand Up @@ -85,94 +87,6 @@
"(Source: Wikipedia: https://en.wikipedia.org/wiki/MariaDB and conversion by chatGPT)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "47eae534-316d-4c29-b1ab-17c6f3d4960f",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "62fd9e24-f611-4190-b207-076bbcdb7bb1",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "bc18fb6b-cbd2-491c-9a82-8195808d23f5",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "eee7e34a-ae49-4c87-8a07-28fc414ba156",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "13669255-7193-47dd-96b5-fee3f710495e",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "40c4c39b-b4e7-4cf6-aac8-4ae7a12c7724",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "6f7cb5d9-0d11-4861-b267-9e490b343e88",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "961a0742-db90-4a7c-831a-4d8db7aa842e",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "1381e994-d81c-4850-ae9e-a5cbf21a92f3",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "c4cdaba8-2c29-40dc-91bb-e9b636e1cedb",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "647763f9-ae71-4673-aaf8-2b483f5aeff2",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "af51619f-2990-4c15-8265-8d1e52978192",
Expand Down Expand Up @@ -738,6 +652,14 @@
"analyze format=json select t2.* from collate_join1 t1 left join collate_join2 t2 using (id) where t2.id is null;"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "47eae534-316d-4c29-b1ab-17c6f3d4960f",
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"id": "7bf0e927-6585-4105-9b44-3a380609da57",
Expand Down Expand Up @@ -2266,6 +2188,234 @@
"\n",
"SET STATEMENT optimizer_switch={optimization=on} FOR SELECT ....\n"
]
},
{
"cell_type": "markdown",
"id": "122ed4d5-7ec5-4554-b973-205fbacfce76",
"metadata": {},
"source": [
"# Problem Diagnosis\n",
"\n",
"Understand the problem before jumping to conclusions.\n",
"\n",
"## MariaDB Error Log\n",
"\n",
"std/stderr by default - means it goes to systemd service - so journalctl -u mariadb.service\n",
"\n",
"Take note of types:\n",
"* \\[Note\\] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB\n",
"* \\[Warning\\] mariadbd: io_uring_queue_init() failed with ENOSYS: check seccomp filters, and the kernel version (newer than 5.1 required)\n",
"* \\[ERROR\\] mariadbd: Can't lock aria control file '/var/lib/mysql/aria_log_control' for exclusive use, error: 11. Will retry\n",
"\n",
"To control what is logged **[log_warnings](https://mariadb.com/kb/en/server-system-variables/#log_warnings)** has levels.\n",
"\n",
"## Analyze format=json / explain\n",
"\n",
"Elements of EXPLAIN {EXTENDED}\n",
"```\n",
"MariaDB [test]> explain extended select seq from seq_1_to_30;\n",
"+------+-------------+-------------+-------+---------------+---------+---------+------+------+----------+-------------+\n",
"| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |\n",
"+------+-------------+-------------+-------+---------------+---------+---------+------+------+----------+-------------+\n",
"| 1 | SIMPLE | seq_1_to_30 | index | NULL | PRIMARY | 8 | NULL | 30 | 100.00 | Using index |\n",
"+------+-------------+-------------+-------+---------------+---------+---------+------+------+----------+-------------+\n",
"```\n",
"\n",
"This a plan. There may be changes. `ANALYZE` runs the query\n",
"\n",
"```\n",
"ANALYZE FORMAT=JSON select seq from seq_1_to_30\\G\n",
"*************************** 1. row ***************************\n",
"ANALYZE: {\n",
" \"query_optimization\": {\n",
" \"r_total_time_ms\": 0.211595358\n",
" },\n",
" \"query_block\": {\n",
" \"select_id\": 1,\n",
" \"r_loops\": 1,\n",
" \"r_total_time_ms\": 0.128058752,\n",
" \"nested_loop\": [\n",
" {\n",
" \"table\": {\n",
" \"table_name\": \"seq_1_to_30\",\n",
" \"access_type\": \"index\",\n",
" \"key\": \"PRIMARY\",\n",
" \"key_length\": \"8\",\n",
" \"used_key_parts\": [\"seq\"],\n",
" \"r_loops\": 1,\n",
" \"rows\": 30,\n",
" \"r_rows\": 30,\n",
" \"r_table_time_ms\": 0.005894714,\n",
" \"r_other_time_ms\": 0.112255834,\n",
" \"r_engine_stats\": {},\n",
" \"filtered\": 100,\n",
" \"r_filtered\": 100,\n",
" \"using_index\": true\n",
" }\n",
" }\n",
" ]\n",
" }\n",
"```\n",
"\n",
"As you can see ANALYZE has measurements, identification of what parts of the index where used.\n",
"\n",
"*r_* is the real what happened during execution. Non *r_* lows rows/filtered where an estimate. When they differ signficantly, try updating table statistics."
]
},
{
"cell_type": "markdown",
"id": "eed9dd6b-6bf2-41e2-9a47-56e0fc929930",
"metadata": {},
"source": [
"## Slow Query Log\n",
"\n",
"Since 10.11 - all dynamic, and session/global.\n",
"\n",
"* **[slow_query_log=1](https://mariadb.com/kb/en/server-system-variables/#slow_query_log)** - boolean variable to enable\n",
"* **[log_slow_query_time=0.4](https://mariadb.com/kb/en/server-system-variables/#log_slow_query_time)** - what constitues a slow query (default of 10s) is too big for many cases\n",
"* **[log_slow_verbosity=full](https://mariadb.com/kb/en/server-system-variables/#log_slow_verbosity)** - how much detail - *full* include EXPLAIN EXTENDED information.\n",
"\n",
"ref: https://mariadb.com/kb/en/slow-query-log-overview/#configuring-the-slow-query-log-time"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "eee7e34a-ae49-4c87-8a07-28fc414ba156",
"metadata": {},
"outputs": [],
"source": [
"## SHOW STATUS"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "13669255-7193-47dd-96b5-fee3f710495e",
"metadata": {},
"outputs": [],
"source": [
"SHOW GLOBAL STATUS"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "40c4c39b-b4e7-4cf6-aac8-4ae7a12c7724",
"metadata": {},
"outputs": [],
"source": [
"SHOW SESSION STATUS"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6f7cb5d9-0d11-4861-b267-9e490b343e88",
"metadata": {},
"outputs": [],
"source": [
"SHOW ENGINE INNODB STATUS"
]
},
{
"cell_type": "markdown",
"id": "7420b9b2-5d2e-43c6-bf39-ad2e8f1be5e6",
"metadata": {},
"source": [
"Last Dead lock infomation\n",
"\n",
"```\n",
"=====================================\n",
"2024-10-22 16:03:46 0x7f64bad016c0 INNODB MONITOR OUTPUT\n",
"=====================================\n",
"Per second averages calculated from the last 50 seconds\n",
"-----------------\n",
"BACKGROUND THREAD\n",
"-----------------\n",
"srv_master_thread loops: 0 srv_active, 0 srv_shutdown, 891 srv_idle\n",
"srv_master_thread log flush and writes: 891\n",
"----------\n",
"SEMAPHORES\n",
"----------\n",
"------------\n",
"TRANSACTIONS\n",
"------------\n",
"Trx id counter 111\n",
"Purge done for trx's n:o < 0 undo n:o < 0 state: running but idle\n",
"History list length 0\n",
"LIST OF TRANSACTIONS FOR EACH SESSION:\n",
"---TRANSACTION (0x7f64ca202340), not started\n",
"0 lock struct(s), heap size 1128, 0 row lock(s)\n",
"--------\n",
"FILE I/O\n",
"--------\n",
"Pending flushes (fsync): 0\n",
"187 OS file reads, 0 OS file writes, 1 OS fsyncs\n",
"0.00 reads/s, 0 avg bytes/read, 0.00 writes/s, 0.00 fsyncs/s\n",
"---\n",
"LOG\n",
"---\n",
"Log sequence number 7704857\n",
"Log flushed up to 7704857\n",
"Pages flushed up to 7704857\n",
"Last checkpoint at 7704841\n",
"----------------------\n",
"BUFFER POOL AND MEMORY\n",
"----------------------\n",
"Total large memory allocated 167772160\n",
"Dictionary memory allocated 857728\n",
"Buffer pool size 8064\n",
"Free buffers 7756\n",
"Database pages 308\n",
"Old database pages 0\n",
"Modified db pages 0\n",
"Percent of dirty pages(LRU & free pages): 0.000\n",
"Max dirty pages percent: 90.000\n",
"Pending reads 0\n",
"Pending writes: LRU 0, flush list 0\n",
"Pages made young 0, not young 0\n",
"0.00 youngs/s, 0.00 non-youngs/s\n",
"Pages read 177, created 131, written 0\n",
"0.00 reads/s, 0.00 creates/s, 0.00 writes/s\n",
"Buffer pool hit rate 1000 / 1000, young-making rate 0 / 1000 not 0 / 1000\n",
"Pages read ahead 0.00/s, evicted without access 0.00/s, Random read ahead 0.00/s\n",
"LRU len: 308, unzip_LRU len: 0\n",
"I/O sum[0]:cur[0], unzip sum[0]:cur[0]\n",
"--------------\n",
"ROW OPERATIONS\n",
"--------------\n",
"0 read views open inside InnoDB\n",
"state: sleeping\n",
"----------------------------\n",
"END OF INNODB MONITOR OUTPUT\n",
"============================\n",
"\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "79da27ff-0406-44b8-a382-74051d4280ac",
"metadata": {},
"source": [
"## Seeking Help\n",
"\n",
"Where to get help:\n",
" \n",
"* https://mariadb.com/kb\n",
"* https://stackoverflow.com/ (for SQL questions)\n",
"* https://dba.stackexchange.com/ (for DBA tuning and adminstration questions)\n",
"* https://mariadb.zulipchat.com"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3af1d4f0-7e63-415c-a7b6-a2e41afbe2d6",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
Expand Down

0 comments on commit 8d3d5c4

Please sign in to comment.