Mysql 5 7中資料量更改統計資料收集的邏輯

2021-09-22 17:03:06 字數 4790 閱讀 1822

今天乙個朋友在問mysql資料修改後什麼時候收集統計資料,我就簡單的找了一下源**,現總結如下。如有錯誤請指出,因為我只是簡單做了一下除錯。

mysql統計資料分為持久化和非持久化

我在看更改邏輯的時候,發現非持久化統計資料是直接更新的或者說是同步更新的。但是持久化統計資料應該是非同步更新的,看起來是典型的生產者消費者模型。mysql中有乙個專門的後台執行緒來負責收集統計資料如下:

mysql> select name,thread_id from performance_schema.threads where name like '%dict_stats_thread%' \g

*************************** 1. row ***************************

name: thread/innodb/dict_stats_thread

thread_id: 25

這個執行緒會從乙個vcoter陣列中取出需要更新統計資料的表的table_id然後更新統計資料,而生產者就是我們的當前執行緒他通過邏輯判斷將需要更新統計資料表的table_id加入到這個陣列,什麼邏輯呢?我們稍後在討論。現在來看乙個簡單的圖:

其實更改邏輯還是相當簡單基本就是兩個函式就總結了,他們是row_update_statistics_if_needed函式和dict_stats_process_entry_from_recalc_pool函式,但是我不知道有沒有漏掉的。

持久化邏輯(persistent)

if (dict_stats_is_persistent_enabled(table)) 

return;

}

if (ut_difftime(ut_time(), table->stats_last_recalc)

< min_recalc_interval) else

其中min_recalc_interval 為巨集定義

#define min_recalc_interval    10 /* seconds */
非持久化邏輯(transient)
if (counter > 16 + n_rows / 16 /* 6.25% */)
14.6.12 configuring optimizer statistics for innodb
這是留個我自己除錯用的

#0  dict_stats_recalc_pool_add (table=0x7fff1805a790) at /root/mysql5.7.14/percona-server-5.7.14-7/storage/innobase/dict/dict0stats_bg.cc:129

#1 0x0000000001b0f4e3 in row_update_statistics_if_needed (table=0x7fff1805a790) at /root/mysql5.7.14/percona-server-5.7.14-7/storage/innobase/row/row0mysql.cc:1206

#2 0x0000000001b10c34 in row_insert_for_mysql_using_ins_graph (mysql_rec=0x7fff180479a0 "\375\036", prebuilt=0x7fff18098c10)

at /root/mysql5.7.14/percona-server-5.7.14-7/storage/innobase/row/row0mysql.cc:1831

#3 0x0000000001b10cbf in row_insert_for_mysql (mysql_rec=0x7fff180479a0 "\375\036", prebuilt=0x7fff18098c10)

at /root/mysql5.7.14/percona-server-5.7.14-7/storage/innobase/row/row0mysql.cc:1857

#4 0x00000000019b04dd in ha_innobase::write_row (this=0x7fff18047460, record=0x7fff180479a0 "\375\036")

at /root/mysql5.7.14/percona-server-5.7.14-7/storage/innobase/handler/ha_innodb.cc:7923

#5 0x0000000000f732af in handler::ha_write_row (this=0x7fff18047460, buf=0x7fff180479a0 "\375\036") at /root/mysql5.7.14/percona-server-5.7.14-7/sql/handler.cc:8228

#6 0x00000000017d1764 in write_record (thd=0x7fff18000b70, table=0x7fff18036420, info=0x7ffff0255a30, update=0x7ffff02559b0)

at /root/mysql5.7.14/percona-server-5.7.14-7/sql/sql_insert.cc:1864

#7 0x00000000017ce909 in sql_cmd_insert::mysql_insert (this=0x7fff18006b00, thd=0x7fff18000b70, table_list=0x7fff18006570)

at /root/mysql5.7.14/percona-server-5.7.14-7/sql/sql_insert.cc:780

#8 0x00000000017d5349 in sql_cmd_insert::execute (this=0x7fff18006b00, thd=0x7fff18000b70) at /root/mysql5.7.14/percona-server-5.7.14-7/sql/sql_insert.cc:3100

#9 0x00000000015a773e in mysql_execute_command (thd=0x7fff18000b70, first_level=true) at /root/mysql5.7.14/percona-server-5.7.14-7/sql/sql_parse.cc:3719

#10 0x00000000015adcae in mysql_parse (thd=0x7fff18000b70, parser_state=0x7ffff0257600) at /root/mysql5.7.14/percona-server-5.7.14-7/sql/sql_parse.cc:5836

#11 0x00000000015a1b6d in dispatch_command (thd=0x7fff18000b70, com_data=0x7ffff0257d70, command=com_query)

at /root/mysql5.7.14/percona-server-5.7.14-7/sql/sql_parse.cc:1447

#12 0x00000000015a099e in do_command (thd=0x7fff18000b70) at /root/mysql5.7.14/percona-server-5.7.14-7/sql/sql_parse.cc:1010

#13 0x00000000016e28f0 in handle_connection (arg=0x68d7220) at /root/mysql5.7.14/percona-server-5.7.14-7/sql/conn_handler/connection_handler_per_thread.cc:312

#14 0x0000000001d7a514 in pfs_spawn_thread (arg=0x37fcbc0) at /root/mysql5.7.14/percona-server-5.7.14-7/storage/perfschema/pfs.cc:2188

#15 0x0000003f74807aa1 in start_thread () from /lib64/libpthread.so.0

#16 0x0000003f740e8bcd in clone () from /lib64/libc.so.6

#0  dict_stats_update (table=0x7fff1805a790, stats_upd_option=dict_stats_recalc_persistent)

at /root/mysql5.7.14/percona-server-5.7.14-7/storage/innobase/dict/dict0stats.cc:3052

#1 0x0000000001cd0362 in dict_stats_process_entry_from_recalc_pool () at /root/mysql5.7.14/percona-server-5.7.14-7/storage/innobase/dict/dict0stats_bg.cc:349

#2 0x0000000001cd05a9 in dict_stats_thread (arg=0x0) at /root/mysql5.7.14/percona-server-5.7.14-7/storage/innobase/dict/dict0stats_bg.cc:438

#3 0x0000003f74807aa1 in start_thread () from /lib64/libpthread.so.0

#4 0x0000003f740e8bcd in clone () from /lib64/libc.so.6

MySQL5 7 union all 大資料量問題

資料庫有3張表,如chinese score math score english score,分別代表學生的語文成績 數學成績 英語成績,每張表的表結構和資料量一致,200萬資料量。表結構如下 id 主鍵 code 學號 name 姓名 score 成績 執行的sql 獲得每個學生的總成績 sel...

Centos 更改MySQL5 7資料庫目錄位置

centos7.3 安裝mysql5.7並修改初始密碼 基於 centos mysql 安裝與主從同步配置詳解 centos 通過yum安裝 rpm分發進行安裝 mysql的幾個人預設目錄如下 目錄目錄內容 usr bin 客戶端程式和指令碼 usr sbin mysqld伺服器 var lib m...

mysql 根據條件統計資料量

目錄 需求 統計總數,未完成數和完成數 方法一 方法二 總結 create table test task id int 11 not null auto increment,done tinyint 1 default null comment 是否完成 donetime int 11 defau...