MySQL在生产环境出现无法启动的问题解决
作者:XuanRanDev
MySQL在生产环境出现无法启动的问题解决
1、事由
今天现服务器重启了,然后服务器启动完成之后我发现为啥后端程序没有启动,排查之后发现是MySQL没有启动连接不上数据库,错误信息如下:
2024-04-16T02:34:01.507696Z 0 [Warning] [MY-000081] [Server] option 'max_allowed_packet': unsigned value 107374182400 adjusted to 1073741824. 2024-04-16T02:34:01.507749Z 0 [Warning] [MY-011070] [Server] 'binlog_format' is deprecated and will be removed in a future release. 2024-04-16T02:34:01.507821Z 0 [Warning] [MY-010915] [Server] 'NO_ZERO_DATE', 'NO_ZERO_IN_DATE' and 'ERROR_FOR_DIVISION_BY_ZERO' sql modes should be used with strict mode. They will be merged with strict mode in a future release. 2024-04-16T02:34:01.507923Z 0 [Warning] [MY-010918] [Server] 'default_authentication_plugin' is deprecated and will be removed in a future release. Please use authentication_policy instead. 2024-04-16T02:34:01.507955Z 0 [System] [MY-010116] [Server] /www/server/mysql/bin/mysqld (mysqld 8.0.36) starting as process 7075 2024-04-16T02:34:01.524054Z 0 [Warning] [MY-013907] [InnoDB] Deprecated configuration parameters innodb_log_file_size and/or innodb_log_files_in_group have been used to compute innodb_redo_log_capacity=1073741824. Please use innodb_redo_log_capacity instead. 2024-04-16T02:34:01.526764Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started. 2024-04-16T02:34:02.677451Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended. 2024-04-16T02:34:02.975637Z 0 [ERROR] [MY-013183] [InnoDB] Assertion failure: trx0types.h:541:m_rsegs_n < 2 thread 47034609473280 InnoDB: We intentionally generate a memory trap. InnoDB: Submit a detailed bug report to http://bugs.mysql.com. InnoDB: If you get repeated assertion failures or crashes, even InnoDB: immediately after the mysqld startup, there may be InnoDB: corruption in the InnoDB tablespace. Please refer to InnoDB: http://dev.mysql.com/doc/refman/8.0/en/forcing-innodb-recovery.html InnoDB: about forcing recovery. 2024-04-16T02:34:02Z UTC - mysqld got signal 6 ; Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware. BuildID[sha1]=911887188a59108d0b2707ced3fa0b5872644b4f Thread pointer: 0x2ac79c0008c0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 2ac7193089e8 thread_stack 0x100000 /www/server/mysql/bin/mysqld(my_print_stacktrace(unsigned char const*, unsigned long)+0x2e) [0x1fe320e] /www/server/mysql/bin/mysqld(print_fatal_signal(int)+0x3a3) [0x107e883] /www/server/mysql/bin/mysqld(my_server_abort()+0x5e) [0x107e98e] /www/server/mysql/bin/mysqld(my_abort()+0xa) [0x1fdd96a] /www/server/mysql/bin/mysqld(ut_dbg_assertion_failed(char const*, char const*, unsigned long)+0x30c) [0x22259ec] /www/server/mysql/bin/mysqld(TrxUndoRsegsIterator::set_next()+0x5f1) [0x21e6dd1] /www/server/mysql/bin/mysqld() [0x21e6e5f] /www/server/mysql/bin/mysqld() [0x21e7dc8] /www/server/mysql/bin/mysqld(trx_purge(unsigned long, unsigned long, bool)+0x13d) [0x21eabdd] /www/server/mysql/bin/mysqld(srv_purge_coordinator_thread()+0xb72) [0x21c3882] /www/server/mysql/bin/mysqld(std::thread::_State_impl<std::thread::_Invoker<std::tuple<Detached_thread, void (*)()> > >::_M_run()+0xb4) [0x20e3374] /www/server/mysql/bin/mysqld() [0x280f0cf] /lib64/libpthread.so.0(+0x7ea5) [0x2ac6e6256ea5] /lib64/libc.so.6(clone+0x6d) [0x2ac6e7d0eb0d] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0): is an invalid pointer Connection ID (thread ID): 0 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash.
百度搜了一下之后说可能是执行的SQL有问题,然后我就想看下MySQL的BinLog日志,但是在服务器执行的时候发现:
但是我明明是有mysql的,mysql和mysqlbinlog正常来说都是在一起的,一番查找之后原因是:我用了宝塔,宝塔的mysqlbinlog日志地址是
/www/server/mysql/bin
这里面的
2、分析binlog日志
./mysqlbinlog --start-datetime=“2024-04-16 02:00:00” /home/mysql/mysql-bin.000007
大概看了binlog日志,发现最后几条的sql并没有问题,然后我就觉得大概率是因为服务器是突然断点重启的,但是此时正在执行sql,造成innodb数据库文件出现错误的原因
(已脱敏数据)
#240416 2:00:56 server id 1 end_log_pos 651706988 CRC32 0xecb191f3 Xid = 25647715 COMMIT/*!*/; # at 651706988 #240416 2:00:56 server id 1 end_log_pos 651707067 CRC32 0x8d36daa2 Anonymous_GTID last_committed=634356 sequence_number=634357 rbr_only=no original_committed_timestamp=1713204056015478 immediate_commit_timestamp=1713204056015478 transaction_length=934 # original_commit_timestamp=1713204056015478 (2024-04-16 02:00:56.015478 CST) # immediate_commit_timestamp=1713204056015478 (2024-04-16 02:00:56.015478 CST) /*!80001 SET @@session.original_commit_timestamp=1713204056015478*//*!*/; /*!80014 SET @@session.original_server_version=80036*//*!*/; /*!80014 SET @@session.immediate_server_version=80036*//*!*/; SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/; # at 651707067 #240416 2:00:56 server id 1 end_log_pos 651707162 CRC32 0x156f48b9 Query thread_id=2911 exec_time=0 error_code=0 SET TIMESTAMP=1713204056/*!*/; BEGIN /*!*/; # at 651707162 #240416 2:00:56 server id 1 end_log_pos 651707435 CRC32 0x9dd48b58 Query thread_id=2911 exec_time=0 error_code=0 SET TIMESTAMP=1713204056/*!*/; UPDATE QRTZ_TRIGGERS SET TRIGGER_STATE = 'ACQUIRED' WHERE SCHED_NAME = 'schedulerName' AND TRIGGER_NAME AND TRIGGTE = 'WAITING' /*!*/; # at 651707435 #240416 2:00:56 server id 1 end_log_pos 651707891 CRC32 0x35e6afbf Query thread_id=2911 exec_time=0 error_code=0 SET TIMESTAMP=1713204056/*!*/; INSERT INTO QRTZ_FIRED_TRIGGERS (SCHED_NAME, ENTRY_ID, TRIGGER_NAME, TRIGGER_GROUP, INSTANCE_NAME, FIRED_TIME 5) /*!*/; # at 651707891 #240416 2:00:56 server id 1 end_log_pos 651707922 CRC32 0x451c13c4 Xid = 25647718 COMMIT/*!*/; # at 651707922 #240416 2:00:56 server id 1 end_log_pos 651708001 CRC32 0xae4efe8c Anonymous_GTID last_committed=634357 sequence_number=634358 rbr_only=no original_committed_timestamp=1713204056017004 immediate_commit_timestamp=1713204056017004 transaction_length=548 # original_commit_timestamp=1713204056017004 (2024-04-16 02:00:56.017004 CST) # immediate_commit_timestamp=1713204056017004 (2024-04-16 02:00:56.017004 CST) /*!80001 SET @@session.original_commit_timestamp=1713204056017004*//*!*/; /*!80014 SET @@session.original_server_version=80036*//*!*/; /*!80014 SET @@session.immediate_server_version=80036*//*!*/; SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/; # at 651708001 #240416 2:00:56 server id 1 end_log_pos 651708118 CRC32 0x1f8b6580 Query thread_id=2882 exec_time=0 error_code=0 SET TIMESTAMP=1713204056/*!*/; BEGIN /*!*/; # at 651708118 #240416 2:00:56 server id 1 end_log_pos 651708439 CRC32 0xb4b71846 Query thread_id=2882 exec_time=0 error_code=0 SET TIMESTAMP=1713204056/*!*/; UPDATE infra_job_log SET end_time='2024-04-16 02:00:56.015004', duration=5, status=1, result='执行支付通知 0 个', update_time='2024-04-16 02:00:56.015661', updater=null WHERE id=383950 AND deleted=0 /*!*/; # at 651708439 #240416 2:00:56 server id 1 end_log_pos 651708470 CRC32 0x93c3d766 Xid = 25647730 COMMIT/*!*/; # at 651708470 #240416 2:00:56 server id 1 end_log_pos 651708549 CRC32 0x75b1af91 Anonymous_GTID last_committed=634358 sequence_number=634359 rbr_only=no original_committed_timestamp=1713204056020270 immediate_commit_timestamp=1713204056020270 transaction_length=938 # original_commit_timestamp=1713204056020270 (2024-04-16 02:00:56.020270 CST) # immediate_commit_timestamp=1713204056020270 (2024-04-16 02:00:56.020270 CST) /*!80001 SET @@session.original_commit_timestamp=1713204056020270*//*!*/; /*!80014 SET @@session.original_server_version=80036*//*!*/; /*!80014 SET @@session.immediate_server_version=80036*//*!*/; SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/; # at 651708549 #240416 2:00:56 server id 1 end_log_pos 651708644 CRC32 0x82977794 Query thread_id=2915 exec_time=0 error_code=0 SET TIMESTAMP=1713204056/*!*/; BEGIN /*!*/; # at 651708644 #240416 2:00:56 server id 1 end_log_pos 651708905 CRC32 0xa05db17c Query thread_id=2915 exec_time=0 error_code=0 SET TIMESTAMP=1713204056/*!*/; UPDATE QRTZ_TRIGGERS SET TRIG'BLOCKED' /*!*/; # at 651708905 #240416 2:00:56 server id 1 end_log_pos 651709172 CRC32 0xf8de225b Query thread_id=2915 exec_time=0 error_code=0 SET TIMESTAMP=1713204056/*!*/; UPDATE QRTZ_TRIGGERSE = 'PAUSED_BLOCKED' /*!*/; # at 651709172 #240416 2:00:56 server id 1 end_log_pos 651709377 CRC32 0xb4ef1017 Query thread_id=2915 exec_time=0 error_code=0 SET TIMESTAMP=1713204056/*!*/; DELETE FROM QRTZ_FIRED_TRIGGERS WHERE SCHED_NAME = 'schedulerName' AND /*!*/; # at 651709377 #240416 2:00:56 server id 1 end_log_pos 651709408 CRC32 0xfc3cd382 Xid = 25647729 COMMIT/*!*/; # at 651709408 #240416 2:00:56 server id 1 end_log_pos 651709487 CRC32 0x90969279 Anonymous_GTID last_committed=634359 sequence_number=634360 rbr_only=no original_committed_timestamp=1713204056026583 immediate_commit_timestamp=1713204056026583 transaction_length=955 # original_commit_timestamp=1713204056026583 (2024-04-16 02:00:56.026583 CST) # immediate_commit_timestamp=1713204056026583 (2024-04-16 02:00:56.026583 CST) /*!80001 SET @@session.original_commit_timestamp=1713204056026583*//*!*/; /*!80014 SET @@session.original_server_version=80036*//*!*/; /*!80014 SET @@session.immediate_server_version=80036*//*!*/; SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/; # at 651709487 #240416 2:00:56 server id 1 end_log_pos 651709582 CRC32 0x954d1385 Query thread_id=2882 exec_time=0 error_code=0 SET TIMESTAMP=1713204056/*!*/; BEGIN /*!*/; # at 651709582 #240416 2:00:56 server id 1 end_log_pos 651709855 CRC32 0x4443046e Query thread_id=2882 exec_time=0 error_code=0 SET TIMESTAMP=1713204056/*!*/; UPDATE QRTZ_TRIGGERS SET TRIGGER_STATE = 'WAITISTATE = 'ACQUIRED' /*!*/; # at 651709855 #240416 2:00:56 server id 1 end_log_pos 651710127 CRC32 0x2a9b0ef0 Query thread_id=2882 exec_time=0 error_code=0 SET TIMESTAMP=1713204056/*!*/; UPDATE QRTZ_TRIGGERS /*!*/; # at 651710127 #240416 2:00:56 server id 1 end_log_pos 651710332 CRC32 0xd8fea345 Query thread_id=2882 exec_time=0 error_code=0 SET TIMESTAMP=1713204056/*!*/; DELETE FROM QRTZ_FIRED_TR /*!*/; # at 651710332 #240416 2:00:56 server id 1 end_log_pos 651710363 CRC32 0xc4b82dea Xid = 25647738 COMMIT/*!*/; # at 651710363 #240416 2:00:56 server id 1 end_log_pos 651710442 CRC32 0x1990f6bf Anonymous_GTID last_committed=634360 sequence_number=634361 rbr_only=no original_committed_timestamp=1713204056032340 immediate_commit_timestamp=1713204056032340 transaction_length=928 # original_commit_timestamp=1713204056032340 (2024-04-16 02:00:56.032340 CST) # immediate_commit_timestamp=1713204056032340 (2024-04-16 02:00:56.032340 CST) /*!80001 SET @@session.original_commit_timestamp=1713204056032340*//*!*/; /*!80014 SET @@session.original_server_version=80036*//*!*/; /*!80014 SET @@session.immediate_server_version=80036*//*!*/; SET @@SESSION.GTID_NEXT= 'ANONYMOUS'/*!*/; # at 651710442 #240416 2:00:56 server id 1 end_log_pos 651710537 CRC32 0x15fddec2 Query thread_id=2882 exec_time=0 error_code=0 SET TIMESTAMP=1713204056/*!*/; BEGIN /*!*/; # at 651710537 #240416 2:00:56 server id 1 end_log_pos 651710807 CRC32 0x87d15abb Query thread_id=2882 exec_time=0 error_code=0 SET TIMESTAMP=1713204056/*!*/; UPDATE QRTZ_TRIGGERS SET TRSTATE = 'WAITING' /*!*/; # at 651710807 #240416 2:00:56 server id 1 end_log_pos 651711260 CRC32 0xb988e5b0 Query thread_id=2882 exec_time=0 error_code=0 SET TIMESTAMP=1713204056/*!*/; INSERT INTO /*!*/; # at 651711260 #240416 2:00:56 server id 1 end_log_pos 651711291 CRC32 0x889caa0e Xid = 25647746 COMMIT/*!*/; SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/; DELIMITER ; # End of log file /*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/; /*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
3、数据恢复
既然原因大概就是这样,我们现在得解决数据恢复的问题,既然是innodb文件的问题,那这个库我的想法就是别用了,导出原有数据库的文件,然后新建一个库,毕竟Innodb的文件我也实在不知道怎么修复,但是现在由于mysql启动不了无法导出数据,按照错误信息的提示,我们在mysql的配置文件中加入以下内容:
# 强制启用恢复模式 innodb_force_recovery = 1
其中参数的含义是:
1
(SRV_FORCE_IGNORE_CORRUPT)
即使检测到损坏的页面,也允许服务器运行。尝试使 SELECT * FROM *
tbl_name*
跳过损坏的索引记录和页面,这有助于转储表。
2
(SRV_FORCE_NO_BACKGROUND)
阻止主线程和任何清除线程运行。如果在清除操作期间发生意外退出,则此恢复值会阻止它。
3
(SRV_FORCE_NO_TRX_UNDO)
崩溃恢复后不运行事务回滚.
4
(SRV_FORCE_NO_IBUF_MERGE)
阻止插入缓冲区合并操作。如果它们会导致崩溃,请不要这样做。不计算表统计信息。此值可能会永久损坏数据文件。使用此值后,请准备好删除并重新创建所有二级索引。将 InnoDB
设置为只读。
5
(SRV_FORCE_NO_UNDO_LOG_SCAN)
启动数据库时不查看撤消日志:InnoDB
甚至将未完成的事务视为已提交。此值可能会永久损坏数据文件。将 InnoDB
设置为只读。
6
(SRV_FORCE_NO_LOG_REDO)
不执行与恢复相关的重做日志前滚。此值可能会永久损坏数据文件。使数据库页处于过时状态,这反过来又可能会给 B 树和其他数据库结构带来更多损坏。将 InnoDB
设置为只读。
建议从1开始逐步往上尝试启动,我是尝试到4之后才启动成功的。
启动成功之后我们此时不要就这样用了!因为我们使用的是恢复模式运行的,顾名思义就是用来恢复数据的!
启动成功后我们在使用dump导出一下数据库的结构和数据(我用的是宝塔就直接点击备份数据了)
备份成功之后我们就可以删除原来的数据库了,因为原来数据库的innodb文件已经损坏了,使用宝塔也删不掉这个数据库,我就想直接删除MySQL的数据文件
4、删除原来数据库
由于使用的是恢复模式,而且innodb的文件已经损坏了,所以我们可以找到mysql配置文件,找到对应的数据文件,然后使用rm -rf 文件删除掉原有的数据库文件。
数据无价!!!
确保已备份全部数据!!!!!!!!
# 删除宝塔的MySQL数据 rm -rf /www/server/mysql
删除完之后,我们重新安装数据库,接着把直接备份的数据库重新导入,MySQL就可以正常启动了。
以上就是MySQL在生产环境出现无法启动的问题解决的详细内容,更多关于MySQL生产环境无法启动的资料请关注脚本之家其它相关文章!