大数据&云计算7月数据库技术通讯:LINUX OS配置问题导致数据库重启


以下截取部分页面:
大数据&云计算7月数据库技术通讯:LINUX OS配置问题导致数据库重启
本文插图

大数据&云计算7月数据库技术通讯:LINUX OS配置问题导致数据库重启
本文插图
以下选取一个经验篇:
ORACLE RAC经常会因为节点间通信或者数据库负载异常 , 从而导致数据库节点发生异常重启 , 影响生产环境部分业务 , 该篇文章就详细描述了数据库节点重启的分析过程 。
问题描述
某客户数据库Oracle RAC 环境中节点出现重启现象 , 几分钟后集群自动恢复 , 影响部分业务 , 分析思路如下:
问题分析
CRS日志分析 , 数据库无法启动 , 数据库监听状态异常:
2020-06-15 05:32:29.990:[/oraapp/11.2.0/grid/bin/oraagent.bin(17438)]CRS-5011:Check of resource "testrac" failed: details at "(:CLSN00007:)" in "/oraapp/11.2.0/grid/log/testrac1/agent/crsd/oraagent_oracle/oraagent_oracle.log"2020-06-15 05:32:49.078:[cssd(8806)]CRS-1662:Member kill requested by node testrac2 for member number 0, group DBtestrac2020-06-15 05:33:19.742:[cssd(8806)]CRS-1608:This node was evicted by node 2, testrac2 details at (:CSSNM00005:) in /oraapp/11.2.0/grid/log/testrac1/cssd/ocssd.log.2020-06-15 05:33:19.742:[cssd(8806)]CRS-1656:The CSS daemon is terminating due to a fatal error Details at (:CSSSC00012:) in /oraapp/11.2.0/grid/log/testrac1/cssd/ocssd.log2020-06-15 05:33:19.742:[cssd(8806)]CRS-1652:Starting clean up of CRSD resources.2020-06-15 05:33:19.835:[cssd(8806)]CRS-1608:This node was evicted by node 2, testrac2 details at (:CSSNM00005:) in /oraapp/11.2.0/grid/log/testrac1/cssd/ocssd.log.2020-06-15 05:33:19.835:[cssd(8806)]CRS-1604:CSSD voting file is offline: /dev/raw/raw1 details at (:CSSNM00058:) in /oraapp/11.2.0/grid/log/testrac1/cssd/ocssd.log.2020-06-15 05:33:19.835:[cssd(8806)]CRS-1604:CSSD voting file is offline: /dev/raw/raw3 details at (:CSSNM00058:) in /oraapp/11.2.0/grid/log/testrac1/cssd/ocssd.log.2020-06-15 05:33:19.835:[cssd(8806)]CRS-1604:CSSD voting file is offline: /dev/raw/raw4 details at (:CSSNM00058:) in /oraapp/11.2.0/grid/log/testrac1/cssd/ocssd.log.2020-06-15 05:33:21.184:[/oraapp/11.2.0/grid/bin/oraagent.bin(14932)]CRS-5016:Process "/oraapp/11.2.0/grid/opmn/bin/onsctli" spawned by agent "/oraapp/11.2.0/grid/bin/oraagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/oraapp/11.2.0/grid/log/testrac1/agent/crsd/oraagent_grid//oraagent_grid.log"2020-06-15 05:33:21.787:[/oraapp/11.2.0/grid/bin/oraagent.bin(14932)]CRS-5016:Process "/oraapp/11.2.0/grid/bin/lsnrctl" spawned by agent "/oraapp/11.2.0/grid/bin/oraagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/oraapp/11.2.0/grid/log/testrac1/agent/crsd/oraagent_grid//oraagent_grid.log"2020-06-15 05:33:21.789:[/oraapp/11.2.0/grid/bin/oraagent.bin(14932)]CRS-5016:Process "/oraapp/11.2.0/grid/bin/lsnrctl" spawned by agent "/oraapp/11.2.0/grid/bin/oraagent.bin" for action "check" failed: details at "(:CLSN00010:)" in "/oraapp/11.2.0/grid/log/testrac1/agent/crsd/oraagent_grid//oraagent_grid.log"2020-06-15 05:33:22.052:[cssd(8806)]CRS-1654:Clean up of CRSD resources finished successfully.2020-06-15 05:33:22.053:[cssd(8806)]CRS-1655:CSSD on node testrac1 detected a problem and started to shutdown.


推荐阅读