虚拟机模拟部署Extended Clusters(四)故障模拟测试,存储链路恢复

  1. 云栖社区>
  2. 博客>
  3. 正文

虚拟机模拟部署Extended Clusters(四)故障模拟测试,存储链路恢复

snowofsummer 2019-09-17 15:04:44 浏览3927
展开阅读全文

asm 磁盘组

当链路恢复之后,磁盘状态显示MISSING(CRS_0000,OCR_0000)。

[grid@prod02 ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.4.0 Production on Tue Sep 17 14:49:06 2019

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> @c

DG_NAME     DG_STATE   TYPE       DSK_NO DSK_NAME    PATH                           MOUNT_S FAILGROUP        STATE
--------------- ---------- ------ ---------- ---------- -------------------------------------------------- ------- -------------------- --------
CRS        MOUNTED    NORMAL       2 CRS_0002    /dev/oracleasm/disks/DISK03               CACHED  ZCDISK        NORMAL
CRS        MOUNTED    NORMAL       5 CRS_0005    /dev/oracleasm/disks/VOTEDB02               CACHED  CRS_0001        NORMAL
CRS        MOUNTED    NORMAL       9 CRS_0009                               MISSING CRS_0000        NORMAL
OCR        MOUNTED    NORMAL       0 OCR_0000                               MISSING OCR_0000        NORMAL
OCR        MOUNTED    NORMAL       1 OCR_0001    /dev/oracleasm/disks/NODE02DATA01           CACHED  OCR_0001        NORMAL


DISK_NUMBER NAME       PATH                          HEADER_STATUS         OS_MB   TOTAL_MB    FREE_MB REPAIR_TIMER V FAILGRO
----------- ---------- -------------------------------------------------- -------------------- ---------- ---------- ---------- ------------ - -------
      0           /dev/oracleasm/disks/NODE01DATA01          MEMBER             5120       0          0        0 N REGULAR
      1           /dev/oracleasm/disks/VOTEDB01              MEMBER             2048       0          0        0 Y REGULAR
      0 OCR_0000                              UNKNOWN            0    5120       3185        12410 N REGULAR
      9 CRS_0009                              UNKNOWN            0    2048       1584        12410 N REGULAR
      1 OCR_0001   /dev/oracleasm/disks/NODE02DATA01          MEMBER             5120    5120       3185        0 N REGULAR
      5 CRS_0005   /dev/oracleasm/disks/VOTEDB02              MEMBER             2048    2048       1648        0 Y REGULAR
      2 CRS_0002   /dev/oracleasm/disks/DISK03              MEMBER             5115    5115       5081        0 Y QUORUM

7 rows selected.


GROUP_NUMBER NAME    COMPATIBILITY                             DATABASE_COMPATIBILITY                      V
------------ ---------- ------------------------------------------------------------ ------------------------------------------------------------ -
       1 OCR    11.2.0.0.0                             11.2.0.0.0                           N
       2 CRS    11.2.0.0.0                             11.2.0.0.0                           Y

SQL>

恢复方法(在存活节点操作)

[grid@prod02 ~]$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.4.0 Production on Tue Sep 17 14:51:45 2019

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> ALTER DISKGROUP crs online all;

Diskgroup altered.

SQL> ALTER DISKGROUP ocr online all;

Diskgroup altered.

SQL>

磁盘组状态显示:

[grid@prod02 ~]$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.4.0 Production on Tue Sep 17 14:52:55 2019

Copyright (c) 1982, 2013, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> @c

DG_NAME     DG_STATE   TYPE       DSK_NO DSK_NAME    PATH                           MOUNT_S FAILGROUP        STATE
--------------- ---------- ------ ---------- ---------- -------------------------------------------------- ------- -------------------- --------
CRS        MOUNTED    NORMAL       2 CRS_0002    /dev/oracleasm/disks/DISK03               CACHED  ZCDISK        NORMAL
CRS        MOUNTED    NORMAL       5 CRS_0005    /dev/oracleasm/disks/VOTEDB02               CACHED  CRS_0001        NORMAL
CRS        MOUNTED    NORMAL       9 CRS_0009    /dev/oracleasm/disks/VOTEDB01               CACHED  CRS_0000        NORMAL
OCR        MOUNTED    NORMAL       0 OCR_0000    /dev/oracleasm/disks/NODE01DATA01           CACHED  OCR_0000        NORMAL
OCR        MOUNTED    NORMAL       1 OCR_0001    /dev/oracleasm/disks/NODE02DATA01           CACHED  OCR_0001        NORMAL


DISK_NUMBER NAME       PATH                          HEADER_STATUS         OS_MB   TOTAL_MB    FREE_MB REPAIR_TIMER V FAILGRO
----------- ---------- -------------------------------------------------- -------------------- ---------- ---------- ---------- ------------ - -------
      0 OCR_0000   /dev/oracleasm/disks/NODE01DATA01          MEMBER             5120    5120       3184        0 N REGULAR
      1 OCR_0001   /dev/oracleasm/disks/NODE02DATA01          MEMBER             5120    5120       3184        0 N REGULAR
      9 CRS_0009   /dev/oracleasm/disks/VOTEDB01              MEMBER             2048    2048       1552        0 Y REGULAR
      5 CRS_0005   /dev/oracleasm/disks/VOTEDB02              MEMBER             2048    2048       1648        0 Y REGULAR
      2 CRS_0002   /dev/oracleasm/disks/DISK03              MEMBER             5115    5115       5081        0 Y QUORUM


GROUP_NUMBER NAME    COMPATIBILITY                             DATABASE_COMPATIBILITY                      V
------------ ---------- ------------------------------------------------------------ ------------------------------------------------------------ -
       1 OCR    11.2.0.0.0                             11.2.0.0.0                           N
       2 CRS    11.2.0.0.0                             11.2.0.0.0                           Y

SQL>

注意事项!!!,否则不能执行online 操作。

磁盘组属性11.2.0.0.0(DATABASE_COMPATIBILITY,COMPATIBILITY)

集群软件和数据库不需要操作,自动恢复。

[grid@prod01 ~]$ crsctl stat res -t 
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.CRS.dg
               ONLINE  ONLINE       prod01                                       
               ONLINE  ONLINE       prod02                                       
ora.LISTENER.lsnr
               ONLINE  ONLINE       prod01                                       
               ONLINE  ONLINE       prod02                                       
ora.OCR.dg
               ONLINE  ONLINE       prod01                                       
               ONLINE  ONLINE       prod02                                       
ora.asm
               ONLINE  ONLINE       prod01                   Started             
               ONLINE  ONLINE       prod02                   Started             
ora.gsd
               OFFLINE OFFLINE      prod01                                       
               OFFLINE OFFLINE      prod02                                       
ora.net1.network
               ONLINE  ONLINE       prod01                                       
               ONLINE  ONLINE       prod02                                       
ora.ons
               ONLINE  ONLINE       prod01                                       
               ONLINE  ONLINE       prod02                                       
ora.registry.acfs
               ONLINE  ONLINE       prod01                                       
               ONLINE  ONLINE       prod02                                       
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       prod02                                       
ora.cvu
      1        ONLINE  ONLINE       prod02                                       
ora.oc4j
      1        ONLINE  ONLINE       prod02                                       
ora.ora.db
      1        ONLINE  ONLINE       prod01                   Open                
      2        ONLINE  ONLINE       prod02                   Open                
ora.prod01.vip
      1        ONLINE  ONLINE       prod01                                       
ora.prod02.vip
      1        ONLINE  ONLINE       prod02                                       
ora.scan1.vip
      1        ONLINE  ONLINE       prod02                                       
[grid@prod01 ~]$

prod01 grid log:

2019-09-17 14:49:12.688: 
[/u01/app/11.2.0/grid/bin/cssdagent(42554)]CRS-5818:Aborted command 'start' for resource 'ora.cssd'. Details at (:CRSAGF00113:) {0:15:28} in /u01/app/11.2.0/grid/log/prod01/agent/ohasd/oracssdagent_root/oracssdagent_root.log.
2019-09-17 14:49:13.787: 
[ohasd(36900)]CRS-2878:Failed to restart resource 'ora.cssd'
2019-09-17 14:49:13.788: 
[ohasd(36900)]CRS-2769:Unable to failover resource 'ora.cssd'.
2019-09-17 14:49:15.267: 
[cssd(44154)]CRS-1713:CSSD daemon is started in clustered mode
2019-09-17 14:49:26.061: 
[cssd(44154)]CRS-1707:Lease acquisition for node prod01 number 1 completed
2019-09-17 14:49:27.365: 
[cssd(44154)]CRS-1605:CSSD voting file is online: /dev/oracleasm/disks/VOTEDB02; details in /u01/app/11.2.0/grid/log/prod01/cssd/ocssd.log.
2019-09-17 14:49:27.373: 
[cssd(44154)]CRS-1605:CSSD voting file is online: /dev/oracleasm/disks/DISK03; details in /u01/app/11.2.0/grid/log/prod01/cssd/ocssd.log.
2019-09-17 14:49:29.932: 
[cssd(44154)]CRS-1601:CSSD Reconfiguration complete. Active nodes are prod01 prod02 .
2019-09-17 14:49:31.272: 
[ohasd(36900)]CRS-2878:Failed to restart resource 'ora.cssd'
2019-09-17 14:49:32.293: 
[ctssd(44276)]CRS-2403:The Cluster Time Synchronization Service on host prod01 is in observer mode.
2019-09-17 14:49:32.600: 
[ctssd(44276)]CRS-2407:The new Cluster Time Synchronization Service reference node is host prod02.
2019-09-17 14:49:32.602: 
[ctssd(44276)]CRS-2401:The Cluster Time Synchronization Service started on host prod01.
2019-09-17 14:49:34.282: 
[ohasd(36900)]CRS-2767:Resource state recovery not attempted for 'ora.diskmon' as its target state is OFFLINE
2019-09-17 14:49:34.282: 
[ohasd(36900)]CRS-2769:Unable to failover resource 'ora.diskmon'.
2019-09-17 14:49:41.580: 
[/u01/app/11.2.0/grid/bin/oraagent.bin(37016)]CRS-5011:Check of resource "+ASM" failed: details at "(:CLSN00006:)" in "/u01/app/11.2.0/grid/log/prod01/agent/ohasd/oraagent_grid/oraagent_grid.log"
2019-09-17 14:49:42.635: 
[ctssd(44276)]CRS-2409:The clock on host prod01 is not synchronous with the mean cluster time. No action has been taken as the Cluster Time Synchronization Service is running in observer mode.
2019-09-17 14:49:46.814: 
[/u01/app/11.2.0/grid/bin/oraagent.bin(37016)]CRS-5011:Check of resource "+ASM" failed: details at "(:CLSN00006:)" in "/u01/app/11.2.0/grid/log/prod01/agent/ohasd/oraagent_grid/oraagent_grid.log"
2019-09-17 14:49:58.993: 
[ohasd(36900)]CRS-2878:Failed to restart resource 'ora.asm'
2019-09-17 14:50:00.024: 
[ohasd(36900)]CRS-2878:Failed to restart resource 'ora.crsd'
2019-09-17 14:50:05.917: 
[crsd(44448)]CRS-1012:The OCR service started on node prod01.
2019-09-17 14:50:05.943: 
[evmd(44295)]CRS-1401:EVMD started on node prod01.
2019-09-17 14:50:07.250: 
[crsd(44448)]CRS-1201:CRSD started on node prod01.
2019-09-17 14:50:08.757: 
[/u01/app/11.2.0/grid/bin/oraagent.bin(44611)]CRS-5011:Check of resource "ora" failed: details at "(:CLSN00007:)" in "/u01/app/11.2.0/grid/log/prod01/agent/crsd/oraagent_oracle/oraagent_oracle.log"
2019-09-17 14:52:02.203: 
[cssd(44154)]CRS-1605:CSSD voting file is online: /dev/oracleasm/disks/VOTEDB01; details in /u01/app/11.2.0/grid/log/prod01/cssd/ocssd.log.
2019-09-17 14:52:02.203: 
[cssd(44154)]CRS-1626:A Configuration change request completed successfully
2019-09-17 14:52:02.217: 
[cssd(44154)]CRS-1601:CSSD Reconfiguration complete. Active nodes are prod01 prod02 .

prod01 asm log:

Tue Sep 17 14:49:52 2019
NOTE: No asm libraries found in the system
MEMORY_TARGET defaulting to 1128267776.
* instance_number obtained from CSS = 1, checking for the existence of node 0... 
* node 0 does not exist. instance_number = 1 
Starting ORACLE instance (normal)
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Initial number of CPU is 8
Number of processor cores in the system is 8
Number of processor sockets in the system is 8
Private Interface 'eth1:1' configured from GPnP for use as a private interconnect.
  [name='eth1:1', type=1, ip=169.254.237.99, mac=00-0c-29-1e-dd-e5, net=169.254.0.0/16, mask=255.255.0.0, use=haip:cluster_interconnect/62]
Public Interface 'eth0' configured from GPnP for use as a public interface.
  [name='eth0', type=1, ip=192.168.0.230, mac=00-0c-29-1e-dd-db, net=192.168.0.0/24, mask=255.255.255.0, use=public/1]
CELL communication is configured to use 0 interface(s):
CELL IP affinity details:
    NUMA status: non-NUMA system
    cellaffinity.ora status: N/A
CELL communication will use 1 IP group(s):
    Grp 0: 
Picked latch-free SCN scheme 3
Using LOG_ARCHIVE_DEST_1 parameter default value as /u01/app/11.2.0/grid/dbs/arch
Autotune of undo retention is turned on. 
LICENSE_MAX_USERS = 0
SYS auditing is disabled
NOTE: Volume support  enabled
Starting up:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options.
ORACLE_HOME = /u01/app/11.2.0/grid
System name:    Linux
Node name:    prod01
Release:    2.6.32-642.el6.x86_64
Version:    #1 SMP Wed Apr 13 00:51:26 EDT 2016
Machine:    x86_64
VM name:    VMWare Version: 6
Using parameter settings in server-side spfile +CRS/prod-cluster01/asmparameterfile/registry.253.993388635
System parameters with non-default values:
  large_pool_size          = 12M
  instance_type            = "asm"
  remote_login_passwordfile= "EXCLUSIVE"
  asm_diskstring           = "/dev/oracleasm/disks/*"
  asm_diskgroups           = "VOTE"
  asm_diskgroups           = "OCR"
  asm_power_limit          = 1
  _asm_storagemaysplit     = TRUE
  diagnostic_dest          = "/u01/app/grid"
Cluster communication is configured to use the following interface(s) for this instance
  169.254.237.99
cluster interconnect IPC version:Oracle UDP/IP (generic)
IPC Vendor 1 proto 2
Tue Sep 17 14:49:53 2019
PMON started with pid=2, OS id=44340 
Tue Sep 17 14:49:53 2019
PSP0 started with pid=3, OS id=44342 
Tue Sep 17 14:49:54 2019
VKTM started with pid=4, OS id=44345 at elevated priority
VKTM running at (1)millisec precision with DBRM quantum (100)ms
Tue Sep 17 14:49:54 2019
GEN0 started with pid=5, OS id=44349 
Tue Sep 17 14:49:54 2019
DIAG started with pid=6, OS id=44351 
Tue Sep 17 14:49:54 2019
PING started with pid=7, OS id=44353 
Tue Sep 17 14:49:54 2019
DIA0 started with pid=8, OS id=44355 
Tue Sep 17 14:49:54 2019
LMON started with pid=9, OS id=44357 
Tue Sep 17 14:49:54 2019
LMD0 started with pid=10, OS id=44359 
* Load Monitor used for high load check 
* New Low - High Load Threshold Range = [7680 - 10240] 
Tue Sep 17 14:49:54 2019
LMS0 started with pid=11, OS id=44361 at elevated priority
Tue Sep 17 14:49:54 2019
LMHB started with pid=12, OS id=44365 
Tue Sep 17 14:49:54 2019
MMAN started with pid=13, OS id=44367 
Tue Sep 17 14:49:54 2019
DBW0 started with pid=14, OS id=44369 
Tue Sep 17 14:49:54 2019
LGWR started with pid=15, OS id=44371 
Tue Sep 17 14:49:54 2019
CKPT started with pid=16, OS id=44373 
Tue Sep 17 14:49:54 2019
SMON started with pid=17, OS id=44375 
Tue Sep 17 14:49:54 2019
RBAL started with pid=18, OS id=44377 
Tue Sep 17 14:49:54 2019
GMON started with pid=19, OS id=44379 
Tue Sep 17 14:49:54 2019
MMON started with pid=20, OS id=44381 
Tue Sep 17 14:49:54 2019
MMNL started with pid=21, OS id=44383 
lmon registered with NM - instance number 1 (internal mem no 0)
Reconfiguration started (old inc 0, new inc 40)
ASM instance 
List of instances:
 1 2 (myinst: 1) 
 Global Resource Directory frozen
* allocate domain 0, invalid = TRUE 
 Communication channels reestablished
* allocate domain 2, invalid = TRUE 
* allocate domain 1, invalid = TRUE 
 * domain 0 valid = 1 according to instance 2 
 * domain 2 valid = 1 according to instance 2 
 * domain 1 valid = 1 according to instance 2 
 Master broadcasted resource hash value bitmaps
 Non-local Process blocks cleaned out
 LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
 Set master node info 
 Submitted all remote-enqueue requests
 Dwn-cvts replayed, VALBLKs dubious
 All grantable enqueues granted
 Submitted all GCS remote-cache requests
 Fix write in gcs resources
Reconfiguration complete
Tue Sep 17 14:49:55 2019
LCK0 started with pid=22, OS id=44386 
ORACLE_BASE not set in environment. It is recommended
that ORACLE_BASE be set in the environment
Tue Sep 17 14:49:57 2019
SQL> ALTER DISKGROUP ALL MOUNT /* asm agent call crs *//* {0:9:7} */ 
NOTE: Diskgroups listed in ASM_DISKGROUPS are
      VOTE
      OCR
NOTE: Diskgroup used for Voting files is:
      CRS
Diskgroup with spfile:CRS
Diskgroup used for OCR is:OCR
NOTE: cache registered group OCR number=1 incarn=0xf62b4c86
NOTE: cache began mount (not first) of group OCR number=1 incarn=0xf62b4c86
NOTE: cache registered group CRS number=2 incarn=0x801b4c85
NOTE: cache began mount (not first) of group CRS number=2 incarn=0x801b4c85
NOTE: cache registered group VOTE number=3 incarn=0xe5ab4c87
NOTE: cache began mount (first) of group VOTE number=3 incarn=0xe5ab4c87
NOTE: Assigning number (1,1) to disk (/dev/oracleasm/disks/NODE02DATA01)
NOTE: Assigning number (1,0) to disk (/dev/oracleasm/disks/NODE01DATA01)
NOTE: cache closing disk 0 of grp 1: (not open) OCR_0000
NOTE: Assigning number (2,5) to disk (/dev/oracleasm/disks/VOTEDB02)
NOTE: Assigning number (2,9) to disk (/dev/oracleasm/disks/VOTEDB01)
NOTE: Assigning number (2,2) to disk (/dev/oracleasm/disks/DISK03)
NOTE: cache closing disk 9 of grp 2: (not open) CRS_0009
ERROR: no read quorum in group: required 2, found 0 disks
NOTE: cache dismounting (clean) group 3/0xE5AB4C87 (VOTE) 
NOTE: messaging CKPT to quiesce pins Unix process pid: 44399, image: oracle@prod01 (TNS V1-V3)
NOTE: dbwr not being msg'd to dismount
NOTE: lgwr not being msg'd to dismount
NOTE: cache dismounted group 3/0xE5AB4C87 (VOTE) 
NOTE: cache ending mount (fail) of group VOTE number=3 incarn=0xe5ab4c87
NOTE: cache deleting context for group VOTE 3/0xe5ab4c87
GMON dismounting group 3 at 4 for pid 23, osid 44399
ERROR: diskgroup VOTE was not mounted
GMON querying group 1 at 5 for pid 23, osid 44399
NOTE: cache closing disk 0 of grp 1: (not open) OCR_0000
NOTE: cache opening disk 1 of grp 1: OCR_0001 path:/dev/oracleasm/disks/NODE02DATA01
NOTE: F1X0 found on disk 1 au 2 fcn 0.7444
NOTE: cache mounting (not first) normal redundancy group 1/0xF62B4C86 (OCR)
kjbdomatt send to inst 2
NOTE: attached to recovery domain 1
NOTE: redo buffer size is 256 blocks (1053184 bytes)
NOTE: LGWR attempting to mount thread 1 for diskgroup 1 (OCR)
Process LGWR (pid 44371) is running at high priority QoS for Exadata I/O
NOTE: LGWR found thread 1 closed at ABA 6.290
NOTE: LGWR mounted thread 1 for diskgroup 1 (OCR)
NOTE: LGWR opening thread 1 at fcn 0.7444 ABA 7.291
NOTE: cache mounting group 1/0xF62B4C86 (OCR) succeeded
NOTE: cache ending mount (success) of group OCR number=1 incarn=0xf62b4c86
GMON querying group 2 at 6 for pid 23, osid 44399
NOTE: cache closing disk 9 of grp 2: (not open) CRS_0009
NOTE: cache opening disk 2 of grp 2: CRS_0002 path:/dev/oracleasm/disks/DISK03
NOTE: cache opening disk 5 of grp 2: CRS_0005 path:/dev/oracleasm/disks/VOTEDB02
NOTE: F1X0 found on disk 5 au 2 fcn 0.197448
NOTE: cache mounting (not first) normal redundancy group 2/0x801B4C85 (CRS)
kjbdomatt send to inst 2
NOTE: attached to recovery domain 2
NOTE: redo buffer size is 256 blocks (1053184 bytes)
NOTE: LGWR attempting to mount thread 2 for diskgroup 2 (CRS)
NOTE: LGWR found thread 2 closed at ABA 25.705
NOTE: LGWR mounted thread 2 for diskgroup 2 (CRS)
NOTE: LGWR opening thread 2 at fcn 0.197448 ABA 26.706
NOTE: cache mounting group 2/0x801B4C85 (CRS) succeeded
NOTE: cache ending mount (success) of group CRS number=2 incarn=0x801b4c85
NOTE: Instance updated compatible.asm to 11.2.0.0.0 for grp 1
SUCCESS: diskgroup OCR was mounted
NOTE: Instance updated compatible.asm to 11.2.0.0.0 for grp 2
SUCCESS: diskgroup CRS was mounted
ORA-15032: not all alterations performed
ORA-15017: diskgroup "VOTE" cannot be mounted
ORA-15040: diskgroup is incomplete
ERROR: ALTER DISKGROUP ALL MOUNT /* asm agent call crs *//* {0:9:7} */
SQL> ALTER DISKGROUP ALL ENABLE VOLUME ALL /* asm agent *//* {0:9:7} */ 
SUCCESS: ALTER DISKGROUP ALL ENABLE VOLUME ALL /* asm agent *//* {0:9:7} */
WARNING: failed to online diskgroup resource ora.OCR.dg (unable to communicate with CRSD/OHASD)
Tue Sep 17 14:50:00 2019
NOTE: [crsd.bin@prod01 (TNS V1-V3) 44448] opening OCR file
Starting background process ASMB
Tue Sep 17 14:50:00 2019
ASMB started with pid=25, OS id=44471 
Tue Sep 17 14:50:00 2019
NOTE: client +ASM1:+ASM registered, osid 44473, mbr 0x0
NOTE: Attempting voting file refresh on diskgroup CRS
NOTE: Refresh completed on diskgroup CRS
. Found 2 voting file(s).
NOTE: Voting file relocation is required in diskgroup CRS
NOTE: Attempting voting file relocation on diskgroup CRS
NOTE: Successful voting file relocation on diskgroup CRS
WARNING: failed to online diskgroup resource ora.CRS.dg (unable to communicate with CRSD/OHASD)
Tue Sep 17 14:50:37 2019
NOTE: client ora1:ora registered, osid 44859, mbr 0x1
Tue Sep 17 14:51:08 2019
ALTER SYSTEM SET local_listener=' (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=192.168.0.232)(PORT=1521))))' SCOPE=MEMORY SID='+ASM1';
Tue Sep 17 14:51:51 2019
NOTE: disk validation pending for group 2/0x801b4c85 (CRS)
NOTE: Attempting voting file refresh on diskgroup CRS
NOTE: Refresh completed on diskgroup CRS
. Found 2 voting file(s).
NOTE: Voting file relocation is required in diskgroup CRS
NOTE: Found /dev/oracleasm/disks/VOTEDB01 for disk CRS_0009
WARNING: ignoring disk  in deep discovery
SUCCESS: validated disks for 2/0x801b4c85 (CRS)
NOTE: membership refresh pending for group 2/0x801b4c85 (CRS)
NOTE: Attempting voting file relocation on diskgroup CRS
NOTE: Successful voting file relocation on diskgroup CRS
Tue Sep 17 14:51:57 2019
GMON querying group 2 at 9 for pid 18, osid 44377
NOTE: cache opening disk 9 of grp 2: CRS_0009 path:/dev/oracleasm/disks/VOTEDB01
SUCCESS: refreshed membership for 2/0x801b4c85 (CRS)
NOTE: Attempting voting file refresh on diskgroup CRS
NOTE: Refresh completed on diskgroup CRS
. Found 2 voting file(s).
NOTE: Voting file relocation is required in diskgroup CRS
NOTE: Attempting voting file relocation on diskgroup CRS
NOTE: voting file allocation on grp 2 disk CRS_0009
Tue Sep 17 14:52:02 2019
NOTE: Successful voting file relocation on diskgroup CRS
NOTE: Voting File refresh pending for group 2/0x801b4c85 (CRS)
NOTE: disk validation pending for group 1/0xf62b4c86 (OCR)
NOTE: Found /dev/oracleasm/disks/NODE01DATA01 for disk OCR_0000
WARNING: ignoring disk  in deep discovery
SUCCESS: validated disks for 1/0xf62b4c86 (OCR)
NOTE: Attempting voting file refresh on diskgroup CRS
NOTE: Refresh completed on diskgroup CRS
. Found 3 voting file(s).
Tue Sep 17 14:52:08 2019
NOTE: group OCR: updated PST location: disk 0001 (PST copy 0)
NOTE: group OCR: updated PST location: disk 0000 (PST copy 1)
NOTE: membership refresh pending for group 1/0xf62b4c86 (OCR)
GMON querying group 1 at 10 for pid 18, osid 44377
NOTE: group OCR: updated PST location: disk 0001 (PST copy 0)
NOTE: group OCR: updated PST location: disk 0000 (PST copy 1)
NOTE: cache opening disk 0 of grp 1: OCR_0000 path:/dev/oracleasm/disks/NODE01DATA01
SUCCESS: refreshed membership for 1/0xf62b4c86 (OCR)
NOTE: Voting File refresh pending for group 1/0xf62b4c86 (OCR)
Tue Sep 17 14:52:17 2019
NOTE: Attempting voting file refresh on diskgroup OCR
NOTE: Refresh completed on diskgroup OCR. No voting file found.

prod01 db log:

Tue Sep 17 14:50:29 2019
Adjusting the default value of parameter parallel_max_servers
from 320 to 120 due to the value of parameter processes (150)
Starting ORACLE instance (normal)
************************ Large Pages Information *******************
Per process system memlock (soft) limit = UNLIMITED
 
Total Shared Global Region in Large Pages = 0 KB (0%)
 
Large Pages used by this instance: 0 (0 KB)
Large Pages unused system wide = 0 (0 KB)
Large Pages configured system wide = 0 (0 KB)
Large Page size = 2048 KB
 
RECOMMENDATION:
  Total System Global Area size is 770 MB. For optimal performance,
  prior to the next instance restart:
  1. Increase the number of unused large pages by 
 at least 385 (page size 2048 KB, total size 770 MB) system wide to
  get 100% of the System Global Area allocated with large pages
********************************************************************
LICENSE_MAX_SESSION = 0
LICENSE_SESSIONS_WARNING = 0
Initial number of CPU is 8
Number of processor cores in the system is 8
Number of processor sockets in the system is 8
Private Interface 'eth1:1' configured from GPnP for use as a private interconnect.
  [name='eth1:1', type=1, ip=169.254.237.99, mac=00-0c-29-1e-dd-e5, net=169.254.0.0/16, mask=255.255.0.0, use=haip:cluster_interconnect/62]
Public Interface 'eth0' configured from GPnP for use as a public interface.
  [name='eth0', type=1, ip=192.168.0.230, mac=00-0c-29-1e-dd-db, net=192.168.0.0/24, mask=255.255.255.0, use=public/1]
Public Interface 'eth0:1' configured from GPnP for use as a public interface.
  [name='eth0:1', type=1, ip=192.168.0.232, mac=00-0c-29-1e-dd-db, net=192.168.0.0/24, mask=255.255.255.0, use=public/1]
CELL communication is configured to use 0 interface(s):
CELL IP affinity details:
    NUMA status: non-NUMA system
    cellaffinity.ora status: N/A
CELL communication will use 1 IP group(s):
    Grp 0: 
Picked latch-free SCN scheme 3
Using LOG_ARCHIVE_DEST_1 parameter default value as /u01/app/oracle/product/11.2.0/db_1/dbs/arch
Autotune of undo retention is turned on. 
LICENSE_MAX_USERS = 0
SYS auditing is disabled
Starting up:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, Real Application Clusters, OLAP, Data Mining
and Real Application Testing options.
ORACLE_HOME = /u01/app/oracle/product/11.2.0/db_1
System name:    Linux
Node name:    prod01
Release:    2.6.32-642.el6.x86_64
Version:    #1 SMP Wed Apr 13 00:51:26 EDT 2016
Machine:    x86_64
VM name:    VMWare Version: 6
Using parameter settings in server-side pfile /u01/app/oracle/product/11.2.0/db_1/dbs/initora1.ora
System parameters with non-default values:
  processes                = 150
  spfile                   = "+OCR/ora/spfileora.ora"
  sga_target               = 768M
  control_files            = "+OCR/ora/controlfile/current.260.1019221637"
  db_block_size            = 8192
  compatible               = "11.2.0.4.0"
  cluster_database         = TRUE
  db_create_file_dest      = "+OCR"
  thread                   = 1
  undo_tablespace          = "UNDOTBS1"
  instance_number          = 1
  remote_login_passwordfile= "EXCLUSIVE"
  db_domain                = ""
  dispatchers              = "(PROTOCOL=TCP) (SERVICE=oraXDB)"
  remote_listener          = "scan:1521"
  audit_file_dest          = "/u01/app/oracle/admin/ora/adump"
  audit_trail              = "DB"
  db_name                  = "ora"
  open_cursors             = 300
  pga_aggregate_target     = 256M
  diagnostic_dest          = "/u01/app/oracle"
Cluster communication is configured to use the following interface(s) for this instance
  169.254.237.99
cluster interconnect IPC version:Oracle UDP/IP (generic)
IPC Vendor 1 proto 2
Tue Sep 17 14:50:35 2019
PMON started with pid=2, OS id=44805 
Tue Sep 17 14:50:35 2019
PSP0 started with pid=3, OS id=44807 
Tue Sep 17 14:50:36 2019
VKTM started with pid=4, OS id=44809 at elevated priority
VKTM running at (1)millisec precision with DBRM quantum (100)ms
Tue Sep 17 14:50:36 2019
GEN0 started with pid=5, OS id=44813 
Tue Sep 17 14:50:36 2019
DIAG started with pid=6, OS id=44815 
Tue Sep 17 14:50:36 2019
DBRM started with pid=7, OS id=44817 
Tue Sep 17 14:50:36 2019
PING started with pid=8, OS id=44819 
Tue Sep 17 14:50:36 2019
ACMS started with pid=9, OS id=44821 
Tue Sep 17 14:50:36 2019
DIA0 started with pid=10, OS id=44823 
Tue Sep 17 14:50:36 2019
LMON started with pid=11, OS id=44825 
Tue Sep 17 14:50:36 2019
LMD0 started with pid=12, OS id=44827 
* Load Monitor used for high load check 
* New Low - High Load Threshold Range = [7680 - 10240] 
Tue Sep 17 14:50:36 2019
LMS0 started with pid=13, OS id=44829 at elevated priority
Tue Sep 17 14:50:36 2019
LMS1 started with pid=14, OS id=44833 at elevated priority
Tue Sep 17 14:50:37 2019
RMS0 started with pid=15, OS id=44837 
Tue Sep 17 14:50:37 2019
LMHB started with pid=16, OS id=44839 
Tue Sep 17 14:50:37 2019
MMAN started with pid=17, OS id=44841 
Tue Sep 17 14:50:37 2019
DBW0 started with pid=18, OS id=44843 
Tue Sep 17 14:50:37 2019
LGWR started with pid=19, OS id=44845 
Tue Sep 17 14:50:37 2019
CKPT started with pid=20, OS id=44847 
Tue Sep 17 14:50:37 2019
SMON started with pid=21, OS id=44849 
Tue Sep 17 14:50:37 2019
RECO started with pid=22, OS id=44851 
Tue Sep 17 14:50:37 2019
RBAL started with pid=23, OS id=44853 
Tue Sep 17 14:50:37 2019
ASMB started with pid=24, OS id=44855 
Tue Sep 17 14:50:37 2019
MMON started with pid=25, OS id=44857 
Tue Sep 17 14:50:37 2019
MMNL started with pid=26, OS id=44861 
starting up 1 dispatcher(s) for network address '(ADDRESS=(PARTIAL=YES)(PROTOCOL=TCP))'...
NOTE: initiating MARK startup 
starting up 1 shared server(s) ...Starting background process MARK

Tue Sep 17 14:50:37 2019
MARK started with pid=28, OS id=44865 
NOTE: MARK has subscribed 
lmon registered with NM - instance number 1 (internal mem no 0)
Reconfiguration started (old inc 0, new inc 16)
List of instances:
 1 2 (myinst: 1) 
 Global Resource Directory frozen
* allocate domain 0, invalid = TRUE 
 Communication channels reestablished
 * domain 0 valid according to instance 2 
 * domain 0 valid = 1 according to instance 2 
 Master broadcasted resource hash value bitmaps
 Non-local Process blocks cleaned out
 LMS 1: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
 LMS 0: 0 GCS shadows cancelled, 0 closed, 0 Xw survived
 Set master node info 
 Submitted all remote-enqueue requests
 Dwn-cvts replayed, VALBLKs dubious
 All grantable enqueues granted
 Submitted all GCS remote-cache requests
 Fix write in gcs resources
Reconfiguration complete
Tue Sep 17 14:50:38 2019
LCK0 started with pid=31, OS id=44873 
Starting background process RSMN
Tue Sep 17 14:50:38 2019
RSMN started with pid=32, OS id=44875 
ORACLE_BASE not set in environment. It is recommended
that ORACLE_BASE be set in the environment
Tue Sep 17 14:50:39 2019
ALTER SYSTEM SET local_listener=' (ADDRESS=(PROTOCOL=TCP)(HOST=192.168.0.232)(PORT=1521))' SCOPE=MEMORY SID='ora1';
ALTER DATABASE MOUNT /* db agent *//* {1:13741:2} */
NOTE: Loaded library: System 
SUCCESS: diskgroup OCR was mounted
NOTE: dependency between database ora and diskgroup resource ora.OCR.dg is established
Successful mount of redo thread 1, with mount id 1637464745
Database mounted in Shared Mode (CLUSTER_DATABASE=TRUE)
Lost write protection disabled
Completed: ALTER DATABASE MOUNT /* db agent *//* {1:13741:2} */
ALTER DATABASE OPEN /* db agent *//* {1:13741:2} */
Picked broadcast on commit scheme to generate SCNs
Thread 1 opened at log sequence 6
  Current log# 2 seq# 6 mem# 0: +OCR/ora/onlinelog/group_2.262.1019221641
Successful open of redo thread 1
MTTR advisory is disabled because FAST_START_MTTR_TARGET is not set
SMON: enabling cache recovery
minact-scn: Inst 1 is a slave inc#:16 mmon proc-id:44857 status:0x2
minact-scn status: grec-scn:0x0000.00000000 gmin-scn:0x0000.00000000 gcalc-scn:0x0000.00000000
[44877] Successfully onlined Undo Tablespace 2.
Undo initialization finished serial:0 start:11652414 end:11652714 diff:300 (3 seconds)
Verifying file header compatibility for 11g tablespace encryption..
Verifying 11g file header compatibility for tablespace encryption completed
SMON: enabling tx recovery
Database Characterset is WE8MSWIN1252
No Resource Manager plan active
Starting background process GTX0
Tue Sep 17 14:50:47 2019
GTX0 started with pid=36, OS id=44919 
Starting background process RCBG
Tue Sep 17 14:50:47 2019
RCBG started with pid=37, OS id=44921 
replication_dependency_tracking turned off (no async multimaster replication found)
Starting background process QMNC
Tue Sep 17 14:50:47 2019
QMNC started with pid=38, OS id=44923 
Tue Sep 17 14:50:50 2019
Completed: ALTER DATABASE OPEN /* db agent *//* {1:13741:2} */
Tue Sep 17 14:50:50 2019
Starting background process CJQ0
Tue Sep 17 14:50:50 2019
CJQ0 started with pid=33, OS id=44946 
Tue Sep 17 14:52:09 2019
NOTE: Found /dev/oracleasm/disks/NODE01DATA01 for disk OCR_0000
SUCCESS: disk OCR_0000 (0.3915103355) replaced in diskgroup OCR
NOTE: disk 0 (OCR_0000) in group 1 (OCR) is online for writes
Tue Sep 17 14:52:29 2019
NOTE: disk 0 (OCR_0000) in group 1 (OCR) is online for reads
Tue Sep 17 14:55:48 2019
Starting background process SMCO
Tue Sep 17 14:55:48 2019
SMCO started with pid=30, OS id=45629 

网友评论

登录后评论
0/500
评论
snowofsummer
+ 关注