asm Normal redundancy storage link is broken,

  1. 云栖社区>
  2. 博客>
  3. 正文

asm Normal redundancy storage link is broken,

snowofsummer 2019-05-27 10:12:54 浏览178
展开阅读全文

2-Node, 11.2.0.4 RAC with ASM. Normal redundancy diskgroups. When the storage link is broken, so that Node-A can not access Storage-B and Node-b can not access Storage-A

测试环境

Release 11.2.0.4.0

磁盘组信息

SQL> @c

DG_NAME     DG_STATE   TYPE       DSK_NO DSK_NAME    PATH                                 MOUNT_S FAILGROUP          STATE
--------------- ---------- ------ ---------- ---------- ------------------------------------------------------------ ------- -------------------- --------
CRS        MOUNTED    NORMAL       0 CRS_0000    /dev/oracleasm/disks/VOTE01                     CACHED  CRS_0000          NORMAL
CRS        MOUNTED    NORMAL       1 CRS_0001    /dev/oracleasm/disks/VOTE02                     CACHED  CRS_0001          NORMAL
CRS        MOUNTED    NORMAL       3 CRS_0003    /dev/oracleasm/disks/VOTEZC                     CACHED  SYSFG3          NORMAL
DATA        MOUNTED    NORMAL       0 DATA_0000    /dev/oracleasm/disks/DATA01                     CACHED  AA           NORMAL
DATA        MOUNTED    NORMAL       1 DATA_0001    /dev/oracleasm/disks/DATA02                     CACHED  BB           NORMAL


DISK_NUMBER NAME       PATH                  HEADER_STATUS        OS_MB    TOTAL_MB    FREE_MB REPAIR_TIMER V FAILGRO
----------- ---------- ------------------------------ -------------------- ---------- ---------- ---------- ------------ - -------
      0 DATA_0000  /dev/oracleasm/disks/DATA01    MEMBER             1019        1019    656           0 N REGULAR
      0 CRS_0000   /dev/oracleasm/disks/VOTE01    MEMBER             2055        2055       1657           0 Y REGULAR
      3 CRS_0003   /dev/oracleasm/disks/VOTEZC    MEMBER             2055        2055       2021           0 Y QUORUM
      1 CRS_0001   /dev/oracleasm/disks/VOTE02    MEMBER             2047        2047       1457           0 Y REGULAR
      1 DATA_0001  /dev/oracleasm/disks/DATA02    MEMBER             1019        1019    656           0 N REGULAR


GROUP_NUMBER NAME    COMPATIBILITY                             DATABASE_COMPATIBILITY                      V
------------ ---------- ------------------------------------------------------------ ------------------------------------------------------------ -
       2 DATA    11.2.0.0.0                             11.2.0.0.0                           N
       1 CRS    11.2.0.0.0                             11.2.0.0.0                           Y

存储链路断开

ora01 只能看到data01 磁盘
ora02只能看到data02 磁盘

ora 01 asm 日志

Mon May 27 09:49:31 2019
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Mon May 27 09:49:55 2019
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Mon May 27 09:50:55 2019
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Mon May 27 09:51:43 2019
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Mon May 27 09:52:05 2019
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12930.trc:
ORA-15300: File +data/ora-cluster/OCRFILE/REGISTRY.255.1009297099 is incompatible with this operation
Mon May 27 09:52:21 2019
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_12732.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Mon May 27 09:52:48 2019
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_13215.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_13215.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_13215.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_13215.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_13215.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_13215.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA02"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Mon May 27 09:53:04 2019
WARNING: Read Failed. group:2 disk:1 AU:1 offset:0 size:4096
WARNING: Read Failed. group:2 disk:1 AU:1 offset:4096 size:4096
ERROR: no read quorum in group: required 2, found 0 disks
ERROR: Could not read PST for grp 2. Force dismounting the disk group.
Mon May 27 09:53:04 2019
NOTE: cache dismounting (not clean) group 2/0x4F1BECB7 (DATA) 
NOTE: messaging CKPT to quiesce pins Unix process pid: 13254, image: oracle@ora01 (B000)
Mon May 27 09:53:04 2019
NOTE: halting all I/Os to diskgroup 2 (DATA)
Mon May 27 09:53:04 2019
NOTE: LGWR doing non-clean dismount of group 2 (DATA)
NOTE: LGWR sync ABA=6.353 last written ABA 6.353
Mon May 27 09:53:04 2019
kjbdomdet send to inst 2
detach from dom 2, sending detach message to inst 2
Mon May 27 09:53:05 2019
ERROR: ORA-15130 in COD recovery for diskgroup 2/0x4f1becb7 (DATA)
ERROR: ORA-15130 thrown in RBAL for group number 2
Mon May 27 09:53:05 2019
List of instances:
 1 2
Dirty detach reconfiguration started (new ddet inc 1, cluster inc 12)
 Global Resource Directory partially frozen for dirty detach
* dirty detach - domain 2 invalid = TRUE 
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_rbal_7577.trc:
ORA-15130: diskgroup "DATA" is being dismounted
 2 GCS resources traversed, 0 cancelled
Dirty Detach Reconfiguration complete
Mon May 27 09:53:05 2019
WARNING: dirty detached from domain 2
NOTE: cache dismounted group 2/0x4F1BECB7 (DATA) 
NOTE: cache deleting context for group DATA 2/0x4f1becb7
SQL> alter diskgroup DATA dismount force /* ASM SERVER:1327230135 */ 
GMON dismounting group 2 at 32 for pid 34, osid 13254
NOTE: Disk DATA_0000 in mode 0x7f marked for de-assignment
NOTE: Disk DATA_0001 in mode 0x7f marked for de-assignment
NOTE:Waiting for all pending writes to complete before de-registering: grpnum 2
Mon May 27 09:53:35 2019
SUCCESS: diskgroup DATA was dismounted
SUCCESS: alter diskgroup DATA dismount force /* ASM SERVER:1327230135 */
Mon May 27 09:53:35 2019
NOTE: diskgroup resource ora.DATA.dg is offline
SUCCESS: ASM-initiated MANDATORY DISMOUNT of group DATA

ora02 asm 日志

Mon May 27 09:53:13 2019
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_ora_12421.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_ora_12421.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_ora_12421.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_ora_12421.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_ora_12421.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_ora_12421.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
WARNING: cache failed reading from group=2(DATA) fn=1 blk=8 count=1 from disk= 0 (DATA_0000) kfkist=0x20 status=0x02 osderr=0x0 file=kfc.c line=11596
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_ora_12421.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
ORA-15080: synchronous I/O operation to a disk failed
WARNING: cache succeeded reading from group=2(DATA) fn=1 blk=8 count=1 from disk= 1 (DATA_0001) kfkist=0x20 status=0x01 osderr=0x0 file=kfc.c line=11637
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_ora_12421.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_ora_12421.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
WARNING: cache failed reading from group=2(DATA) fn=8 blk=0 count=1 from disk= 0 (DATA_0000) kfkist=0x20 status=0x02 osderr=0x0 file=kfc.c line=11596
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_ora_12421.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
ORA-15080: synchronous I/O operation to a disk failed
Mon May 27 09:53:13 2019
NOTE: repairing group 2 file 1 extent 0
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_r000_12428.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
WARNING: cache succeeded reading from group=2(DATA) fn=8 blk=0 count=1 from disk= 1 (DATA_0001) kfkist=0x20 status=0x01 osderr=0x0 file=kfc.c line=11637
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_r000_12428.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
WARNING: block repair initiating disk offline
NOTE: disk 65534 is not a part of group 2 anymore so cannot be offlined.
NOTE: process _r000_+asm2 (12428) initiating offline of disk 0.3916345468 (DATA_0000) with mask 0x7e in group 2
NOTE: checking PST: grp = 2
Mon May 27 09:53:13 2019
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_ora_12421.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_ora_12421.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
WARNING: cache failed reading from group=2(DATA) fn=8 blk=60 count=1 from disk= 0 (DATA_0000) kfkist=0x20 status=0x02 osderr=0x0 file=kfc.c line=11596
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_ora_12421.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
ORA-15080: synchronous I/O operation to a disk failed
NOTE: repairing group 2 file 8 extent 0
WARNING: cache succeeded reading from group=2(DATA) fn=8 blk=60 count=1 from disk= 1 (DATA_0001) kfkist=0x20 status=0x01 osderr=0x0 file=kfc.c line=11637
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_r001_12430.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_r001_12430.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Mon May 27 09:53:14 2019
NOTE: repairing group 2 file 8 extent 0
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_r002_12432.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
WARNING: block repair initiating disk offline
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_r002_12432.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_ora_12421.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_ora_12421.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
WARNING: cache failed reading from group=2(DATA) fn=8 blk=120 count=1 from disk= 0 (DATA_0000) kfkist=0x20 status=0x02 osderr=0x0 file=kfc.c line=11596
WARNING: block repair initiating disk offline
Mon May 27 09:53:14 2019
GMON checking disk modes for group 2 at 7 for pid 28, osid 12428
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_ora_12421.trc:
ORA-15025: could not open disk "/dev/oracleasm/disks/DATA01"
ORA-27041: unable to open file
Linux-x86_64 Error: 6: No such device or address
Additional information: 3
ORA-15080: synchronous I/O operation to a disk failed
WARNING: cache succeeded reading from group=2(DATA) fn=8 blk=120 count=1 from disk= 1 (DATA_0001) kfkist=0x20 status=0x01 osderr=0x0 file=kfc.c line=11637
NOTE: checking PST for grp 2 done.
NOTE: initiating PST update: grp = 2, dsk = 0/0xe96eb07c, mask = 0x6a, op = clear
GMON updating disk modes for group 2 at 8 for pid 28, osid 12428
NOTE: group DATA: updated PST location: disk 0001 (PST copy 0)
Mon May 27 09:53:15 2019
 Received dirty detach msg from inst 1 for dom 2
Mon May 27 09:53:15 2019
NOTE: repairing group 2 file 8 extent 0
NOTE: disk 65534 is not a part of group 2 anymore so cannot be offlined.
NOTE: disk 65534 is not a part of group 2 anymore so cannot be offlined.
WARNING: block repair initiating disk offline
Mon May 27 09:53:15 2019
List of instances:
 1 2
Dirty detach reconfiguration started (new ddet inc 1, cluster inc 12)
 Global Resource Directory partially frozen for dirty detach
* dirty detach - domain 2 invalid = TRUE 
 26 GCS resources traversed, 0 cancelled
Dirty Detach Reconfiguration complete
Mon May 27 09:53:16 2019
NOTE: SMON starting instance recovery for group DATA domain 2 (mounted)
NOTE: SMON skipping disk 0 (mode=00000015)
NOTE: F1X0 found on disk 1 au 2 fcn 0.690
NOTE: PST enabling heartbeating (grp 2)
NOTE: disk 65534 is not a part of group 2 anymore so cannot be offlined.
NOTE: PST update grp = 2 completed successfully 
NOTE: initiating PST update: grp = 2, dsk = 0/0xe96eb07c, mask = 0x7e, op = clear
GMON updating disk modes for group 2 at 9 for pid 28, osid 12428
NOTE: group DATA: updated PST location: disk 0001 (PST copy 0)
NOTE: cache closing disk 0 of grp 2: DATA_0000
NOTE: starting recovery of thread=1 ckpt=6.354 group=2 (DATA)
NOTE: disk 65534 is not a part of group 2 anymore so cannot be offlined.
NOTE: disk 65534 is not a part of group 2 anymore so cannot be offlined.
NOTE: ASM recovery sucessfully read ACD from one mirror side
Errors in file /u01/app/grid/diag/asm/+asm/+ASM2/trace/+ASM2_smon_10850.trc:
ORA-15062: ASM disk is globally closed
ORA-15062: ASM disk is globally closed
NOTE: SMON waiting for thread 1 recovery enqueue
NOTE: SMON about to begin recovery lock claims for diskgroup 2 (DATA)
SUCCESS: extent 0 of file 8 group 2 repaired - all online mirror sides found readable, no repair required
SUCCESS: extent 0 of file 8 group 2 repaired - all online mirror sides found readable, no repair required
NOTE: waiting for instance recovery of group 2
NOTE: disk 65534 is not a part of group 2 anymore so cannot be offlined.
NOTE: PST update grp = 2 completed successfully 
SUCCESS: extent 0 of file 8 group 2 repaired - all online mirror sides found readable, no repair required
SUCCESS: extent 0 of file 1 group 2 repaired - all online mirror sides found readable, no repair required
NOTE: SMON successfully validated lock domain 2
NOTE: advancing ckpt for group 2 (DATA) thread=1 ckpt=6.354
NOTE: SMON did instance recovery for group DATA domain 2
Mon May 27 09:53:39 2019
WARNING: Disk 0 (DATA_0000) in group 2 will be dropped in: (12960) secs on ASM inst 2
Mon May 27 09:56:42 2019
WARNING: Disk 0 (DATA_0000) in group 2 will be dropped in: (12775) secs on ASM inst 2
Mon May 27 09:59:45 2019
WARNING: Disk 0 (DATA_0000) in group 2 will be dropped in: (12592) secs on ASM inst 2
Mon May 27 10:02:49 2019
WARNING: Disk 0 (DATA_0000) in group 2 will be dropped in: (12409) secs on ASM inst 2
Mon May 27 10:05:52 2019
WARNING: Disk 0 (DATA_0000) in group 2 will be dropped in: (12226) secs on ASM inst 2
Mon May 27 10:08:55 2019
WARNING: Disk 0 (DATA_0000) in group 2 will be dropped in: (12043) secs on ASM inst 2

ASM 状态信息

ORA01

DG_NAME     DG_STATE   TYPE       DSK_NO DSK_NAME    PATH                                 MOUNT_S FAILGROUP          STATE
--------------- ---------- ------ ---------- ---------- ------------------------------------------------------------ ------- -------------------- --------
CRS        MOUNTED    NORMAL       0 CRS_0000    /dev/oracleasm/disks/VOTE01                     CACHED  CRS_0000          NORMAL
CRS        MOUNTED    NORMAL       1 CRS_0001    /dev/oracleasm/disks/VOTE02                     CACHED  CRS_0001          NORMAL
CRS        MOUNTED    NORMAL       3 CRS_0003    /dev/oracleasm/disks/VOTEZC                     CACHED  SYSFG3          NORMAL
DATA        DISMOUNTED           0        /dev/oracleasm/disks/DATA01                     CLOSED              NORMAL


DISK_NUMBER NAME       PATH                  HEADER_STATUS        OS_MB    TOTAL_MB    FREE_MB REPAIR_TIMER V FAILGRO
----------- ---------- ------------------------------ -------------------- ---------- ---------- ---------- ------------ - -------
      0           /dev/oracleasm/disks/DATA01    MEMBER             1019           0      0           0 N REGULAR
      0 CRS_0000   /dev/oracleasm/disks/VOTE01    MEMBER             2055        2055       1657           0 Y REGULAR
      3 CRS_0003   /dev/oracleasm/disks/VOTEZC    MEMBER             2055        2055       2021           0 Y QUORUM
      1 CRS_0001   /dev/oracleasm/disks/VOTE02    MEMBER             2047        2047       1457           0 Y REGULAR


GROUP_NUMBER NAME    COMPATIBILITY                             DATABASE_COMPATIBILITY                      V
------------ ---------- ------------------------------------------------------------ ------------------------------------------------------------ -
       0 DATA    0.0.0.0.0                             0.0.0.0.0                              N
       1 CRS    11.2.0.0.0                             11.2.0.0.0                           Y

ora02

DG_NAME     DG_STATE   TYPE       DSK_NO DSK_NAME    PATH                                 MOUNT_S FAILGROUP          STATE
--------------- ---------- ------ ---------- ---------- ------------------------------------------------------------ ------- -------------------- --------
CRS        MOUNTED    NORMAL       0 CRS_0000    /dev/oracleasm/disks/VOTE01                     CACHED  CRS_0000          NORMAL
CRS        MOUNTED    NORMAL       1 CRS_0001    /dev/oracleasm/disks/VOTE02                     CACHED  CRS_0001          NORMAL
CRS        MOUNTED    NORMAL       3 CRS_0003    /dev/oracleasm/disks/VOTEZC                     CACHED  SYSFG3          NORMAL
DATA        MOUNTED    NORMAL       0 DATA_0000                                     MISSING AA           NORMAL
DATA        MOUNTED    NORMAL       1 DATA_0001    /dev/oracleasm/disks/DATA02                     CACHED  BB           NORMAL


DISK_NUMBER NAME       PATH                  HEADER_STATUS        OS_MB    TOTAL_MB    FREE_MB
----------- ---------- ------------------------------ -------------------- ---------- ---------- ----------
      0 DATA_0000                      UNKNOWN                0        1019    656
      0 CRS_0000   /dev/oracleasm/disks/VOTE01    MEMBER             2055        2055       1657
      3 CRS_0003   /dev/oracleasm/disks/VOTEZC    MEMBER             2055        2055       2021
      1 DATA_0001  /dev/oracleasm/disks/DATA02    MEMBER             1019        1019    656
      1 CRS_0001   /dev/oracleasm/disks/VOTE02    MEMBER             2047        2047       1457

网友评论

登录后评论
0/500
评论
snowofsummer
+ 关注