Oracle 无备份情况恢复ocr和olr

  1. 云栖社区>
  2. 博客>
  3. 正文

Oracle 无备份情况恢复ocr和olr

小麦苗 2016-12-10 17:41:24 浏览1087
展开阅读全文

Oracle 无备份情况恢复ocr和olr



 Oracle 11.2 GI中OLR损坏的修复方法 

一、先关闭GI,命令不再贴出来了,参见:
http://blog.csdn.net/msdnchina/article/details/49402299

二、模拟OLR损坏

[root@hostb cdata]# ls -lrt
total 2884
drwxr-xr-x 2 grid oinstall      4096 Oct 30 13:21 hostb
drwxr-xr-x 2 grid oinstall      4096 Oct 30 19:06 localhost drwxrwxr-x 2 grid oinstall      4096 Dec 22 13:27 hosta-cluster
-rw------- 1 root oinstall 272756736 Jan  8 13:38 hostb.olr
[root@hostb cdata]# mv hostb.olr  back_hostb.olr ----------->  mv掉OLR
[root@hostb cdata]# ls -lrt
total 2884
drwxr-xr-x 2 grid oinstall      4096 Oct 30 13:21 hostb
drwxr-xr-x 2 grid oinstall      4096 Oct 30 19:06 localhost
drwxrwxr-x 2 grid oinstall      4096 Dec 22 13:27 hosta-cluster
-rw------- 1 root oinstall 272756736 Jan  8 13:38 back_hostb.olr
[root@hostb cdata]# /u01/app/11.2.0/grid/bin/crsctl start crs   --------->起不来。
CRS-4124: Oracle High Availability Services startup failed.
CRS-4000: command Start failed, or completed with errors.
[root@hostb cdata]# pwd
/u01/app/11.2.0/grid/cdata
[root@hostb cdata]#


[oracle@hostb ohasd]$ pwd
/u01/app/11.2.0/grid/log/hostb/ohasd
[oracle@hostb ohasd]$ ls -lrt
total 95720
-rw-r--r-- 1 root root 10540929 Nov  4 03:51 ohasd.l09
-rw-r--r-- 1 root root 10556558 Nov  8 16:28 ohasd.l08
-rw-r--r-- 1 root root 10558423 Nov 13 12:39 ohasd.l07
-rw-r--r-- 1 root root 10558697 Nov 18 05:44 ohasd.l06
-rw-r--r-- 1 root root 10557602 Nov 22 18:15 ohasd.l05
-rw-r--r-- 1 root root 10532670 Nov 27 08:24 ohasd.l04
-rw-r--r-- 1 root root 10556869 Dec  2 01:30 ohasd.l03
-rw-r--r-- 1 root root 10557883 Dec  6 23:49 ohasd.l02
-rw-r--r-- 1 root root 10542530 Dec 21 11:49 ohasd.l01
-rw-r--r-- 1 root root     6700 Jan  8 13:42 ohasdOUT.log
-rw-r--r-- 1 root root  2986628 Jan  8 13:42 ohasd.log
[oracle@hostb ohasd]$ vi ohasd.log

2016-01-08 13:42:59.562: [ default][3149834016] Initializing OLR
2016-01-08 13:42:59.564: [  OCROSD][3149834016]utopen:6m': failed in stat OCR file/disk /u01/app/11.2.0/grid/cdata/hostb.olr, errno=2, os err string=No such file or directory
2016-01-08 13:42:59.564: [  OCROSD][3149834016]utopen:7: failed to open any OCR file/disk, errno=2, os err string=No such file or directory
2016-01-08 13:42:59.564: [  OCRRAW][3149834016]proprinit: Could not open raw device
2016-01-08 13:42:59.564: [  OCRAPI][3149834016]a_init:16!: Backend init unsuccessful : [26]
2016-01-08 13:42:59.564: [  CRSOCR][3149834016] OCR context init failure.  Error: PROCL-26: Error while accessing the physical storage Operating system error [No such file or directory] [2]
2016-01-08 13:42:59.565: [ default][3149834016] Created alert : (:OHAS00106:) :  OLR initialization failed, error: PROCL-26: Error while accessing the physical storage Operating System error [No such file or directory] [2]
2016-01-08 13:42:59.565: [ default][3149834016][PANIC] OHASD exiting; Could not init OLR
2016-01-08 13:42:59.565: [ default][3149834016] Done.


开始对本地节点(主机名为hostb)进行OLR的recreate,参考文章:
Top 5 Grid Infrastructure Startup Issues (文档 ID 1368382.1) ,搜索关键字为"OLR"

[root@hostb bin]# ./crsctl stop crs
CRS-4544: Unable to connect to OHAS
CRS-4000: Command Stop failed, or completed with errors.




第一步:# /u01/app/11.2.0/grid/crs/install/rootcrs.pl -deconfig -force

[root@hostb install]# /u01/app/11.2.0/grid/crs/install/rootcrs.pl -deconfig -force
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
PRCR-1119 : Failed to look up CRS resources of ora.cluster_vip_net1.type type
PRCR-1068 : Failed to query resources
Cannot communicate with crsd
PRCR-1070 : Failed to check if resource ora.gsd is registered
Cannot communicate with crsd
PRCR-1070 : Failed to check if resource ora.ons is registered
Cannot communicate with crsd

CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Stop failed, or completed with errors.
CRS-4544: Unable to connect to OHAS
CRS-4000: Command Stop failed, or completed with errors.
Successfully deconfigured Oracle clusterware stack on this node
[root@hostb install]#


此时的/u01/app/11.2.0/grid/cdata目录情况:

[grid@hostb cdata]$ ls -lrt
total 13124
drwxr-xr-x 2 grid oinstall      4096 Oct 30 13:21 hostb
drwxr-xr-x 2 grid oinstall      4096 Oct 30 19:06 localhost
drwxrwxr-x 2 grid oinstall      4096 Dec 22 13:27 hosta-cluster
-rw------- 1 root oinstall 272756736 Jan  8 13:38 back_hostb.olr
-rw-r--r-- 1 root root      10485760 Jan 12 18:08 hostb.olr
[grid@hostb cdata]$



第二步:# /u01/app/11.2.0/grid/root.sh

[root@hostb install]# /u01/app/11.2.0/grid/root.sh
Performing root user operation for Oracle 11g

The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.

Entries will be added to the /etc/oratab file as needed by database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
User ignored Prerequisites during installation
OLR initialization - successful
Adding Clusterware entries to upstart
CRS-2672: Attempting to start 'ora.mdnsd' on 'hostb'
CRS-2676: Start of 'ora.mdnsd' on 'hostb' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'hostb'
CRS-2676: Start of 'ora.gpnpd' on 'hostb' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'hostb'
CRS-2672: Attempting to start 'ora.gipcd' on 'hostb'
CRS-2676: Start of 'ora.cssdmonitor' on 'hostb' succeeded
CRS-2676: Start of 'ora.gipcd' on 'hostb' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'hostb'
CRS-2672: Attempting to start 'ora.diskmon' on 'hostb'
CRS-2676: Start of 'ora.diskmon' on 'hostb' succeeded
CRS-2676: Start of 'ora.cssd' on 'hostb' succeeded
PRKO-2190 : VIP exists for node hostb, VIP name hostb-vip
PRCS-1037 : Single Client Access Name VIPs already exist
PRCS-1028 : Single Client Access Name listeners already exist
OC4J could not be created as it already exists
PRCR-1086 : resource ora.oc4j is already registered
PRCR-1086 : resource ora.cvu is already registered
PRCC-1014 : scan1 was already running
PRCR-1004 : Resource ora.scan1.vip is already running
PRCR-1079 : Failed to start resource ora.scan1.vip
CRS-5702: Resource 'ora.scan1.vip' is already running on 'hostb'

PRCC-1014 : LISTENER_SCAN1 was already running
PRCR-1004 : Resource ora.LISTENER_SCAN1.lsnr is already running
PRCR-1079 : Failed to start resource ora.LISTENER_SCAN1.lsnr
CRS-5702: Resource 'ora.LISTENER_SCAN1.lsnr' is already running on 'hostb'

PRCC-1014 : oc4j was already running
PRCR-1004 : Resource ora.oc4j is already running
PRCR-1079 : Failed to start resource ora.oc4j
CRS-5702: Resource 'ora.oc4j' is already running on 'hostb'
CRS-2672: Attempting to start 'ora.registry.acfs' on 'hostb'
CRS-2676: Start of 'ora.registry.acfs' on 'hostb' succeeded
PRCC-1014 : cvu was already running
PRCR-1004 : Resource ora.cvu is already running
PRCR-1079 : Failed to start resource ora.cvu
CRS-5702: Resource 'ora.cvu' is already running on 'hostb'
Preparing packages for installation...
cvuqdisk-1.0.9-1
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@hostb install]#


完成后的/u01/app/11.2.0/grid/cdata目录情况:

[grid@hostb cdata]$ ls -lrt
total 5612
drwxr-xr-x 2 grid oinstall      4096 Oct 30 19:06 localhost
drwxrwxr-x 2 grid oinstall      4096 Dec 22 13:27 hosta-cluster
-rw------- 1 root oinstall 272756736 Jan  8 13:38 back_hostb.olr
drwxr-xr-x 2 grid oinstall      4096 Jan 12 18:19 hostb
-rw------- 1 root oinstall 272756736 Jan 12 18:19 hostb.olr
[grid@hostb cdata]$

[grid@hostb cdata]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS      
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               ONLINE  ONLINE       hostb                                      
ora.DATAFILE.dg
               ONLINE  ONLINE       hostb                                      
ora.FRA.dg
               ONLINE  ONLINE       hostb                                      
ora.LISTENER.lsnr
               ONLINE  ONLINE       hostb                                      
ora.asm
               ONLINE  ONLINE       hostb                   Started            
ora.gsd
               OFFLINE OFFLINE      hostb                                      
ora.net1.Network ONLINE  ONLINE       hostb                                      
ora.ons
               ONLINE  ONLINE       hostb                                      
ora.registry.acfs
               ONLINE  ONLINE       hostb                                      
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       hostb                                      
ora.cvu
      1        ONLINE  ONLINE       hostb                                      
ora.orcl.db
      1        ONLINE  OFFLINE                                                  
      2        ONLINE  ONLINE       hostb                   Open               
ora.orcl.orclxa.svc
      1        ONLINE  ONLINE       hostb                                      
ora.orcla.vip
      1        ONLINE  INTERMEDIATE hostb                   FAILED OVER        
ora.hostb.vip
      1        ONLINE  ONLINE       hostb                                      
ora.oc4j
      1        ONLINE  ONLINE       hostb                                      
ora.scan1.vip
      1        ONLINE  ONLINE       hostb                                      
[grid@hostb cdata]$


[grid@hostb cdata]$ lsnrctl status

LSNRCTL for linux: version 11.2.0.3.0 - Production on 12-JAN-2016 18:21:34

Copyright (c) 1991, 2011, Oracle.  All rights reserved.

Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))
STATUS of the LISTENER
------------------------
Alias                     LISTENER
Version                   TNSLSNR for Linux: Version 11.2.0.3.0 - Production
Start Date                12-JAN-2016 18:18:21
Uptime                    0 days 0 hr. 3 min. 13 sec
Trace Level               off
Security                  ON: Local OS Authentication
SNMP                      OFF
Listener Parameter File   /u01/app/11.2.0/grid/network/admin/listener.ora
Listener Log File         /u01/app/grid/diag/tnslsnr/hostb/listener/alert/log.xml
Listening Endpoints Summary...
  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.188.182)(PORT=1528)))
  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.188.184)(PORT=1528)))
Services Summary...
Service "+ASM" has 1 instance(s).
  Instance "+ASM2", status READY, has 1 handler(s) for this service...
Service "orcl" has 1 instance(s).
  Instance "orcl2", status READY, has 1 handler(s) for this service...
Service "orclXDB" has 1 instance(s).
  Instance "orcl2", status READY, has 1 handler(s) for this service...
Service "orclxa" has 1 instance(s).
  Instance "orcl2", status READY, has 1 handler(s) for this service...
The command completed successfully
[grid@hostb cdata]$

[oracle@hostb ~]$ sqlplus / as sysdba

SQL*Plus: Release 11.2.0.3.0 Production on Tue Jan 12 18:29:52 2016

Copyright (c) 1982, 2011, Oracle.  All rights reserved.


Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options

SQL> select open_mode from v$database;

OPEN_MODE
--------------------
READ WRITE



Oracle RAC无备份情况恢复ocr和olr


原文地址:
http://www.2cto.com/database/201404/290110.html      

今天是2014-04-01,忙碌了一天终于有时间写点东西了。前段时间写了RAc中ocr存在备份的情况下如何恢复,今天写一下在没有备份情况下如何去重建ocr和olr呢?
最大的悲剧莫过于数据库坏了备份没了。让我想起了前几年流行的一句话:“最大的悲剧是人活着,钱没了”。总之备份重于一切。
情景介绍:
在11G中ocr的asm磁盘组被用户不小心删了且所有备份都误删,olr呢也损坏了。至此悲剧重演。
恢复过程:(其实就是使用root.sh重建ocr的过程,重建之后可能需要重新注册相关资源如listener or database 实例等!)
1、清空所有节点cluster配置信息:(11G中存在rootcrs.sh脚本,该脚本默认存在于$GRID_HOME/crs/install)。
注,该脚本需要在root下执行,否则会有错误提示
[grid@rac-one install]$ ./rootcrs.pl
You must be logged in as root to run this script.
Log in as root and rerun this script.
2014-04-01 17:08:12: Not running as authorized user
Insufficient privileges to execute this script.
root or administrative privileges needed to run the script.
了解该脚本的功能可以查看-help。如升级、回退、重新配置等待》
[root@rac-two install]# ./rootcrs.pl  -help
Usage:
      rootcrs.pl [-verbose] [-upgrade [-force] | -patch]
                 [-paramfile ]
                 [-deconfig [-deinstall] [-keepdg] [-force] [-lastnode]]
                 [-downgrade -oldcrshome -version [-force] [-lastnode]] 
                 [-unlock [-crshome ] [-nocrsstop]]
                 [-init]
 
      Options:
       -verbose    Run this script in verbose mode
       -upgrade    Oracle HA is being upgraded from previous version
       -patch      Oracle HA is being upgraded to a patch version
       -paramfile  Complete path of file specifying HA parameter values
       -lastnode   Force the node this script is executing on to be considered
                   as the last node of deconfiguration or downgrade, and perform
                   actions associated with deconfiguring or downgrading the
                   last node
       -downgrade  Downgrade the clusterware
       -version    For use with downgrade; special handling is required if
                   downgrading to 9i. This is the old crs version in the format
                   A.B.C.D.E (e.g 11.1.0.6.0).
       -deconfig   Remove Oracle Clusterware to allow it to be uninstalled or reinstalled
       -force      Force the execution of steps in delete or dwongrade that cannot
                   be verified to be safe
       -deinstall  Reset the permissions on CRS home during de-configuration
       -keepdg     Keep existing diskgroups during de-configuration
       -unlock     Unlock CRS home
       -crshome    Complete path of crs home. Use with unlock option
       -oldcrshome For use with downgrade. Complete path of the old crs home
       -nocrsstop  used with unlock option to reset permissions on an inactive grid home
       -init       Reset the permissions of all files and directories under CRS home
 
 
      If neither -upgrade nor -patch is supplied, a new install is performed
 
      To see the full manpage for this program, execute:
        perldoc rootcrs.pl
 
[root@rac-two install]#
在root下清除所有节点cluster 信息:
节点2:
[grid@rac-one install]$ su
Password:
[root@rac-one install]# ./rootcrs.pl -deconfig -force
Using configuration parameter file: ./crsconfig_params
Network exists: 1/192.168.4.0/255.255.255.0/eth0, type static
VIP exists: /rac-one-vip/192.168.4.113/192.168.4.0/255.255.255.0/eth0, hosting node rac-one
VIP exists: /rac-two-vip/192.168.4.114/192.168.4.0/255.255.255.0/eth0, hosting node rac-two
GSD exists
ONS exists: Local port 6100, remote port 6200, EM port 2016
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'rac-one'
CRS-2677: Stop of 'ora.registry.acfs' on 'rac-one' succeeded
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac-one'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac-one'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac-one'
CRS-2673: Attempting to stop 'ora.oc4j' on 'rac-one'
CRS-2673: Attempting to stop 'ora.GIDG.dg' on 'rac-one'
CRS-2673: Attempting to stop 'ora.DATADG.dg' on 'rac-one'
CRS-2677: Stop of 'ora.DATADG.dg' on 'rac-one' succeeded
CRS-2677: Stop of 'ora.GIDG.dg' on 'rac-one' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac-one'
CRS-2677: Stop of 'ora.oc4j' on 'rac-one' succeeded
CRS-2672: Attempting to start 'ora.oc4j' on 'rac-two'
CRS-2677: Stop of 'ora.asm' on 'rac-one' succeeded
CRS-2676: Start of 'ora.oc4j' on 'rac-two' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac-one' has completed
CRS-2677: Stop of 'ora.crsd' on 'rac-one' succeeded
CRS-2673: Attempting to stop 'ora.crf' on 'rac-one'
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac-one'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac-one'
CRS-2673: Attempting to stop 'ora.asm' on 'rac-one'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac-one'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac-one'
CRS-2677: Stop of 'ora.crf' on 'rac-one' succeeded
CRS-2677: Stop of 'ora.evmd' on 'rac-one' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rac-one' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rac-one' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac-one' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac-one'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac-one' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac-one'
CRS-2677: Stop of 'ora.cssd' on 'rac-one' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac-one'
CRS-2677: Stop of 'ora.drivers.acfs' on 'rac-one' succeeded
CRS-2677: Stop of 'ora.gipcd' on 'rac-one' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac-one'
CRS-2677: Stop of 'ora.gpnpd' on 'rac-one' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac-one' has completed
CRS-4133: Oracle High Availability Services has been stopped.
Removing Trace File Analyzer
Successfully deconfigured Oracle clusterware stack on this node
[root@rac-one install]#
节点1,因为我的RAC是双节点的,那么在清除最后一个使用lastnode参数:
[grid@rac-two crs]$ cd install/
[grid@rac-two install]$ su
Password:
[root@rac-two install]# ./rootcrs.pl -deconfig -force -lastnode
Using configuration parameter file: ./crsconfig_params
CRS resources for listeners are still configured
Network exists: 1/192.168.4.0/255.255.255.0/eth0, type static
VIP exists: /rac-two-vip/192.168.4.114/192.168.4.0/255.255.255.0/eth0, hosting node rac-two
GSD exists
ONS exists: Local port 6100, remote port 6200, EM port 2016
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'rac-two'
CRS-2677: Stop of 'ora.registry.acfs' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.crsd' on 'rac-two'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac-two'
CRS-2673: Attempting to stop 'ora.oc4j' on 'rac-two'
CRS-2673: Attempting to stop 'ora.GIDG.dg' on 'rac-two'
CRS-2673: Attempting to stop 'ora.DATADG.dg' on 'rac-two'
CRS-2677: Stop of 'ora.DATADG.dg' on 'rac-two' succeeded
CRS-2677: Stop of 'ora.oc4j' on 'rac-two' succeeded
CRS-2677: Stop of 'ora.GIDG.dg' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac-two'
CRS-2677: Stop of 'ora.asm' on 'rac-two' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac-two' has completed
CRS-2677: Stop of 'ora.crsd' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac-two'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac-two'
CRS-2673: Attempting to stop 'ora.asm' on 'rac-two'
CRS-2677: Stop of 'ora.evmd' on 'rac-two' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac-two'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac-two' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac-two'
CRS-2677: Stop of 'ora.cssd' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac-two'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac-two'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac-two'
CRS-2676: Start of 'ora.diskmon' on 'rac-two' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac-two' succeeded
CRS-4611: Successful deletion of voting disk +GIDG.
ASM de-configuration trace file location: /tmp/asmcadc_clean2014-04-01_05-14-52-PM.log
ASM Clean Configuration START
ASM Clean Configuration END
 
ASM with SID +ASM1 deleted successfully. Check /tmp/asmcadc_clean2014-04-01_05-14-52-PM.log for details.
 
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac-two'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac-two'
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac-two'
CRS-2673: Attempting to stop 'ora.asm' on 'rac-two'
CRS-2677: Stop of 'ora.mdnsd' on 'rac-two' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rac-two' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac-two'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac-two'
CRS-2677: Stop of 'ora.cssd' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.crf' on 'rac-two'
CRS-2677: Stop of 'ora.crf' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac-two'
CRS-2677: Stop of 'ora.gipcd' on 'rac-two' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac-two'
CRS-2677: Stop of 'ora.gpnpd' on 'rac-two' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac-two' has completed
CRS-4133: Oracle High Availability Services has been stopped.
Removing Trace File Analyzer
Successfully deconfigured Oracle clusterware stack on this node
[root@rac-two install]#
二、重建ocr和olr,使用root.sh脚本完成重建,其实这就是在安装RAC中执行的脚本,默认位置为:$GRID_HOME/;
eg:
节点1:
[root@rac-two grid]# ./root.sh
Performing root user operation for Oracle 11g
 
The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/11.2.0/grid
 
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.
 
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
User ignored Prerequisites during installation
Installing Trace File Analyzer
CRS-2672: Attempting to start 'ora.mdnsd' on 'rac-two'
CRS-2676: Start of 'ora.mdnsd' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rac-two'
CRS-2676: Start of 'ora.gpnpd' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac-two'
CRS-2672: Attempting to start 'ora.gipcd' on 'rac-two'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac-two' succeeded
CRS-2676: Start of 'ora.gipcd' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac-two'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac-two'
CRS-2676: Start of 'ora.diskmon' on 'rac-two' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac-two' succeeded
 
ASM created and started successfully.
 
Disk Group GIDG created successfully.
 
clscfg: -install mode specified
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Successful addition of voting disk 498646ba39604f86bf697c9748a67697.
Successful addition of voting disk 2e1bd16f9e6d4f36bf93550dc8268725.
Successful addition of voting disk 3fbd31a0b2634feabfa1115a504cbbe6.
Successfully replaced voting disk group with +GIDG.
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   498646ba39604f86bf697c9748a67697 (/dev/asm-diske) [GIDG]
 2. ONLINE   2e1bd16f9e6d4f36bf93550dc8268725 (/dev/asm-diskd) [GIDG]
 3. ONLINE   3fbd31a0b2634feabfa1115a504cbbe6 (/dev/asm-diskf) [GIDG]
Located 3 voting disk(s).
CRS-2672: Attempting to start 'ora.asm' on 'rac-two'
CRS-2676: Start of 'ora.asm' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.GIDG.dg' on 'rac-two'
CRS-2676: Start of 'ora.GIDG.dg' on 'rac-two' succeeded
Preparing packages for installation...
cvuqdisk-1.0.9-1
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@rac-two grid]#
另外注意,如果asm磁盘损坏了那么需要重新修复磁盘,然后会自动重新创建之前的diskgroup.
至此olr和ocr创建成功
节点2:
[root@rac-one grid]# ./root.sh
Performing root user operation for Oracle 11g
 
The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/11.2.0/grid
 
Enter the full pathname of the local bin directory: [/usr/local/bin]:
The contents of "dbhome" have not changed. No need to overwrite.
The contents of "oraenv" have not changed. No need to overwrite.
The contents of "coraenv" have not changed. No need to overwrite.
 
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_params
User ignored Prerequisites during installation
Installing Trace File Analyzer
OLR initialization - successful
Adding Clusterware entries to upstart
CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node rac-two, number 1, and is terminating
An active cluster was found during exclusive startup, restarting to join the cluster
Preparing packages for installation...
cvuqdisk-1.0.9-1
Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@rac-one grid]#
3、检查资源信息:
现在呢开始使用srvctl和crsctl工具,这是oracle RAC 经常使用的,另外还有一个工具是oifcfg配置网卡信息等。不在 介绍了。

[root@rac-one bin]# ./crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[root@rac-one bin]# ./crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS      
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.GIDG.dg
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
ora.asm
               ONLINE  ONLINE       rac-one                  Started            
               ONLINE  ONLINE       rac-two                  Started            
ora.gsd
               OFFLINE OFFLINE      rac-one                                     
               OFFLINE OFFLINE      rac-two                                     
ora.net1.network
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
ora.ons
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
ora.registry.acfs
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       rac-one                                     
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       rac-two                                     
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       rac-two                                     
ora.cvu
      1        ONLINE  ONLINE       rac-two                                     
ora.oc4j
      1        ONLINE  ONLINE       rac-two                                     
ora.rac-one.vip
      1        ONLINE  ONLINE       rac-one                                     
ora.rac-two.vip
      1        ONLINE  ONLINE       rac-two                                     
ora.scan1.vip
      1        ONLINE  ONLINE       rac-one                                     
ora.scan2.vip
      1        ONLINE  ONLINE       rac-two                                     
ora.scan3.vip
      1        ONLINE  ONLINE       rac-two                                     
[root@rac-one bin]#
[root@rac-one bin]# su - grid
[grid@rac-one ~]$ sqlplus / as sysasm
 
SQL*Plus: Release 11.2.0.4.0 Production on Tue Apr 1 17:40:53 2014
 
Copyright (c) 1982, 2013, Oracle.  All rights reserved.
 
 
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
 
SQL> select name,state from v$asm_diskgroup;
 
NAME                           STATE
------------------------------ -----------
GIDG                           MOUNTED
DATADG                         DISMOUNTED
 
SQL> alter diskgroup datadg mount;
 
Diskgroup altered.
 
SQL>
4、添加资源
可以看到目前启动的缺少本地监听和database资源和实例,那么下面开始重新注册这些信息到ocr中,
注册listener:
注:使用grid用户:


[grid@rac-one ~]$ srvctl add listener -l listener
PRCN-2061 : Failed to add listener ora.LISTENER.lsnr
PRCN-2065 : Port(s) 1521 are not available on the nodes given
PRCN-2067 : Port 1521 is not available across node(s) "rac-two-vip"
[grid@rac-one ~]$
提示1521该端口被ora.rac-two.vip所使用,暂且跳过。

[grid@rac-one ~]$ crsctl stop resource ora.rac-two.vip
CRS-2673: Attempting to stop 'ora.rac-two.vip' on 'rac-two'
CRS-2677: Stop of 'ora.rac-two.vip' on 'rac-two' succeeded
[grid@rac-one ~]$ srvctl add listener -l listener
[grid@rac-one ~]$ crsctl start resource ora.rac-two.vip
CRS-2672: Attempting to start 'ora.rac-two.vip' on 'rac-two'
CRS-2676: Start of 'ora.rac-two.vip' on 'rac-two' succeeded
[grid@rac-one ~]$ srvctl config listener
Name: LISTENER
Network: 1, Owner: grid
Home: 
End points: TCP:1521
[grid@rac-one ~]$
[grid@rac-one ~]$
[grid@rac-one ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS      
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATADG.dg
               ONLINE  ONLINE       rac-one                                     
               OFFLINE OFFLINE      rac-two                                     
ora.GIDG.dg
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
ora.LISTENER.lsnr
               OFFLINE OFFLINE      rac-one                                     
               OFFLINE OFFLINE      rac-two                                     
ora.asm
               ONLINE  ONLINE       rac-one                  Started            
               ONLINE  ONLINE       rac-two                  Started            
ora.gsd
               OFFLINE OFFLINE      rac-one                                     
               OFFLINE OFFLINE      rac-two                                     
ora.net1.network
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
ora.ons
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
ora.registry.acfs
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       rac-one                                     
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       rac-two                                     
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       rac-two                                     
ora.cvu
      1        ONLINE  ONLINE       rac-two                                     
ora.oc4j
      1        ONLINE  ONLINE       rac-two                                     
ora.rac-one.vip
      1        ONLINE  ONLINE       rac-one                                     
ora.rac-two.vip
      1        ONLINE  ONLINE       rac-two                                     
ora.scan1.vip
      1        ONLINE  ONLINE       rac-one                                     
ora.scan2.vip
      1        ONLINE  ONLINE       rac-two                                     
ora.scan3.vip
      1        ONLINE  ONLINE       rac-two                                     
[grid@rac-one ~]$
至此监听添加到ocr完毕。下面开始添加db,注意使用oracle用户:
继续添加:


[grid@rac-one ~]$ su - oracle
Password:
[oracle@rac-one ~]$ srvctl add database -h
 
Adds a database configuration to the Oracle Clusterware.
 
Usage: srvctl add database -d -o [-c {RACONENODE | RAC | SINGLE} [-e ] [-i ] [-w ]] [-m ] [-p ] [-r {PRIMARY | PHYSICAL_STANDBY | LOGICAL_STANDBY | SNAPSHOT_STANDBY}] [-s ] [-t ] [-n ] [-y {AUTOMATIC | MANUAL | NORESTART}] [-g ""] [-x ] [-a ""] [-j ""]
    -d      Unique name for the database
    -o         ORACLE_HOME path
    -c                Type of database: RAC One Node, RAC, or Single Instance
    -e         Candidate server list for RAC One Node database
    -i           Instance name prefix for administrator-managed RAC One Node database (default first 12 characters of )
    -w             Online relocation timeout in minutes
    -x           Node name. -x option is specified for single-instance databases
    -m              Domain for database. Must be set if database has DB_DOMAIN set.
    -p              Server parameter file path
    -r                Role of the database (primary, physical_standby, logical_standby, snapshot_standby)
    -s       Startup options for the database. Examples of startup options are OPEN, MOUNT, or 'READ ONLY'.
    -t        Stop options for the database. Examples of shutdown options are NORMAL, TRANSACTIONAL, IMMEDIATE, or ABORT.
    -n             Database name (DB_NAME), if different from the unique name given by the -d option
    -y            Management policy for the database (AUTOMATIC, MANUAL, or NORESTART)
    -g ""   Comma separated list of database server pool names
    -a ""    Comma separated list of disk groups
    -j ""    Comma separated list of ACFS paths where database's dependency will be set
    -h                       Print usage
[oracle@rac-one ~]$ srvctl add database -d Rac -o /u01/app/oracle/product/11.2.0/db_1/ -c RAC
[oracle@rac-one ~]$
[oracle@rac-one ~]$ srvctl add instance -h
 
Adds a database instance configuration to the Oracle Clusterware.
 
Usage: srvctl add instance -d -i -n [-f]
    -d      Unique name for the database
    -i                Instance name
    -n           Node name
    -f                       Force the add operation even though some resource(s) will be stopped
    -h                       Print usage
[oracle@rac-one ~]$ srvctl add instance -d Rac -i Rac1 -n rac-two
[oracle@rac-one ~]$ srvctl add instance -d Rac -i Rac2 -n rac-one
[oracle@rac-one ~]$
[oracle@rac-one ~]$ srvctl config database -d Rac
Database unique name: Rac
Database name:
Oracle home: /u01/app/oracle/product/11.2.0/db_1/
Oracle user: oracle
Spfile:
Domain:
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools: Rac
Database instances: Rac2,Rac1
Disk Groups:
Mount point paths:
Services:
Type: RAC
Database is administrator managed
[oracle@rac-one ~]$


重启crs。

[root@rac-one bin]# ./crsctl start cluster -all
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac-one'
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rac-two'
CRS-2676: Start of 'ora.cssdmonitor' on 'rac-one' succeeded
CRS-2676: Start of 'ora.cssdmonitor' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rac-two'
CRS-2672: Attempting to start 'ora.cssd' on 'rac-one'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac-one'
CRS-2672: Attempting to start 'ora.diskmon' on 'rac-two'
CRS-2676: Start of 'ora.diskmon' on 'rac-one' succeeded
CRS-2676: Start of 'ora.diskmon' on 'rac-two' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac-two' succeeded
CRS-2676: Start of 'ora.cssd' on 'rac-one' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'rac-two'
CRS-2672: Attempting to start 'ora.ctssd' on 'rac-one'
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rac-one'
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rac-two'
CRS-2676: Start of 'ora.ctssd' on 'rac-one' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'rac-one'CRS-2676: Start of 'ora.ctssd' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'rac-two'
CRS-2676: Start of 'ora.evmd' on 'rac-one' succeeded
CRS-2676: Start of 'ora.evmd' on 'rac-two' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rac-one' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac-one'
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rac-two'
CRS-2676: Start of 'ora.asm' on 'rac-two' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rac-two'
CRS-2676: Start of 'ora.crsd' on 'rac-two' succeeded
CRS-2676: Start of 'ora.asm' on 'rac-one' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rac-one'
CRS-2676: Start of 'ora.crsd' on 'rac-one' succeeded
再次查看(等2分钟):


[root@rac-one bin]# ./crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS      
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATADG.dg
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
ora.GIDG.dg
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
ora.LISTENER.lsnr
               ONLINE  ONLINE       rac-one                                     
               ONLINE  INTERMEDIATE rac-two                  Not All Endpoints R
                                                             egistered          
ora.asm
               ONLINE  ONLINE       rac-one                  Started            
               ONLINE  ONLINE       rac-two                  Started            
ora.gsd
               OFFLINE OFFLINE      rac-one                                     
               OFFLINE OFFLINE      rac-two                                     
ora.net1.network
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
ora.ons
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
ora.registry.acfs
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       rac-one                                     
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       rac-two                                     
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       rac-two                                     
ora.cvu
      1        ONLINE  ONLINE       rac-two                                     
ora.oc4j
      1        ONLINE  ONLINE       rac-two                                     
ora.rac-one.vip
      1        ONLINE  ONLINE       rac-one                                     
ora.rac-two.vip
      1        ONLINE  ONLINE       rac-two                                     
ora.rac.db
      1        ONLINE  ONLINE       rac-two                  Open               
      2        ONLINE  ONLINE       rac-one                  Open               
ora.scan1.vip
      1        ONLINE  ONLINE       rac-one                                     
ora.scan2.vip
      1        ONLINE  ONLINE       rac-two                                     
ora.scan3.vip
      1        ONLINE  ONLINE       rac-two


为什么会存在一个监听在rac-two上无法注册呢?
查看该节点监听:


[grid@rac-two admin]$ ps -ef | grep LISTENER
grid      6120     1  0 17:02 ?        00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER -inherit
grid      8511     1  0 18:28 ?        00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN1 -inherit
grid     11010     1  0 18:42 ?        00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER -inherit
grid     11246  9191  0 18:44 pts/1    00:00:00 grep LISTENER
原来是启动了两个,全部停掉再次重启:

[grid@rac-two admin]$ id
uid=1100(grid) gid=1000(oinstall) groups=1000(oinstall),1100(asmadmin),1200(dba),1300(asmdba),1301(asmoper)
[grid@rac-two admin]$ lsnrctl stop
 
LSNRCTL for Linux: Version 11.2.0.4.0 - Production on 01-APR-2014 18:45:11
 
Copyright (c) 1991, 2013, Oracle.  All rights reserved.
 
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=LISTENER)))
The command completed successfully
[grid@rac-two admin]$ ps -ef | grep LISTENER
grid      6120     1  0 17:02 ?        00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER -inherit
grid      8511     1  0 18:28 ?        00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN1 -inherit
grid     11452  9191  0 18:45 pts/1    00:00:00 grep LISTENER
[grid@rac-two admin]$ exit
logout
[root@rac-two bin]# kill -9 6120
[root@rac-two bin]# ps -ef | grep LISTENER
grid      8511     1  0 18:28 ?        00:00:00 /u01/app/11.2.0/grid/bin/tnslsnr LISTENER_SCAN1 -inherit
root     11518 11280  0 18:46 pts/1    00:00:00 grep LISTENER
[root@rac-two bin]# su - grid
[grid@rac-two ~]$ crsctl status res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS      
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATADG.dg
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
ora.GIDG.dg
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
ora.LISTENER.lsnr
               ONLINE  ONLINE       rac-one                                     
               OFFLINE OFFLINE      rac-two                                     
ora.asm
               ONLINE  ONLINE       rac-one                  Started            
               ONLINE  ONLINE       rac-two                  Started            
ora.gsd
               OFFLINE OFFLINE      rac-one                                     
               OFFLINE OFFLINE      rac-two                                     
ora.net1.network
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
ora.ons
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
ora.registry.acfs
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       rac-two                                     
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       rac-one                                     
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       rac-one                                     
ora.cvu
      1        ONLINE  ONLINE       rac-one                                     
ora.oc4j
      1        ONLINE  ONLINE       rac-one                                     
ora.rac-one.vip
      1        ONLINE  ONLINE       rac-one                                     
ora.rac-two.vip
      1        ONLINE  ONLINE       rac-two                                     
ora.rac.db
      1        ONLINE  ONLINE       rac-two                  Open               
      2        ONLINE  ONLINE       rac-one                  Open               
ora.scan1.vip
      1        ONLINE  ONLINE       rac-two                                     
ora.scan2.vip
      1        ONLINE  ONLINE       rac-one                                     
ora.scan3.vip
      1        ONLINE  ONLINE       rac-one                                     
[grid@rac-two ~]$ crsctl start resource ora.LISTENER.lsnr
CRS-2672: Attempting to start 'ora.LISTENER.lsnr' on 'rac-two'
CRS-2676: Start of 'ora.LISTENER.lsnr' on 'rac-two' succeeded
[grid@rac-two ~]$
[grid@rac-two ~]$ crsctl stat res -t
--------------------------------------------------------------------------------
NAME           TARGET  STATE        SERVER                   STATE_DETAILS      
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATADG.dg
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
ora.GIDG.dg
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
ora.LISTENER.lsnr
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
ora.asm
               ONLINE  ONLINE       rac-one                  Started            
               ONLINE  ONLINE       rac-two                  Started            
ora.gsd
               OFFLINE OFFLINE      rac-one                                     
               OFFLINE OFFLINE      rac-two                                     
ora.net1.network
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
ora.ons
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
ora.registry.acfs
               ONLINE  ONLINE       rac-one                                     
               ONLINE  ONLINE       rac-two                                     
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       rac-two                                     
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       rac-one                                     
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       rac-one                                     
ora.cvu
      1        ONLINE  ONLINE       rac-one                                     
ora.oc4j
      1        ONLINE  ONLINE       rac-one                                     
ora.rac-one.vip
      1        ONLINE  ONLINE       rac-one                                     
ora.rac-two.vip
      1        ONLINE  ONLINE       rac-two                                     
ora.rac.db
      1        ONLINE  ONLINE       rac-two                  Open               
      2        ONLINE  ONLINE       rac-one                  Open               
ora.scan1.vip
      1        ONLINE  ONLINE       rac-two                                     
ora.scan2.vip
      1        ONLINE  ONLINE       rac-one                                     
ora.scan3.vip
      1        ONLINE  ONLINE       rac-one                                     
[grid@rac-two ~]$


至此所有问题得到彻底解决。



Oracle database 11g rac损坏ocr和votedisk恢复实验 
本人的操作环境:oracle database rdbms 11g rac on OEL5.5
查看一下表决磁盘和ocr的信息:
[root@rac1 bin]# pwd
/u01/app/11.2.0/grid/bin
[root@rac1 bin]# ./crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   5122b184495d4fe9bf1fad29647807ba (ORCL:VOL1) [OCRVOTI]
Located 1 voting disk(s).
[root@rac1 bin]# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          3
         Total space (kbytes)     :     262120
         Used space (kbytes)      :       2700
         Available space (kbytes) :     259420
         ID                       :  268167937
         Device/File Name         :   +OCRVOTI
                                    Device/File integrity check succeeded
                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check succeeded
查看当前ocr备份情况,ocr
[root@rac1 bin]# ./ocrconfig -showbackup

rac2     2014/03/25 12:04:28     /u01/app/11.2.0/grid/cdata/rac-cluster/backup00.ocr

rac2     2014/03/21 16:16:32     /u01/app/11.2.0/grid/cdata/rac-cluster/backup01.ocr

rac2     2014/03/21 12:16:31     /u01/app/11.2.0/grid/cdata/rac-cluster/backup02.ocr

rac2     2014/03/25 12:04:28     /u01/app/11.2.0/grid/cdata/rac-cluster/day.ocr

rac2     2014/03/19 14:26:16     /u01/app/11.2.0/grid/cdata/rac-cluster/week.ocr
可以进行手工备份:
[root@rac1 bin]# ./ocrconfig -local -manualbackup
rac1     2014/03/25 14:33:39     /u01/app/11.2.0/grid/cdata/rac1/backup_20140325_143339.olr

rac1     2014/03/25 10:34:33     /u01/app/11.2.0/grid/cdata/rac1/backup_20140325_103433.olr

rac1     2014/03/25 09:53:32     /u01/app/11.2.0/grid/cdata/rac1/backup_20140325_095332.olr

rac1     2014/03/25 09:53:18     /u01/app/11.2.0/grid/cdata/rac1/backup_20140325_095318.olr

rac1     2014/03/18 10:57:51     /u01/app/11.2.0/grid/cdata/rac1/backup_20140318_105751.olr

在asmcmd的md_backup命令备份磁盘组,顺便查看该磁盘组都存放什么??
[grid@rac1 ~]$ asmcmd -p
ASMCMD [+] > md_backup /home/grid/ocrvote2.bak -G OCRVOTI
Disk group metadata to be backed up: OCRVOTI
Current alias directory path: rac-cluster
Current alias directory path: rac-cluster/ASMPARAMETERFILE
Current alias directory path: rac-cluster/OCRFILE
也可以手工导出ocr内容
[root@rac1 bin]# ./ocrconfig -export /home/grid/ocr2.bak
我们可以破坏存放ocr的设备文件
[root@rac1 bin]# dd if=/dev/zero of=/dev/sdg bs=1024k count=1
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.002366 seconds, 443 MB/s
然哈停止集群:
[root@rac1 bin]# ./crsctl stop has
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac1'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'rac1'
CRS-2673: Attempting to stop 'ora.oc4j' on 'rac1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac1'
CRS-2673: Attempting to stop 'ora.OCRVOTI.dg' on 'rac1'
CRS-2673: Attempting to stop 'ora.registry.acfs' on 'rac1'
CRS-2673: Attempting to stop 'ora.test.db' on 'rac1'
CRS-2673: Attempting to stop 'ora.gsd' on 'rac1'
CRS-2677: Stop of 'ora.gsd' on 'rac1' succeeded
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac1' su 忽略。。。。。
我们在启动clusterware 发现无法启动了
[root@rac1 bin]# ./crsctl start has
CRS-4123: Oracle High Availability Services has been started.
[root@rac1 bin]# ./crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
CRS-4534: Cannot communicate with Event Manager

ocr和vote disk损坏恢复步骤大致如下:

1)停止所有节点clusterware
# crsctl stop crs
# crsctl stop crs -f
2)以root用户在其中一个节点独占模式启动clusterware
# crsctl start crs -excl -nocrs
备注:如果发现crsd在运行,那么通过如下命令将之停止。
# crsctl stop resource ora.crsd -init
3)创建新的存放ocr和vote disk的磁盘组,磁盘组名和原有的一致(如果想改变位置,需修改/etc/oracle/ocr.loc文件)
备注:如发现无法创建等情况,可以采用如下删除磁盘组等排错思路
SQL> drop diskgroup disk_group_name force including contents;
4)还原ocr,并检查
# ocrconfig -restore file_name
# ocrcheck
5)恢复表决磁盘,并检查
# crsctl replace votedisk +asm_disk_group
# crsctl query css votedisk
6)停止独占模式运行的clusterware
# crsctl stop crs -f
7)所有节点正常启动clusterware
# crsctl start crs
8)CVU验证所有RAC节点OCR的完整性
$ cluvfy comp ocr -n all -verbose





About Me

...............................................................................................................................

● 本文整理自网络

● 本文在itpub(http://blog.itpub.net/26736162)、博客园(http://www.cnblogs.com/lhrbest)和个人微信公众号(xiaomaimiaolhr)上有同步更新

● 本文itpub地址:http://blog.itpub.net/26736162/abstract/1/

● 本文博客园地址:http://www.cnblogs.com/lhrbest

● 本文pdf版及小麦苗云盘地址:http://blog.itpub.net/26736162/viewspace-1624453/

● 数据库笔试面试题库及解答:http://blog.itpub.net/26736162/viewspace-2134706/

● QQ群:230161599     微信群:私聊

● 联系我请加QQ好友(646634621),注明添加缘由

● 于 2017-06-02 09:00 ~ 2017-06-30 22:00 在魔都完成

● 文章内容来源于小麦苗的学习笔记,部分整理自网络,若有侵权或不当之处还请谅解

● 版权所有,欢迎分享本文,转载请保留出处

...............................................................................................................................

拿起手机使用微信客户端扫描下边的左边图片来关注小麦苗的微信公众号:xiaomaimiaolhr,扫描右边的二维码加入小麦苗的QQ群,学习最实用的数据库技术。

img_e3029f287d989cd04bd75432ecc1c172.png
DBA笔试面试讲解
欢迎与我联系

网友评论

登录后评论
0/500
评论
小麦苗
+ 关注