Oracle RAC 10.2.0.5升级到11.2.0.4遇到的问题

最近有这个需求所以提前在家里的测试环境做了相关的测试，将Oracle RAC 10.2.0.5 for Linux升级到Oracle RAC 11.2.0.4，不对物理结构做任何变更，直接升级RAC。升级的过程大致分为如下几步：

1）.升级Grid Infrastructure（Clusterware）。 <<<<如下图所示，直接用Grid Infrastructure 11.2.0.4版本的安装介质升级。

2）.升级ASM。 <<<< 升级GI可以附带一起就升级了ASM，如下图所示，选中“Upgrade Cluster Oracle Automatic Storage Management（Oracle ASM）”即可。

3）.升级Oracle Database软件。 <<<< 升级Oracle Database软件最简单，采用out-of-place的方式新装一套DB软件即可。
4）.升级数据字典。 <<<< 升级数据字典也较为简单，主要工具是Pre-Upgrade Tools脚本，和单机版数据库升级没有太大区别。

测试的整个升级过程在自动升级ASM的时候卡了一下，其他步骤都较为顺利，下面就将ASM升级遇到的问题做一下总结。

ASM的升级是在RAC所有节点都完成rootupgrade.sh脚本的执行，点击图形化的下一步开始进行的。这里有一点需要说明，如果执行完rootupgrade.sh脚本之后，后续的图形界面没有了（比如服务器重启等原因，那么还可以直接执行$GI_HOME/cfgtoollogs/configToolFailedCommands脚本继续完成后续配置工作。

在所有RAC节点执行完rootupgrade.sh脚本之后，通过11g查看Clusterware资源情况：
[root@rac01 bin]# ./crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.LISTENER.lsnr
ONLINE ONLINE rac01
ONLINE ONLINE rac02
ora.gsd
OFFLINE OFFLINE rac01
OFFLINE OFFLINE rac02
ora.net1.network
ONLINE ONLINE rac01
ONLINE ONLINE rac02
ora.ons
ONLINE ONLINE rac01
ONLINE ONLINE rac02
ora.registry.acfs
OFFLINE OFFLINE rac01
OFFLINE OFFLINE rac02
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE rac01
ora.cvu
1 ONLINE ONLINE rac01
ora.oc4j
1 ONLINE ONLINE rac01
ora.orcl.db
1 ONLINE ONLINE rac01 <<<< 10.2.0.5版本的DB资源
ora.orcl.orcl1.inst
1 ONLINE ONLINE rac01 <<<< 10.2.0.5版本的DB实例资源
ora.orcl.orcl2.inst
1 ONLINE ONLINE rac02 <<<< 10.2.0.5版本的DB实例资源
ora.rac01.ASM1.asm
1 ONLINE ONLINE rac01 <<<< 10.2.0.5版本的ASM实例资源
ora.rac01.vip
1 ONLINE ONLINE rac01
ora.rac02.ASM2.asm
1 ONLINE ONLINE rac02 <<<< 10.2.0.5版本的ASM实例资源
ora.rac02.vip
1 ONLINE ONLINE rac02
ora.scan1.vip
1 ONLINE ONLINE rac01

如果在升级ASM之前，10.2.0.5版本的ASM实例资源在运行，那么后续ASM升级操作将收到如下的报错：
INFO: Command /d01/grid/bin/asmca -silent -upgradeASM -oui_internal
INFO: ... GenericInternalPlugIn.handleProcess（） entered.
INFO: ... GenericInternalPlugIn: getting configAssistantParmas.
INFO: ... GenericInternalPlugIn: checking secretArguments.
INFO: ... GenericInternalPlugIn: starting read loop.
INFO: Read: ASMSNMP_PASSWORD_PROMPT
INFO: Processing: ASMSNMP_PASSWORD_PROMPT for argument tag -asmMonitorPassword
INFO: End of argument passing to stdin
INFO: Read:
INFO: Read: ORA-01034: ORACLEnotavailable
INFO: Read:
INFO: Read:
INFO: oracle.assistants.asm:The plug-in AutomaticStorageManagementConfigurationAssistanthasfailed its perform method

解决办法是停止老版本的asm实例，启动11g的ASM实例：
[root@rac01 bin]# ./crsctl stop res ora.rac01.ASM1.asm
CRS-2673: Attempting to stop "ora.rac01.ASM1.asm" on "rac01"
CRS-2677: Stop of "ora.rac01.ASM1.asm" on "rac01" succeeded

[root@rac01 bin]# ./crsctl stop res ora.rac02.ASM2.asm
CRS-2673: Attempting to stop "ora.rac02.ASM2.asm" on "rac02"
CRS-2677: Stop of "ora.rac02.ASM2.asm" on "rac02" succeeded

[root@rac01 bin]# ./crsctl start res ora.asm –init <<<< 需要在RAC所有节点执行启动ASM操作。
CRS-2672: Attempting to start "ora.drivers.acfs" on "rac01"
CRS-2676: Start of "ora.drivers.acfs" on "rac01" succeeded
CRS-2672: Attempting to start "ora.asm" on "rac01"
CRS-2676: Start of "ora.asm" on "rac01" succeeded
该问题可以参考文章：《ASM Upgrade from 10.2.0.5 to 11.2.0.3 Fails : INS-20802, ORA-1034 （文档 ID 1534279.1）》

注意：RAC可以在只升级Clusterware，不升级ASM，不升级DB的情况下正常运行，老版本的ASM、DB和LISTENER等就需要以上的兼容性资源来启动和管理。

完成以上操作后，再次执行$GRID_HOME/cfgtoollogs/configToolFailedCommands脚本，查看日志收到类似如下的报错，ASMCA升级失败：......
/opt/grid/oui/bin/runConfig.sh ORACLE_HOME=/opt/grid MODE=perform ACTION=configure RERUN=false $*
------
...
The plug-in Automatic Storage Management Configuration Assistant is running

The plug-in Automatic Storage Management Configuration Assistant has failed its perform method
......

手动执行asmca -silent -upgradeASM -asmsnmpPassword oracle123升级ASM会收到相同的报错。

该问题是个bug，参考如下的文章即可解决：
1. Rerun the command as:/opt/grid/bin/asmca -silent -postConfigureASM -oui_internal -sysAsmPassword -asmMonitorPassword Replace above asmadmin passwd and asmdba passwd with the real password.

2. Check resource status:
crsctl stat res -t

It should show all resources （except gsd and oc4j） are ONLINE now. gsd and oc4j OFFLINE is intended status for 11.2.0.1.Another workaround as per BUG 16749410 - RUNCONFIG.SH FAILS WHEN UPGRADING ASM FROM 11.1 TO 11.2:

1. Create a response file for configToolAllCommands named cfgrsp.properties, for example <GRID_HOME>/cfgtoollogs/cfgrsp.properties. This file must be populated with 2 entries
"oracle.assistants.asm|S_ASMPASSWORD"
"oracle.assistants.asm|S_ASMMONITORPASSWORD"

The value for S_ASMPASSWORD MUST be left blank, S_ASMMONITORPASSWORD will be set to the desired ASM Monitor Password. The file should look as follows （assuming the ASMMONITORPASSWORD will be set to oracle）: oracle.assistants.asm|S_ASMPASSWORD=
oracle.assistants.asm|S_ASMMONITORPASSWORD=oracle
2. Once the response file is created, execute configToolAllCommands as follows: <GRID_HOME>/cfgtoollogs/configToolAllCommands RESPONSE_FILE=<GRID_HOME>/cfgtoollogs/cfgrsp.properties
该问题参考文章：《11gR2 Rerun command in configToolFailedCommands Failed （文档 ID 1191814.1）》

完成整个升级过程后ASM实例的资源并不存在，asmca无法正常工作，需要以GI所有者操作系统用户执行下面的命令手动添加ASM实例：
./srvctl add asm -l LISTENER -p /u01/app/11.2.0/grid/dbs/spfile+ASM.ora -d "/dev/rhdisk*"

至此，ASMCA也可以正常工作了。

--end--更多Oracle相关信息见Oracle 专题页面 http://www.linuxidc.com/topicnews.aspx？tid=12本文永久更新链接地址