Oracle数据库shutdown immediate被hang住的几个原因
实验操作环境：
操作系统：Red Hat Enterprise Linux ES release 4 (Nahant Update 6)
数据库 ： Oracle Database 10g Release 10.2.0.4.0 – Production 32bit
今晚使用shutdown immediate（其实是执行stop_oracle.sh脚本关闭数据库，如下所示）关闭数据库的时候，
1: [oracle@gsp-orasvr02 scripts]$ more stop_oracle.sh 2: lsnrctl stop LISTENER 3: sleep 15 4: sqlplus /nolog 5: conn / as sysdba; 6: alter system switch logfile; 7: alter system checkpoint; 8: shutdown immediate; 9: exit 10: EOF
在另外一个会话中使用tail -20f 命令查看告警日志的输出，结果发现数据库等待了很长时间都没有正常关闭，，hang住在下面地方：
Active call for process 11121 user 'oracle' program 'oracle@get-orasvr02 (S000)'
Active call for process 7162 user 'oracle' program 'oracle@get-orasvr02 (S011)'
截图如下
这时解决办法是找出hang住的进程并杀掉(当时操作没有截图，也没有保存输出信息),因为有些session无法被pmon进程清理，导致数据库无法顺利关闭，需要手工杀掉进程。首先使用ps 和grep找到这两个进程。
[ oracle@get-orasvr02 bdump]$ ps -ef | grep oracle | grep S000
[ oracle@get-orasvr02 bdump]$ ps -ef | grep oracle | grep S011
然后使用kill -9 processesid杀掉这两个进程即可，杀掉这两个进程后，从告警日志里面看到里面跳到关闭dispatcher 。如下所示：
1: [ oracle@get-orasvr02 bdump]$ tail alert_epps.log 2: Current log# 3 seq# 242223 mem# 1: /u02/oradata/epps/redo03_01.log 3: Sun Jan 5 05:14:50 2014 4: Starting control autobackup 5: Control autobackup written to DISK device 6: handle '/u01/app/oracle/product/10.2.0/db_1/dbs/c-2179993557-20140105-0e' 7: Sun Jan 5 05:14:54 2014 8: ALTER SYSTEM ARCHIVE LOG 9: Sun Jan 5 05:14:55 2014 10: Thread 1 cannot allocate new log, sequence 242224 11: Checkpoint not complete 12: Current log# 3 seq# 242223 mem# 0: /u01/app/oracle/oradata/epps/redo03_1.log 13: Current log# 3 seq# 242223 mem# 1: /u02/oradata/epps/redo03_01.log 14: Sun Jan 5 05:14:58 2014 15: Thread 1 advanced to log sequence 242224 (LGWR switch) 16: Current log# 5 seq# 242224 mem# 0: /u01/app/oracle/oradata/epps/redo05_1.log 17: Current log# 5 seq# 242224 mem# 1: /u02/oradata/epps/redo05_02.log 18: Sun Jan 5 07:31:56 2014 19: Thread 1 advanced to log sequence 242225 (LGWR switch) 20: Current log# 2 seq# 242225 mem# 0: /u01/app/oracle/oradata/epps/redo02_1.log 21: Current log# 2 seq# 242225 mem# 1: /u02/oradata/epps/redo02_02.log 22: Sun Jan 5 07:32:20 2014 23: Starting background process EMN0 24: Shutting down instance: further logons disabled 25: EMN0 started with pid=43, OS id=7062 26: Sun Jan 5 07:32:21 2014 27: Stopping background process CJQ0 28: Sun Jan 5 07:32:21 2014 29: Stopping background process QMNC 30: Sun Jan 5 07:32:23 2014 31: Stopping background process MMNL 32: Sun Jan 5 07:32:34 2014 33: Background process MMNL not dead after 10 seconds 34: Sun Jan 5 07:32:34 2014 35: Killing background process MMNL 36: Sun Jan 5 07:32:35 2014 37: Stopping background process MMON 38: Sun Jan 5 07:33:05 2014 39: Background process MMON not dead after 30 seconds 40: Sun Jan 5 07:33:05 2014 41: Killing background process MMON 42: Sun Jan 5 07:33:06 2014 43: Shutting down instance (immediate) 44: License high water mark = 561 45: Sun Jan 5 07:33:06 2014 46: Stopping Job queue slave processes, flags = 7 47: Sun Jan 5 07:33:06 2014 48: Process OS id : 6088 alive after kill 49: Errors in file /u01/app/oracle/admin/epps/udump/epps_ora_7055.trc 50: Sun Jan 5 07:33:09 2014 51: Waiting for Job queue slaves to complete 52: Sun Jan 5 07:33:09 2014 53: Job queue slave processes stopped 54: Sun Jan 5 07:38:10 2014 55: Active call for process 11121 user 'oracle' program 'oracle@get-orasvr02 (S000)' 56: Active call for process 7162 user 'oracle' program 'oracle@get-orasvr02 (S011)' 57: SHUTDOWN: waiting for active calls to complete. 58: Sun Jan 5 07:57:28 2014 59: Waiting for dispatcher 'D000' to shutdown 60: Waiting for dispatcher 'D001' to shutdown 61: Waiting for dispatcher 'D002' to shutdown 62: Waiting for dispatcher 'D003' to shutdown 63: Waiting for dispatcher 'D004' to shutdown 64: Waiting for dispatcher 'D005' to shutdown 65: Waiting for dispatcher 'D006' to shutdown 66: Sun Jan 5 07:59:29 2014 67: All dispatchers and shared servers shutdown 68: Sun Jan 5 08:04:30 2014 69: SHUTDOWN: Active processes prevent shutdown operation 70: Sun Jan 5 08:09:32 2014 71: SHUTDOWN: Active processes prevent shutdown operation
Oracle的官方文档介绍、解释如下
The database is waiting for pmon to clean up processes, but pmon is unable to
clean them. The client connections to the server are causing the shutdown
immediate or normal to hang. Killing them allows pmon to clean up and release
the associated Oracle processes and resources.
What resources are we talking about?
1) Any non committed transactions must be rolled back
2) Any temporary space (sort segments / lobs / session temporary tables) must be freed
3) The session itself and any associated memory consumed by the session.
4) Internal locks / enqueues must be cleaned up
