jump to navigation

PRKP-1001 : Error starting instance …. followed by CRS-0215 31 December 2009

Posted by David Alejo Marcos in ASM, RAC, RMAN.
Tags: , ,
trackback

Well, I am sure this is going to be the last post for 2009 as somehow I am running out of time to fix more things and write about them here.

The problem:

I was asked to create a POC (Proof Of Concept) RAC database on one of our QA clusters as we may decide to share the cluster for two different businesses.

The specification was quite simple, 3 node RAC running on Linux, ASM (using ASMLib) and Oracle 10.2.0.4.1.

The creation of the ASM disks was quite simple, as well as the creation of the database from a full RMAN backup using the “duplicate target” command.

Once all components were registered on the cluster, I went for the start of the database:

srvctl start database -d …

I had the following error:

PRKP-1001 : Error starting instance iiiiiii on node nnnnnna
CRS-0215: Could not start resource ‘ora.dddddd.iiiiii1.inst’.

After having a look on the logfiles and in google, the answers did not look really good. Some people suggested to stop the crs (impossible for us as it was shared with QA and the POC database and could not justify the downtime). Other people suggested a patch to fix a bug (Bug 4337645), which I thought it was unlikely as it was running before without problems.

The Solution:

Well, it was quite easy now that I know the answer. The steps I followed were:

1.- Try to start the database using sqlplus:

sqlplus ‘/as sysdba’

SQL*Plus: Release 10.2.0.4.0 – Production on Tue Dec 15 10:52:39 2009

Copyright (c) 1982, 2007, Oracle.  All Rights Reserved.

Connected to an idle instance.

SQL> startup nomount
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file ‘+DATA_POC/dddddd/spfileddddd.ora’
ORA-17503: ksfdopn:2 Failed to open file +DATA_POC/dddddd/spfileddddd.ora
ORA-15077: could not locate ASM instance serving a required diskgroup
SQL> exit

2.- check the disk on ASMLib (as ROOT):

/etc/init.d/oracleasm listdisks

/etc/init.d/oracleasm querydisk

3.- I knew the ASM instance was up and running as the QA database was working. So the problem had to be on the new diskgroups. The following sql statement confirmed my suspicions, the diskgroups were not mounted:

SQL> select name, state from v$asm_diskgroup;

SQL> select name, state from v$asm_diskgroup;

NAME                           STATE
—————————— —————–
CTRLLOG                        MOUNTED
DATA                             MOUNTED
FRA                                MOUNTED
DATA_POC                      DISMOUNTED
FRA_POC                         DISMOUNTED
LOGCTL_POC                   DISMOUNTED

3.- Mounting the diskgroups (on all 3 ASM instances) was fairly simple:

SQL> alter diskgroup data_poc mount;

Diskgroup altered.

SQL>  select name, state from v$asm_diskgroup;

NAME                           STATE
—————————— —————–
CTRLLOG                        MOUNTED
DATA                             MOUNTED
FRA                                MOUNTED
DATA_POC                      MOUNTED
FRA_POC                         DISMOUNTED
LOGCTL_POC                   DISMOUNTED

and the same for fra_poc and logctl_poc.

once all 3 diskgroups were mounted on all 3 nodes, I try to start the database again using srvctl.

Couple of things:

1.- ASMLib have to be execute on all servers

2.- ASM Diskgroups have to be mounted and all ASM instances.

3.- When moving from a single database to a RAC database, remember to set the cluster_database and any other RAC related parameters.

Happy new year.

About these ads

Comments»

1. Sunil - 14 May 2010

Hey.. RAC Guru….

Tou are doing excellent job…

2. Peter Michie - 20 February 2012

This is a generic message. In my case for some reason LOCAL_LISTENER entry was missing from tnsnames.ora

3. Najeeb Ahmed - 21 December 2012

Thank u so much David… was new to rac, was stuck on this starting the instance, understood the concept of the asm and database behind the scenes…. thanks


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: