Twenty-eighth bullet of the \\\\\\ ASM Translation Series: ASM INTERNAL Partnership and Status Table

Keywords: SQL Database Oracle

Original text: Partnership and Status Table
Author: Bane Radulovic
Translator: Qiu Dalong, an expert in the technology of Warbury Science and Technology Database, is mainly involved in the implementation, testing, maintenance and optimization of the company's products. In-depth research on SQL optimization, data migration, backup disaster recovery and so on. Rich experience in data migration, fault recovery and database performance tuning under high concurrency.
Revision: Wei Xinghua
Editorial Responsibility: Zhong Peiyi

The Partnership and Status Table for short PST table contains information about all disks in a disk group - disk number, disk status, partner's disk number, heartbeat information and failure group information (11G version and above).

The AU 1 of each disk is reserved for the PST table, but not every disk has information about the PST table.

PST count

There is only one PST table in the external redundant disk group.

In a normal redundant disk group, there are at least two PST tables. If there are three or more fail groups in the disk group, there will be three PST tables.

In high redundant disk groups, there are at least three PST tables of information. If there are four fail groups, there will be four PST tables, and if there are five or more fail groups, there will be five PST tables.

Look at the following example. Note that the disk group created consists of five disks:

Create external redundant disk groups

SQL> CREATE DISKGROUP DG1 EXTERNAL REDUNDANCY
DISK '/dev/sdc5', '/dev/sdc6', '/dev/sdc7', '/dev/sdc8', '/dev/sdc9';

Diskgroup created.

View alert logs for asm

Sat Aug 31 20:44:59 2013
SQL> CREATE DISKGROUP DG1 EXTERNAL REDUNDANCY
DISK '/dev/sdc5', '/dev/sdc6', '/dev/sdc7', '/dev/sdc8', '/dev/sdc9'
Sat Aug 31 20:44:59 2013
NOTE: Assigning number (2,0) to disk (/dev/sdc5)
NOTE: Assigning number (2,1) to disk (/dev/sdc6)
NOTE: Assigning number (2,2) to disk (/dev/sdc7)
NOTE: Assigning number (2,3) to disk (/dev/sdc8)
NOTE: Assigning number (2,4) to disk (/dev/sdc9)
...
NOTE: initiating PST update: grp = 2
Sat Aug 31 20:45:00 2013
GMON updating group 2 at 50 for pid 22, osid 9873
NOTE: group DG1: initial PST location: disk 0000 (PST copy 0)
Sat Aug 31 20:45:00 2013
NOTE: PST update grp = 2 completed successfully
...

The output of the alert log is very detailed. As you can see, ASM has created only one PST table on disk 0000, or / dev/sdc5.

Creating Normal Redundant Disk Groups

SQL> drop diskgroup DG1;

Diskgroup dropped.

SQL> CREATE DISKGROUP DG1 NORMAL REDUNDANCY
DISK '/dev/sdc5', '/dev/sdc6', '/dev/sdc7', '/dev/sdc8', '/dev/sdc9';

Diskgroup created.

Check the alert log of ASM:

Sat Aug 31 20:49:28 2013
SQL> CREATE DISKGROUP DG1 NORMAL REDUNDANCY
DISK '/dev/sdc5', '/dev/sdc6', '/dev/sdc7', '/dev/sdc8', '/dev/sdc9'
Sat Aug 31 20:49:28 2013
NOTE: Assigning number (2,0) to disk (/dev/sdc5)
NOTE: Assigning number (2,1) to disk (/dev/sdc6)
NOTE: Assigning number (2,2) to disk (/dev/sdc7)
NOTE: Assigning number (2,3) to disk (/dev/sdc8)
NOTE: Assigning number (2,4) to disk (/dev/sdc9)
...
Sat Aug 31 20:49:28 2013
NOTE: group 2 PST updated.
NOTE: initiating PST update: grp = 2
Sat Aug 31 20:49:28 2013
GMON updating group 2 at 68 for pid 22, osid 9873
NOTE: group DG1: initial PST location: disk 0000 (PST copy 0)
NOTE: group DG1: initial PST location: disk 0001 (PST copy 1)
NOTE: group DG1: initial PST location: disk 0002 (PST copy 2)
Sat Aug 31 20:49:28 2013
NOTE: PST update grp = 2 completed successfully
...

This time we saw the creation of three PST tables, located at disk 0000, disk 0001, disk 0002, i.e. / dev/sdc5, / dev/sdc6, / dev/sdc7.

High redundancy disk group

SQL> drop diskgroup DG1;

Diskgroup dropped.

SQL> CREATE DISKGROUP DG1 HIGH REDUNDANCY
DISK '/dev/sdc5', '/dev/sdc6', '/dev/sdc7', '/dev/sdc8', '/dev/sdc9';

Diskgroup created.

Check the alert log of ASM:

Sat Aug 31 20:51:52 2013
SQL> CREATE DISKGROUP DG1 HIGH REDUNDANCY
DISK '/dev/sdc5', '/dev/sdc6', '/dev/sdc7', '/dev/sdc8', '/dev/sdc9'
Sat Aug 31 20:51:52 2013
NOTE: Assigning number (2,0) to disk (/dev/sdc5)
NOTE: Assigning number (2,1) to disk (/dev/sdc6)
NOTE: Assigning number (2,2) to disk (/dev/sdc7)
NOTE: Assigning number (2,3) to disk (/dev/sdc8)
NOTE: Assigning number (2,4) to disk (/dev/sdc9)
...
Sat Aug 31 20:51:53 2013
NOTE: group 2 PST updated.
NOTE: initiating PST update: grp = 2
Sat Aug 31 20:51:53 2013
GMON updating group 2 at 77 for pid 22, osid 9873
NOTE: group DG1: initial PST location: disk 0000 (PST copy 0)
NOTE: group DG1: initial PST location: disk 0001 (PST copy 1)
NOTE: group DG1: initial PST location: disk 0002 (PST copy 2)
NOTE: group DG1: initial PST location: disk 0003 (PST copy 3)
NOTE: group DG1: initial PST location: disk 0004 (PST copy 4)
Sat Aug 31 20:51:53 2013
NOTE: PST update grp = 2 completed successfully
...

This time we saw ASM create five PST tables.

PST relocation

PST is redistributed in the following cases:

  • The disk on which the PST is located is not available (when the ASM instance starts)
  • Disk offline
  • IO error occurred in reading and writing of PST
  • Disk is normally deleted (manual or background)

In these cases, if there are other disks available in the same failed group, the PST will be reassigned to other disks; otherwise, it will be reassigned to other failed groups without a copy of the PST.

Let's look at an example:

SQL> drop diskgroup DG1;

Diskgroup dropped.

SQL> CREATE DISKGROUP DG1 NORMAL REDUNDANCY
DISK '/dev/sdc5', '/dev/sdc6', '/dev/sdc7', '/dev/sdc8';

Diskgroup created.

The alert log of ASM shows that the copy information of the PST table is located on disk 0, 1 and 2 after the disk has just been created.

NOTE: group DG1: initial PST location: disk 0000 (PST copy 0)
NOTE: group DG1: initial PST location: disk 0001 (PST copy 1)
NOTE: group DG1: initial PST location: disk 0002 (PST copy 2)

We delete disk 0:

SQL> select disk_number, name, path from v$asm_disk_stat
where group_number = (select group_number from v$asm_diskgroup_stat where name='DG1');

DISK_NUMBER NAME                           PATH
----------- ------------------------------ ----------------
          3 DG1_0003                       /dev/sdc8
          2 DG1_0002                       /dev/sdc7
          1 DG1_0001                       /dev/sdc6
          0 DG1_0000                       /dev/sdc5

SQL> alter diskgroup DG1 drop disk DG1_0000;

Diskgroup altered.

Look again at ASM's alert log:

Sat Aug 31 21:04:29 2013
SQL> alter diskgroup DG1 drop disk DG1_0000
...
NOTE: initiating PST update: grp 2 (DG1), dsk = 0/0xe9687ff6, mask = 0x6a, op = clear
Sat Aug 31 21:04:37 2013
GMON updating disk modes for group 2 at 96 for pid 24, osid 16502
NOTE: group DG1: updated PST location: disk 0001 (PST copy 0)
NOTE: group DG1: updated PST location: disk 0002 (PST copy 1)
NOTE: group DG1: updated PST location: disk 0003 (PST copy 2)
...

We saw that the copy information of PST was transferred from disk 0 to disk 3.

Disk Partners

The Partnership of a disk is on a normal or high redundant disk group, and there is a "symmetrical" relationship between the two disks. In external redundancy, there is no disk partnership. For a discussion on this topic, see How many partners (Article 6).

PST Availability

The data of the PST table is available before other ASM metadata. When a mount request for a disk group is initiated, the GMON process reads all disks in the disk group and finds and confirms a copy of the PST used. Once you confirm that there are enough PST copies, mount the disk group. From this point on, the information of the PST table is stored in the cache of the ASM instance; in the PGA of the GMON process, it is protected by PT.n.0 enqueue exclusive lock.

For other ASM instances in the same cluster, PST tables are cached in their own GMON PGA after startup, plus shared PT.n.0 enqueue.

Only GMON processes with exclusive PT enqueue locks (CKPT in 10gR1) can update PST table information on disk.

PST (GMON) tracing

Every time you try mount disk groups, the GMON trace file records PST information. Note that I'm talking about trying, not mounting. In other words, GMON records information whether mount is successful or not. This information may be valuable for Oracle Support to diagnose mount failures for disk groups.

Following is information about a disk group mount recorded in a typical GMON trace file:

=============== PST ====================
grpNum:    2
grpTyp:    2
state:     1
callCnt:   103
bforce:    0x0
(lockvalue) valid=1 ver=0.0 ndisks=3 flags=0x3 from inst=0 (I am 1) last=0
--------------- HDR --------------------
next:    7
last:    7
pst count:       3
pst locations:   1  2  3
incarn:          4
dta size:        4
version:         0
ASM version:     168820736 = 10.1.0.0.0
contenttype:     0
--------------- LOC MAP ----------------
0: dirty 0       cur_loc: 0      stable_loc: 0
1: dirty 0       cur_loc: 0      stable_loc: 0
--------------- DTA --------------------
1: sts v v(rw) p(rw) a(x) d(x) fg# = 0 addTs = 0 parts: 2 (amp) 3 (amp)
2: sts v v(rw) p(rw) a(x) d(x) fg# = 0 addTs = 0 parts: 1 (amp) 3 (amp)
3: sts v v(rw) p(rw) a(x) d(x) fg# = 0 addTs = 0 parts: 1 (amp) 2 (amp)
...

The section marked === PST === shows the disk group number (grpNum), type (grpTyp), and status. The part of the HDR tag shows the number of copies of the PST (pst count) and the disk number where it is located. DTA - shows the state of the disk on which the PST is located.

Conclusion

PST (Partnership and Status Table) contains information about all disks in a disk group - disk number, disk status, partner disk number, heartbeat information and failure group information (after 11G).

The No. 1 AU of each ASM disk is reserved for the PST table, but only some disks have PST data. Since PST is an important ASM metadata, as long as the number of failed groups is sufficient, there will be three copies in normal redundancy and five copies in high redundancy.

Posted by ernielou on Wed, 02 Jan 2019 20:03:08 -0800