Use of disklabel, MBR and GPT

rockyhotas

Hello!
After partitioning a new disk with MBR, creating two primary partitions, I was trying to define a disklabel in each of them. Quite soon I realized it was not possible. After an IRC conversation, which was very instructive yesterday, here are some considerations.

An important premise is that disklabel, MBR and GPT are data structures intended to describe a whole disk, not a single partition: internally, they then divide the disk into partitions. These single partitions can not host themselves a disklabel, an MBR or a GPT to further subdivide them, because they are not an entire disk.

Maybe this is trivial, but during the installation of NetBSD a disklabel is actually nested inside a primary MBR partition. So, I would like to share some further clarification.

Assume that you have this example MBR:

1. NTFS, Windows system partition
2. NTFS, Windows data
3. (empty)
4. (empty)

(for simplicity, I will not consider the case of extended partitions, but it would be similar).

Assume that, when installing NetBSD with sysinst(8), partition 3. is chosen for NetBSD. sysinst(8) does not change MBR in this case (thus preserving primary partitions 1. and 2.).

There are several other options:

sysinst(8) may create a brand new MBR (warning: this will destroy all the existing data on the disk), with primary partitions defined by the user, who can then choose which of them will be used for NetBSD;
through sysinst(8) the user may also modify an existing MBR, and in this case it's his responsibility to preserve the partitions dedicated to other OSs and to manage only the free space in the disk for NetBSD;
sysinst(8) also offers the chance to Use the entire disk: in this case, it creates anyway a MBR, with a primary partition which extends to the whole disk, used for NetBSD.

As you may note, the use of MBR is never abandoned. This is done for two reasons:

to grant the compatibility with PC/DOS (and other OSs like Windows this way can still use the same disk);
to grant a regular boot on PC/IBM hardware (where, without a valid MBR, the BIOS could not even try to search a bootloader on the disk).

Note that the above behaviour is about amd64; for other ports it may change.

Whatever choices the user makes, in the end there will be an MBR primary partition dedicated exclusively to NetBSD. Inside it, sysinst(8) installs disklabel. disklabel(8) allows to further divide this NetBSD-dedicated MBR partition, originating disklabel partitions a (which is generally used for the NetBSD root directory /), b (generally used for swap), e (it may be /home, for example), f (for example, /usr), g (for example, /var), etc.; in amd64, partitions c and d are reserved, to represent respectively the NetBSD MBR primary partition and the whole disk, as shown here.

It may seem that this disklabel, nested into an MBR primary partition, is used to further divide internally this partition: so, for another MBR primary partition on the same disk, there may be another disklabel which can be used the same way. This is wrong.

Disklabel is an alternative, and not an integration, to MBR.

In the case of an amd64 NetBSD installation inside a PC/DOS disk, the disklabel is placed inside the MBR primary partition dedicated to NetBSD, in the second block (and the first block is used for the NetBSD bootloader). This is done to make sure that the disklabel data structure is preserved and not altered or deleted by the action of other OSs: in fact, the MBR primary partition dedicated to NetBSD is not used by DOS or similar OSs, but it is recognized by them. Without this issue, disklabel (whose location is flexible) could be placed in the second sector of the disk (while MBR is in the first), not the second sector of a specific partition.

When the machine is booted with NetBSD, disklabel is the only way NetBSD looks at the disk, the entire disk; MBR in this case is not considered at all. (Obviously, during the installation, a reference to the NetBSD bootloader is placed in the boot sector of MBR, so that NetBSD can be booted). Disklabel provides exactly all the information MBR provides, but in a way NetBSD can understand. It provides the full size of the disk (disklabel partition d for amd64), the size and location of the usable portion of the disk (disklabel partition c, for example, which represents in amd64 the NetBSD primary partition), and the structure of this portion of the disk (disklabel custom partitions a, b, e, f, g, etc.). Thinking about MBR, it actually provides the same information, but in a way the other OSs can understand: the full size of the disk, the size and location of the usable portion(s) of the disk and their structure (all the primary partitions whose type is DOS-compatible).

Disklabel and MBR are then just two alternative ways to represent the same reality, the whole disk. The former is used by NetBSD and the latter is used by DOS-compatible OSs like Windows, for the same disk. The location of disklabel (nested inside an MBR primary partition) is just a consequence of possible compatibility issues with other OSs. But both MBR and disklabel are meant to be unique inside a disk (and in systems where PC/IBM compatibility issues are absent, they should not even coexist: disklabel only is used; on the other hand, in a Windows PC, only MBR is used). A disk can not host more than one MBR, or more than one disklabel. My initial intention of creating multiple disklabels, one for each MBR primary partition, was wrong.

Here, Figure 2.1 well describes the way disklabel (and so, NetBSD) looks at the whole disk. Refer to the blue labels, on the right of the image. The same Figure also well describes the way MBR (and so, a DOS-compatible OS) looks at the whole disk: refer to the red labels, in the center of the image.

The fact that NetBSD only uses disklabel for his awareness of the disk is also proved by the /dev directory: for each disk, for example wd0, only the disklabel partitions are available: wd0a, wd0b, etc. (along with the corresponding raw partitions rwd0a, rwd0b, etc.). There is no way to reference the other MBR primary partitions (so, installing a disklabel on them would be impossible, other than uncorrect), because from the NetBSD (and disklabel) perspective, they simply do not exist: it only exists a portion of the disk which is not used by any disklabel partition (and where actually are placed the eventual other MBR primary partitions).

All the above considerations are assuming that a single NetBSD OS is installed in a disk. They may change if two or multiple NetBSD systems are installed in the same disk, but this is a possible conflicting and confusing scenario, which I would discourage.

This is different on FreeBSD, where the MBR primary partitions are accessible for each disk: for example, /dev/ada0s1 refers to the first MBR primary partition of disk ada0 (s means slice, another way MBR primary partitions are called). In FreeBSD, /dev also has ada0s1a, ada0s1b, etc., if ada0s1 has a UFS filesystem). I don't know if FreeBSD also manages in a different way the coexistence of MBR and disklabel.

GPT is used for the same purpose as MBR and disklabel: it describes the structure of an entire disk. There's however a difference in the way NetBSD integrates with it: while MBR is an alien system for NetBSD, GPT is supported. So, for example, if a new disk is added to an existing NetBSD system, and this disk is partitioned with GPT, there's no need of a disklabel. For each new partition, GPT in NetBSD automatically creates a wedge (dk0, dk1, etc.). Wedges are the equivalent of a partition, not a disk, so a disklabel can not be placed on them (also, /dev has no dk0a, dk0b, etc., but only dk0, dk1, etc.). A new filesystem (FFSv2, for example) can be directly created on a wedge. I don't know if a disklabel is installed when GPT is used to partition a NetBSD system disk (the one hosting the root / directory), but I don't think so.

GPT is available to partition a system disk when performing a UEFI installation, but IIRC a traditional BIOS installation does not offer GPT.

As well, if a new disk is added to an existing NetBSD system (so, it is not a boot disk), it may be partitioned using only disklabel, not MBR or GPT. Avoiding in amd64 the reserved partitions c and d, the other letters may be arbitrary related to any data contents (with any mountpoint).

These observations would not have been possible without the help of the users in the #netbsd channel on Freenode IRC. I would like to thank them so much. If any of them wants to integrate these notes and/or correct something, is welcome.

Thank you again!

Rocky

The original message is from the @netbsd-users mailing list: I thought it could also be useful here.

rvp

rockyhotas A lot of this complexity can be understood better if you know a bit of the history of the various pieces here:

MBRs (MS-DOS world) were developed independently of disklabels (Unix world). Both serve the same function: logically splitting an HDD into smaller pieces. Then, the Unix workstation market collapsed and disklabels had to be fitted into the now-dominant PC-hardware's MBR scheme. Then a new standard appeared to replace MBR: GPT.

But, history aside, what interests me here is that you feel that certain partitioning schemes are not possible/allowed. Can you state exactly which? Because it seems to me that as long as you have:

a) a supported partitioning scheme (MBR, GPT, disklabel)
b) a supported filesystem (FFS, ext2, MSDOS, VFAT, ...) and
c) can get absolute sector/block addresses

then NetBSD should have very few limitations on reading any partition/slice whether MBR, GPT or disklabel.

rvp

rockyhotas So, maybe any partition scheme you use is good, now, even disklabel alone.

Correct.

rockyhotas A disk can not host more than one MBR, or more than one disklabel. My initial intention of creating multiple disklabels, one for each MBR primary partition, was wrong.

The first part about the MBR is true, but, I'm not sure that I understand the second part. Can you clarify?

rockyhotas

Obviously, disklabel creates custom partitions only in the portion of the disk which is related to NetBSD, the only one where it is intended to act. It leaves unchanged any other area of the disk. If MBR did not exist, the extension of disklabel partitions a, b, e, f, etc. could span the whole disk, and in fact this is the case of my last example, where disklabel only is used, without MBR.

In a similar way, when MBR is used from other OSs, it is intended to create and modify all the primary partitions, except the one where NetBSD is installed (unless a full disk wipe must be performed): in other words, it is indended to modify the disk, except the portion used by NetBSD. This way, NetBSD can coexist with a PC/IBM, DOS-compatible system.

Please, also check out Martin's reply and the whole thread in the Mailing List.

flfederation

The fact that NetBSD only uses disklabel for his awareness of the disk is also proved by the /dev directory: for each disk, for example wd0, only the disklabel partitions are available: wd0a, wd0b, etc. (along with the corresponding raw partitions rwd0a, rwd0b, etc.). There is no way to reference the other MBR primary partitions (so, installing a disklabel on them would be impossible, other than uncorrect), because from the NetBSD (and disklabel) perspective, they simply do not exist

I was of the impression that the whole disk as suggested by the diagram https://www.netbsd.org/docs/guide/en/chap-inst.html#chap-inst-installdoc was addressable by wd0c. on the amd64 port, it is typically addressable by wd0d. Coming to BSD from GNU/Linux, I expected /dev/wd0 to exist, but once I realised that would be wd0c (I am not using the amd64 port) it helped. For example, I've written an image to USB using dd now; the "c" (whole disk) "partition" ("d" on amd64) makes that possible.

rvp

flfederation I was of the impression that the whole disk as suggested by the diagram https://www.netbsd.org/docs/guide/en/chap-inst.html#chap-inst-installdoc was addressable by wd0c. on the amd64 port, it is typically addressable by wd0d.

This is correct. rwd0d is the whole ATA disk (first to last sector) on amd64; and rsd0d is the whole SCSI (incl. USB-sticks) disk.

rockyhotas

rvp Yes, the history is very useful to understand it. I took history for granted in my post.

But, history aside, what interests me here is that you feel that certain partitioning schemes are not possible/allowed.
Can you state exactly which?

Uhm, this was not my intention while writing. My post is just an attempt to clarify how it is possible that MBR coexists with disklabel, with disklabel nested in an MBR partition.

I thought that sysinst(8) would not allow you to use disklabel only (without MBR) in the NetBSD root partition on amd64, because, if a PC/IBM compatible BIOS does not recognize an MBR, it does not generate a boot. However, later in the Mailing List thread Martin said:

newer sysinst versions allow to kill the whole disk content (including the MBR and whatever partitions exist), and start from scratch (as if the disk had been totaly wiped before).

So, maybe any partition scheme you use is good, now, even disklabel alone.

rockyhotas

rvp

The first part about the MBR is true, but, I'm not sure that I understand the second part. Can you clarify?

It's in the beginning of my message:

After partitioning a new disk with MBR, creating two primary partitions, I was trying to define a disklabel in each of them. Quite soon I realized it was not possible.

I tried this because

It may seem that this disklabel, nested into an MBR primary partition, is used to further divide internally this partition: so, for another MBR primary partition on the same disk, there may be another disklabel which can be used the same way. This is wrong.

I didn't know that disklabel is used to describe the whole disk, hence my error.

rvp

rockyhotas It's in the beginning of my message:

rockyhotas After partitioning a new disk with MBR, creating two primary partitions, I was trying to define a disklabel in each of them. Quite soon I realized it was not possible.

But, that's a pretty standard use-case, right?:

1. VFAT (C:)
2. Linux (/)
3. NetBSD (disklabel)
4. OpenBSD (disklabel)

Now, you have 2 disklabels on the disk.
And, if the above is possible (and it is), then this should be possible also (also a common use-case):

1. Linux (/)
2. NetBSD (RELEASE: disklabel)
3. NetBSD (STABLE: disklabel)
4. NetBSD (CURRENT: disklabel)

Notice that to install more NetBSD partitions, I have chosen to remove the VFAT partition.

Now we have 3 disklabels. The logic is straight-forward: if one disklabel can be created inside an MBR partition (and no doubt you can do this), then it should also be possible to create disklabels inside other MBR partitions.

Edit: to be honest, I haven't tried to do this myself, but, it seems like it should be possible. We'll see.

rockyhotas

rvp If you follow my reasoning in the first post, I am dealing with a single NetBSD installation. I wanted to create a second disklabel for a data partition, but not a second installation of NetBSD. As I wrote:

All the above considerations are assuming that a single NetBSD OS is installed in a disk. They may change if two or multiple NetBSD systems are installed in the same disk, but this is a possible conflicting and confusing scenario, which I would discourage.

In your examples, there are n different disklabels, but also n different instances of a BSD OS.
You can do this because each OS thinks it is alone and uses a disklabel to describe the whole disk. As soon as this disklabel does not modify the other MBR primary partitions, it all works fine; and it is maybe the only way to have these multiple OSs work in the same disk. What you create is an "illusion" for each of your OSs: they don't know about the other OSs in the same hard disk.

The logic is straight-forward: if one disklabel can be created inside an MBR partition (and no doubt you can do this), then it should also be possible to create disklabels inside other MBR partitions.

Yes, it's logical, but the disklabel (as I wrote in the first post) is placed inside an MBR primary partition just because it's the safest place to put it. It could have alternatively be placed in the second sector of the disk, and in that case you couldn't have installed multiple BSD OSs with multiple disklabels.

Edit: to be honest, I haven't tried to do this myself, but, it seems like it should be possible. We'll see.

It surely works, but only because each OS thinks there is no other disklabel in this disk. What instead you can not do is to install a single NetBSD instance with its disklabel, and create another disklabel in another MBR primary partition from the NetBSD instance you have just installed.

rockyhotas

rvp I think we just have different goals.

Your goal is (IIUC) to have multiple OSs in the same disk. It is possible and yes, multiple disklabels are created, one per OS.

My goal was to have a single OS with its disklabel in a disk and to write, from this OS, another disklabel in another MBR primary partition, to manage it (subdivide its space, etc.). This is not possible. If, from my NetBSD installation, I want to access the other MBR primary partition (which I assume it's empty and only dedicated to store data from NetBSD), I must know its offset and its extension, and I must use the single disklabel installed with NetBSD to manage it.

rvp

rockyhotas What you create is an "illusion" for each of your OSs: they don't know about the other OSs in the same hard disk.

Of course! It's an OS, not a partition-recovery program, so it's not going to scan disk-blocks looking for disklabels. It's just going to use what it is pointed at.

rockyhotas My goal was to have a single OS with its disklabel in a disk and to write, from this OS, another disklabel in another MBR primary partition, to manage it (subdivide its space, etc.). This is not possible

Now things are getting more interesting.

Writing is possible. In fact, there is more than one way of creating/writing a new disklabel from an existing NetBSD partition. Since you seem to have a very good understanding of the disk-related fundamentals under NetBSD, I'll just refer you to the dkctl(8) man-page.

Once the disklabel inside an MBR partition has been created, using it is a also possible, but, it is not straight-forward because slices are not disks after all. So, you make disks out of slices by (a)busing ccdconfig(8). Then the kernel will happily create "disk devices" for you which you can then further disklabel, newfs, mount, etc.

All this is a bit obscure, and only useful as a learning exercise. Have fun.

rockyhotas

rvp

Writing is possible. In fact, there is more than one way of creating/writing a new disklabel from an existing NetBSD partition. Since you seem to have a very good understanding of the disk-related fundamentals under NetBSD, I'll just refer you to the dkctl(8) man-page.

Thanks, but actually I used dkctl(8) only to create wedges and don't know it well. If you have an example of how to use it for disklabels, it would be great. I thought a disklabel inside an MBR primary partition could be created only by sysinst(8) during NetBSD installation, not inside an MBR primary partition of a new disk, added to an existing system.

Once the disklabel inside an MBR partition has been created, using it is a also possible, but, it is not straight-forward because slices are not disks after all. So, you make disks out of slices by (a)busing ccdconfig(8).

This is amazing! I didn't know it. So, you can create a ccd<number> device, for example, joining n partitions, each from a different disk? But if I understood, these can be just one MBR primary partition per disk, because you can not access from NetBSD's /dev files the single MBR primary partitions of a disk (while you can, instead, in FreeBSD).

In fact, in the example, there are ccdconfig ccd0 32 0 /dev/sd2e /dev/sd3e /dev/sd4e /dev/sd5e and sd2e, sd3e, etc. are different disks IIUC.

rvp

rockyhotas If you have an example of how to use it for disklabels, it would be great.
...
But if I understood, these can be just one MBR primary partition per disk, because you can not access from NetBSD's...

A bit of sideways thinking is required for this. Re-read my previous post carefully and then read the CCD config chapter and the example commands in the Guide. Then, the "Aha!" moment should arrive, and you'll be on your way to figuring the whole thing out on your own. Gambatte!