Disk management from Installation ISO

netbsdnoob

Hello everyone,

I was wondering what tools can you use to comfortably check the installed disks, how many GB they have and how they are partitioned from the tools that are available in the installation disk without using sysinst.

I tried fdisk but seems a bit hard to use.
So Ideally i would like to:
1) list the disks: i know i can do this with hw.disknames, are there other ways? like lsblk on linux?
2) check the size for each of the disks?
3) Check which partitions are on a disk and how much space they take?
4) Format disks/partitions

rvp

netbsdnoob 1) list the disks: i know i can do this with hw.disknames

That's the correct way to get all names. You can also use disklabel wd0 to list slices on a NetBSD MBR partition; or for GPT-partitioned disks, list wedges using dkctl wd0 listwedges.

netbsdnoob 2) check the size for each of the disks?

gpt -r header wd0                   # or, sd1, ld2, etc...

netbsdnoob 3) Check which partitions are on a disk and how much space they take?

gpt -r show wd0

netbsdnoob 4) Format disks/partitions

After you add GPT partitions using gpt:

dkctl wd0 listwedges
newfs -O2 /dev/rdk10              # use correct wedge name
newfs -O2 NAME=GPT_label          # OR, you can also use a GPT label instead

JuvenalUrbino

Unfortunately there's nothing like lsblk on NetBSD. @vermaden wrote a script for FreeBSD mimicking lsblk functionality; I've been meaning to port it to NetBSD, but the differences between FreeBSD and NetBSD in disk management and userland tools are considerable, so that would turn out more like a complete rewrite.

See this thread on DaemonForums where I wrote few things about NetBSD disk and partition management.
More in-depth content is available in the wiki at 2.2.2 and 3.5-6 for MBR partitions, and inside UEFI installation guide and gpt(8) for GPT partitions.

On MBR, NetBSD will by default encapsulate a BSD disklabel within one of the 4 allowed primary partitions [1]. The BSD label will contain a NetBSD slice, a sort of logical volume containing the "real" partitions. These partitions cannot be "seen" by other operating systems which do not understand the BSD disklabel (the slice will be reported as a single unknown partition outside of BSD). On BSD however, the slice can be inspected and partitions can be created, altered or deleted using the disklabel(8) utility. MBR primary partitions themselves instead, similarly to many other operating systems, are administered through fdisk(8).

However, since you probably performed a UEFI/GPT installation on your T450, things are going to be easier as you won't be dealing with disklabel(8).
NetBSD has a very nice, intuitive and powerful utility to handle GPT disk layout and partitions, which is the aforementioned gpt command. Combined with hw.disknames and dkctl, gpt can return a lot of useful information about storage devices and what they contain.

[1] The BSD disklabel may also be directly written onto the MBR, giving the disk a "BSD" partition layout. This makes it incompatible with any possibly coexisting non-BSD system.

A couple of examples:

1. workstation (GPT/UEFI)

Note: use nvmectl(8) instead of atactl(8) for NVMe drives.

~ $ atactl wd1 identify | egrep 'Model|Device type|Capacity'
Model: ST1000DM010-2EP102, Rev: CC43, Serial #:             Z9A8LA8A
Device type: ATA, fixed
Capacity 1000 Gbytes, 1953525168 sectors, 512 bytes/sector
~ $ gpt show wd1
       start        size  index  contents
           0           1         PMBR
           1           1         Pri GPT header
           2          32         Pri GPT table
          34        2014         Unused
        2048  1953521664      1  GPT part - Windows basic data
  1953523712        1423         Unused
  1953525135          32         Sec GPT table
  1953525167           1         Sec GPT header
~ $ dkctl wd1 listwedges
/dev/rwd1: 1 wedge:
dk4: 00006d79-79c0-3cf0-b28a-d501f2da0000, 1953521664 blocks at 2048, type: ntfs

~ $ atactl wd2 identify | egrep 'Model|Device type|Capacity'
Model: ST500DM002-1BD142, Rev: KC65, Serial #:             S2AQ3W02
Device type: ATA, fixed
Capacity 500 Gbytes, 976773168 sectors, 512 bytes/sector
~ $ gpt show wd2
      start       size  index  contents
          0          1         PMBR
          1          1         Pri GPT header
          2         32         Pri GPT table
         34       2014         Unused
       2048     524288      1  GPT part - EFI System
     526336   33554432      2  GPT part - NetBSD FFSv1/FFSv2
   34080768   33554432      3  GPT part - NetBSD swap
   67635200  201326592      4  GPT part - NetBSD FFSv1/FFSv2
  268961792   25165824      5  GPT part - NetBSD FFSv1/FFSv2
  294127616  671088640      6  GPT part - NetBSD FFSv1/FFSv2
  965216256   11556879         Unused
  976773135         32         Sec GPT table
  976773167          1         Sec GPT header
~ $ dkctl wd2 listwedges
/dev/rwd2: 6 wedges:
dk5: netbsd-boot, 524288 blocks at 2048, type: msdos
dk6: netbsd-root, 33554432 blocks at 526336, type: ffs
dk7: netbsd-swap, 33554432 blocks at 34080768, type: swap
dk8: netbsd-usr, 201326592 blocks at 67635200, type: ffs
dk9: netbsd-var, 25165824 blocks at 268961792, type: ffs
dk10: netbsd-home, 671088640 blocks at 294127616, type: ffs

~ $ atactl wd3 identify | egrep 'Model|Device type|Capacity'
Model: KINGSTON SM2280S3G2120G, Rev: SAFM01.V, Serial #: 50026B7271019588
Device type: ATA, fixed
Capacity 120 Gbytes, 234441648 sectors, 512 bytes/sector
~ $ gpt show wd3
      start       size  index  contents
          0          1         PMBR
          1          1         Pri GPT header
          2         32         Pri GPT table
         34       2014         Unused
       2048    1228800      1  GPT part - EFI System
    1230848  216426496      2  GPT part - Linux data
  217657344   16784271      3  GPT part - Linux swap
  234441615         32         Sec GPT table
  234441647          1         Sec GPT header
~ $ dkctl wd3 listwedges
/dev/rwd3: 3 wedges:
dk11: EFI System Partition, 1228800 blocks at 2048, type: msdos
dk12: 8bccca62-644a-4848-adcb-a9f44bb75169, 216426496 blocks at 1230848, type: ext2fs
dk13: bace07a3-d4b2-5742-95a8-8206800f4fb8, 16784271 blocks at 217657344, type: swap

2. server (GPT/BIOS)

The server main NetBSD partition occupies internal GPT disk (wd0) almost entirely (excluding a small unencrypted FFS boot partition) and is configured as a CGD-ecrypted logical disk. A BSD label is encapsulated inside the CGD disk, with a single slice containing multiple logical partitions.
An external disk is connected via USB (sd0). This is a GPT disk with a single ZFS partition containing a single zpool, in which multiple ZFS datasets have been created.

~ $ atactl wd0 identify | egrep 'Model|Device type|Capacity'
Model: ST750LM022 HN-M750MBB, Rev: 2BA30001, Serial #: S330J9DF726740
Device type: ATA, fixed
Capacity 750 Gbytes, 1465149168 sectors, 512 bytes/sector
~ $ gpt show wd0
       start        size  index  contents
           0           1         PMBR (active)
           1           1         Pri GPT header
           2          32         Pri GPT table
          34        4062         Unused
        4096      262144      1  GPT part - NetBSD FFSv1/FFSv2
      266240  1464881152      2  GPT part - NetBSD Cryptographic Disk
  1465147392        1743         Unused
  1465149135          32         Sec GPT table
  1465149167           1         Sec GPT header
~ $ dkctl wd0 listwedges
/dev/rwd0: 2 wedges:
dk1: cgd.conf, 262144 blocks at 4096, type: ffs
dk2: syscgd, 1464881152 blocks at 266240, type: cgd
~ $ disklabel -iI cgd0      # display BSD partitions inside the encrypted BSD slice
Enter '?' for help
partition>P
7 partitions:
#        size    offset     fstype [fsize bsize cpg/sgs]
 a:  20971520         0     4.2BSD      0     0     0  # (Cyl.      0 -  10239)
 b:  16777216  20971520       swap                     # (Cyl.  10240 -  18431)
 d: 1464881152         0     unused      0     0        # (Cyl.      0 - 715273)
 e:  62914560  37748736     4.2BSD      0     0     0  # (Cyl.  18432 -  49151)
 f:  16777216 100663296     4.2BSD      0     0     0  # (Cyl.  49152 -  57343)
 g: 1347440640 117440512     4.2BSD      0     0     0  # (Cyl.  57344 - 715273

~ $ atactl sd0 identify | egrep 'Model|Device type|Capacity'
Model: WDC WD20SDRW-11VUUS0, Rev: 01.01A01, Serial #:      WD-WXH2A11NVY67
Device type: ATA, fixed
Capacity 2000 Gbytes, 3906963632 sectors, 512 bytes/sector
~ $ gpt show sd0                                                 

       start        size  index  contents
           0           1         PMBR
           1           1         Pri GPT header
           2          32         Pri GPT table
          34        4062         Unused
        4096  3906957312      1  GPT part - ZFS
  3906961408        2015         Unused
  3906963423          32         Sec GPT table
  3906963455           1         Sec GPT header
~ $ dkctl sd0 listwedges                                         

/dev/rsd0: 1 wedge:
dk0: zfs-data, 3906957312 blocks at 4096, type: zfs
~ $ zpool list
NAME   SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
zfs   1.81T   153G  1.66T         -     1%     8%  1.00x  ONLINE  -
~ $ zfs list
NAME        USED  AVAIL  REFER  MOUNTPOINT
zfs         153G  1.61T    25K  /zfs
zfs/data   6.96G   493G  6.96G  /zfs/data
zfs/music  22.9G  77.1G  22.9G  /zfs/music
zfs/p2p     108G  1.61T   108G  /zfs/p2p
zfs/pkg    4.43G  95.6G  4.43G  /zfs/pkg
zfs/snap   10.6G   489G  10.6G  /zfs/snap

Concerning mounted partitions, df(1) and mount(8) can be quite informative:

1. workstation

~ $ mount                                                                                       

/dev/dk6 on / type ffs (log, nodevmtime, local)
/dev/dk9 on /var type ffs (log, noatime, local)
/dev/dk8 on /usr type ffs (log, noatime, local)
/dev/dk5 on /boot type msdos (nodev, nosuid, read-only, local)
/dev/dk10 on /home type ffs (log, noatime, local)
tmpfs on /tmp type tmpfs (local)
kernfs on /kern type kernfs (local)
ptyfs on /dev/pts type ptyfs (local)
procfs on /proc type procfs (local)
tmpfs on /var/shm type tmpfs (local)
tmpfs on /usr/pkg/emul/linux/dev/shm type tmpfs (local)
pid2778@rigel:/net on /net type nfs (hidden)
pid2778@rigel:/media on /media type nfs (hidden)
~ $ df -hm
Filesystem   1M-blocks      Used     Avail %Cap Mounted on
/dev/dk6         15883      1144     13945   7% /
/dev/dk9         11913      2777      8539  24% /var
/dev/dk8         95303     11607     78930  12% /usr
/dev/dk5           255         0       255   0% /boot
/dev/dk10       322610      2362    304117   0% /home
tmpfs             5721       104      5617   1% /tmp
kernfs               0         0         0 100% /kern
ptyfs                0         0         0 100% /dev/pts
procfs               0         0         0 100% /proc
tmpfs             5721        11      5710   0% /var/shm
tmpfs             4087         0      4087   0% /usr/pkg/emul/linux/dev/shm

2. server

~ $ mount                                                                                                                                

/dev/cgd0a on / type ffs (log, noatime, local)
/dev/cgd0f on /var type ffs (log, noatime, local)
/dev/cgd0e on /usr type ffs (log, noatime, local)
/dev/dk1 on /etc/cgd type ffs (local)
/dev/cgd0g on /home type ffs (log, noatime, local)
kernfs on /kern type kernfs (local)
ptyfs on /dev/pts type ptyfs (local)
procfs on /proc type procfs (local)
tmpfs on /var/shm type tmpfs (local)
tmpfs on /tmp type tmpfs (local)
tmpfs on /usr/pkg/emul/linux32/dev/shm type tmpfs (local)
procfs on /emul/linux/proc type procfs (read-only, local)
zfs on /zfs type zfs (local)
zfs/data on /zfs/data type zfs (NFS exported, local)
zfs/music on /zfs/music type zfs (NFS exported, local)
zfs/p2p on /zfs/p2p type zfs (NFS exported, local)
zfs/pkg on /zfs/pkg type zfs (NFS exported, local)
zfs/snap on /zfs/snap type zfs (local)
~ $ df -hm
Filesystem    1M-blocks       Used      Avail %Cap Mounted on
/dev/cgd0a         9927        722       8708   7% /
/dev/cgd0f         7941       2931       4612  38% /var
/dev/cgd0e        29783      11917      16376  42% /usr
/dev/dk1            120         56         58  49% /etc/cgd
/dev/cgd0g       647729      57870     557472   9% /home
kernfs                0          0          0 100% /kern
ptyfs                 0          0          0 100% /dev/pts
procfs                0          0          0 100% /proc
tmpfs              2022          0       2022   0% /var/shm
tmpfs              2022          0       2022   0% /tmp
tmpfs             13266          0      13266   0% /usr/pkg/emul/linux32/dev/shm
procfs                0          0          0 100% /emul/linux/proc
zfs             1684927          0    1684927   0% /zfs
zfs/data         512000       7122     504877   1% /zfs/data
zfs/music        102400      23472      78927  22% /zfs/music
zfs/p2p         1795074     110147    1684927   6% /zfs/p2p
zfs/pkg          102400       4532      97867   4% /zfs/pkg
zfs/snap         512000      10846     501153   2% /zfs/snap

netbsdnoob

rvp what is the naming convention here?

I see on my system wd0 and dk1 dk2. How do i interpret these?

Also is there a more human-friendly way of showing size with gpt -r show wd0 when it comes to the size?
I mean showing e.g., 8GB instead of 16156623 or something like that.?

netbsdnoob

rvp @JuvenalUrbino ok now let's say i booted from a live image,

and want to prepare a disk for installing netbsd without sysinst, what are the minimum number of partitions i need if i am using gpt, it would be interesting to know with and without UEFI.
So gpt + bios and gpt + UEFI

rvp

netbsdnoob I see on my system wd0 and dk1 dk2. How do i interpret these?

wd0 is the whole (S)ATA disk. SCSI disks would be sd0. NVME disks ld0 (I think).
dk1 and dk2 are wedges the system creates from the partitions on a GPT-partitioned disk. The installation section in the Guide has more details.

netbsdnoob Also is there a more human-friendly way of showing size with gpt -r show wd0 when it comes to the size?
I mean showing e.g., 8GB instead of 16156623 or something like that.?

Well, there's a reason for that sector display: it's very precise--you can pretty easily see, to take an example, that there are no overlaps in the partitions.

Should be pretty easy to write a "humanizing" front-end like this

JuvenalUrbino

netbsdnoob I see on my system wd0 and dk1 dk2. How do i interpret these?

wd* is for ATA drives
sd* is for SCSI and USB drives
ld* is for logical disks (this includes SDs and PCIe drives)
md* is for memory disks
dk* is a form of hardware abstraction on NetBSD, grouping all the above under a disk-like interface, the so called wedge. Partitions inside physical disks are configured as dk wedges by the kernel as soon as the disk is attached. The dk(4) driver is conceptually similar to FreeBSD's GEOM.

JuvenalUrbino

rvp the system creates from the partitions on a GPT-partitioned disk.

It can work also on other partition layout forms (MBR, BSD label, Amiga RDB). But only GPT and APM are configured in the GENERIC kernel:

~ $ config -x /netbsd | grep DKWEDGE_METHOD
options         DKWEDGE_METHOD_GPT      # Supports GPT partitions as wedges
#options        DKWEDGE_METHOD_BSDLABEL # Support disklabel entries as wedges
#options        DKWEDGE_METHOD_MBR      # Support MBR partitions as wedges
options         DKWEDGE_METHOD_APPLE    # Support Apple partitions as wedges
#options        DKWEDGE_METHOD_RDB      # Support RDB partitions as wedges

rvp

JuvenalUrbino It can work also on other partition layout forms (MBR, BSD label, Amiga RDB). But only GPT and APM are configured in the GENERIC kernel:

Yah, and I've never really understood why DKWEDGE_METHOD_APPLE is enabled in an x86 GENERIC but DKWEDGE_METHOD_MBR isn't.

rvp

netbsdnoob o gpt + bios and gpt + UEFI

You mean BIOS + MBR and UEFI + GPT? (any other combination is pretty uncommon)

For UEFI + GPT, on a new disk, you need at least 3 partitions:

An EFI System Partition (MS-DOS/VFAT/...) for the EFI bootloader files
NetBSD EFI partition for /
NetBSD swap partn.

For BIOS + MBR, you need 1 primary NetBSD-type partition. You then divide this NetBSD partition using disklabel to create a (for /) and b (for swap) "slices".

All this should be there in the Guide.

JuvenalUrbino

rvp any other combination is pretty uncommon

I'd argue that GPT on BIOS on NetBSD is rather common; works fine, is integrated in sysinst, and has been supported for a very long time.

     gpt biosboot [-A] [-c bootcode] [-b startsec] [-i index] [-L label]
             The biosboot command allows the user to configure the partition
             that contains the primary bootstrap program, used during boot(8).

         The -A options sets the PMBR partition active.  This should not
         normally be necessary, but some firmware might require it.  If -A
         is omitted, the active flag will be cleared from the PMBR label.

         The -c option allows the user to specify the filename from which
         gpt should read the bootcode.  The default is to read from
         /usr/mdec/gptmbr.bin.

         The partition that should contain the primary bootstrap code,
         (similar to that installed via installboot(8)) is selected using
         the -i, -L and -b options.  One of these three options is
         required.  The -i option selects the partition given by the
         index.  The -L option selects the partition by label.  If there
         are multiple partitions with the same label, the first one found
         will be used.  The -b option selects the partition starting at
         block startsec.

rvp

JuvenalUrbino I'd argue that GPT on BIOS on NetBSD is rather common; works fine, is integrated in sysinst, and has been supported for a very long time.

There is also a mini-example at the end in that gpt(8) manpage for BIOS + GPT, but I've never tried it out. Nor have I tried persuading sysinst to do it for me (it's a temperamental beast at the best of times, and don't want it trashing my partitions).

JuvenalUrbino

rvp There is also a mini-example at the end in that gpt(8) manpage for BIOS + GPT, but I've never tried it out. Nor have I tried persuading sysinst to do it for me (it's a temperamental beast at the best of times, and don't want it trashing my partitions).

I've done it since NetBSD-7 days or so, and for what it's worth, in my case it just worked 😉.