SSD TRIM in NetBSD HEAD (-current)

nia

Solid state drives are complicated. TRIM is an operation in the SATA command set that can be used to mark specific blocks of a SSD unused, which helps the SSD retain good performance when large amounts of data have been deleted. It requires operating system support. macOS actually didn't support TRIM for quite a long time, so there's at least some drives available in the wild that cope well with not being regularly TRIMed.

NetBSD 7.0 added a fdiscard system call, which allows portions of a device or file to be TRIMed.

A discard mount option for the FFS file system has also been available for some time:

             discard     Use DISCARD/TRIM commands if disk and driver support
                         it.

                         EXPERIMENTAL - negatively influences filesystem
                         performance by increasing fragmentation, causes free
                         block map inconsistency on unclean shutdown, and is
                         incompatible with log.  If log option is also used,
                         discard is automatically disabled.

(per the mount(8) manual page)

The state of the art has changed over the years, and various SSD manufacturers now recommend occasional routine use of the TRIM command. So the drawbacks of mounting with discard probably aren't worthwhile.

In February 2022, mrg committed a blkdiscard(8) command to NetBSD that utilizes the fdiscard system call to a device or file.

According to this mailing list thread about blkdiscard, at least the wd disk driver (used for SATA-ish drives), dk (used for GPT partitions), and ld support fdiscard. cgd doesn't know about fdiscard yet, so all of this might not be effective on encrypted partitions, yet.

Before (re-)installing NetBSD it's now possible to drop to a shell and TRIM a complete drive to re-initialize it before installation:

# blkdiscard -v /dev/rwd0d

(This will wipe the drive!)

It's also possible to write a blank file, then use blkdiscard to mark sections of your drive unused:

# dd if=/dev/zero of=./testfile bs=4m count=10000
# blkdiscard -v ./testfile

This could be performed as a routine maintenance task, e.g. by cron(8).

Mosfet

Thnx for the info Nia. A couple of quick questions, regarding TRIM:

Does it work on mounted partitions? For example, on linux I can run: fstrim -a -v
so this way I do TRIM everything mounted (supporting TRIM).
Is blkdiscard compatible with journaling filesystems (aka log)?

nia

Mosfet

If you run blkdiscard on a full partition it will wipe it. You can the "create file then discard it" thing on any mounted file system.
Yes

Mosfet

nia
So, lets say that I have 3 SSD drives: 1 mounted on / (with 10000 MB free space), 1 mounted on /mnt/disk (with 5000 MB free space) and 1 mounted on /mnt/misc (with 2000 MB free space).

Based on your original post, it is my understanding that the proposed use case, is to run something like this:

dd if=/dev/zero of=/zerofile bs=1m count=10000 && blkdiscard -v /zerofile && rm /zerofile
dd if=/dev/zero of=/mnt/disk/zerofile bs=1m count=5000 && blkdiscard -v /mnt/disk/zerofile && rm /mnt/disk/zerofile
dd if=/dev/zero of=/mnt/misc/zerofile bs=1m count=2000 && blkdiscard -v /mnt/misc/zerofile && rm /mnt/misc/zerofile

Is this correct?

JuvenalUrbino

Hi nia, thanks for this. Your post on current-users appeared just at the same time as I was thinking of writing about ATA TRIM on NetBSD, in view of addressing the use of blkdiscard(8) on HEAD. As @mrg told on the mailing list, the implementation is still lacking an utility like fstrim, capable of looking for unallocated sectors and discarding them as part of periodical system maintenance. As I understand, fdiscard() is netbsd-only, so in order to use it as back-end for fstrim, some work should be done to provide kernel glue to the Linux fstrim code. I can see this happening in the near future, though.

For the time being, however, the trimming options currently provided by the NetBSD kernel aren't equivalent to what most users refer to when they think about periodical fstrim as a way to keep write performance on SSDs (see the reply above by @Mosfet ).

blkdiscard, can however prove very useful in discarding unused blocks and zeroing disks before a fresh installation, or on external storage. By the way, are we sure that wd(4) and ld(4) are supported but sd(4) is not?

nia

JuvenalUrbino I'm curious why you're dismissive of blkdiscard as a regular maintenance tool on a live system?

JuvenalUrbino

nia it can't be automated, and has to be used wisely. I've been discarding old .dump backups, since they're single files occupying large amounts of space, so the benefit of discarding the invalid data pages deriving from their deletion appears worthwhile; and the process can be easily automated too. On file servers, I can fancy more uses.
Thinking about desktop though, with all the small write/delete operations done every day, it seems a bit impractical to trim unallocated sectors by relying on blkdiscard exclusively in a efficient, safe and easy to reproduce way.

JuvenalUrbino

Mosfet
dd if=/dev/zero of=/zerofile bs=1m count=10000 && blkdiscard -v /zerofile && rm /zerofile

Wouldn't recurrent zero-filling accelerate wearing out and reduce the SSD lifespan?

I was thinking of using LVM instead for this purpose.

# create a logical volume taking all the free space left on vg0 
$ lvm lvcreate -l100%FREE -n free vg0
# discard invalid data pages on the 'free' LV
$ blkdiscard -v /dev/vg0/free
# delete the 'free' LV
$ lvm lvremove vg0/free

If the disk is already in use and and was not initialized as a LVM group in the first place, one may at least reserve the remaining space as a LVM group:

$ modload dm && service lvm onestart
# create a standard partition occupying all space left on device
$ gpt add -a 2m -t ffs -l "LVM” wd0
# initialize the partition as LVM (e.g., assuming it's dk5)
$ lvm pvcreate /dev/rdk5
# create a volume group on the partition
$ lvm vgcreate vg0 /dev/rdk5

Does this look reasonable? cc. @nia

EDIT: yes, apparently this is could be done and somebody wrote a script to automate the process.

Mosfet

JuvenalUrbino

Wouldn't recurrent zero-filling accelerate wearing out and reduce the SSD lifespan?

I am pretty sure that it would, so I would definitely not use that, in a daily cron job (I probably wouldn't use that on weekly basis either, though I would consider it for running on monthly basis, if there is no better alternative).

EDIT: I just remembered that transmission torrent client, has the option to do preallocation of the files to be downloaded (create for example a 2 GB file, without actually writing at once 2 GB of data). Knowing that transmission exists in pkgsrc, I thought to take a look at the code. Voila:

https://github.com/transmission/transmission/blob/690cf50e539c20c77daf9d7db390d6306a3da157/libtransmission/file-posix.cc

#ifdef HAVE_POSIX_FALLOCATE
bool full_preallocate_posix(tr_sys_file_t handle, uint64_t size)
{
    return posix_fallocate(handle, 0, size) == 0;
}
#endif

Since netbsd implements posix_fallocate:
https://man.netbsd.org/NetBSD-9.2/posix_fallocate.2

someone, just has to mimic transmission code... :P

JuvenalUrbino

Mosfet EDIT: I just remembered that transmission torrent client, has the option to do preallocation of the files to be downloaded (create for example a 2 GB file, without actually writing at once 2 GB of data). Knowing that transmission exists in pkgsrc, I thought to take a look at the code. Voila:

That's neat! I run a transmission-daemon server, and didn't know that.

Mosfet someone, just has to mimic transmission code... :P

Having this supported at filesystem-level as an optional tunable (mount option or what else) would be a nice feature, wouldn't it?

P.S. have a look at the forum formatting guide whenyou have time ;)

rvp

Mosfet Since netbsd implements posix_fallocate:

I think not:

$ fgrep -r fallocate /usr/src/sys/ufs/ /usr/src/sys/miscfs/ /usr/src/sys/fs/
/usr/src/sys/ufs/chfs/chfs_vnops.c:             { &vop_fallocate_desc, genfs_eopnotsupp },
/usr/src/sys/ufs/ext2fs/ext2fs_vnops.c: { &vop_fallocate_desc, genfs_eopnotsupp },      /* fallocate */
/usr/src/sys/ufs/ffs/ffs_vnops.c:       { &vop_fallocate_desc, genfs_eopnotsupp },      /* fallocate */
/usr/src/sys/ufs/lfs/lfs_vnops.c:       { &vop_fallocate_desc, genfs_eopnotsupp },      /* fallocate */
/usr/src/sys/ufs/mfs/mfs_vnops.c:       { &vop_fallocate_desc, genfs_eopnotsupp },      /* fallocate */
/usr/src/sys/fs/tmpfs/tmpfs_vnops.c:    { &vop_fallocate_desc,          genfs_eopnotsupp },
[...]
$

At least, not on UFS or tmpfs. Not sure about ZFS.

#include <err.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int
main(int argc, char* argv[])
{
	int fd, rc = EXIT_FAILURE;
	off_t len;
	char* fn;

	if (argc != 3)
		errx(rc, "Usage: %s filename len", getprogname());

	if ((len = atoi(argv[2])) <= 0)
		errx(rc, "%zd: bad length", len);

	fn = argv[1];
	if ((fd = open(fn, O_WRONLY | O_CREAT, 0600)) == -1)
		err(rc, "%s: open failed", fn);
	if (ftruncate(fd, len) == -1)
		err(rc, "%s: ftruncate failed", fn);
	if ((rc = posix_fallocate(fd, 0, len)) != 0)
		errc(1, rc, "%s: posix_fallocate failed", fn);
	if (write(fd, &rc, 1) != 1)
		err(rc, "%s: write failed", fn);
	if (fsync(fd) == -1)
		err(rc, "%s: fsync failed", fn);
	if (close(fd) == -1)
		err(rc, "%s: close failed", fn);
	return 0;
}

$ cc fallocate.c
$ ./a.out file 10000
a.out: file: posix_fallocate failed: Operation not supported
$

nia

On posix_fallocate... any idea how long I spent trying to figure out what was wrong with a certain piece of software Not Working before realizing I had to teach it to use ftruncate() instead? ;)

dpostolov

Hi! Sorry for my bad English...

In NetBSD 10-BETA blkdiscard -v /dev/rwd0d successfully works for SATA SSD, but does blkdiscard works for nvme m.2 SSD disks? The command blkdiscard -v rld0d print that driver or device is not supported. Or I am set wrong device rld0d?

The workaround: use FreeBSD USB stick and trim -f nvd0 for nvme disk.

rvp

dpostolov but does blkdiscard works for nvme m.2 SSD disks?

Sorry, doesn't look like the any of the Data Set Management (DSM) commands (to implement TRIM) are implemented in the nvme driver.