Various issues getting netbsd up and running on amd64

electrolytes

I recently tried setting up a netbsd box for fun on an amd64 laptop. There have been/still are several issues I encountered, some of which I've implemented hacky workarounds for.

For the installer:

sysinst with 9.0 simply wouldn't install, it would abort trap no matter what I did after partitioning (either at partitioning when doing anything but the default full-disk, or at install).
sysinst with 9.99.71 worked, but only with the default full-disk partitioning, no changes to any filesystem or layouts. This also required wiping the first many bytes off the drive (I took 2048mb, because 1mb and 10mb didn't work).
if there's an lvm on the drive, prior to install, it has to be destroyed (gpt destroy wd0 for example, using the utility console), or else sysinst gets lethally confused.
I guess a manual install ought to work far better here...

Next, pkgsrc cannot install anything by default. I have gcc 7.3.0 from the comp set, but it's looking for libstdc++.so.6, which doesn't exist. Comp gives libstdc++.so.9 instead. For pkgsrc, changing mk.conf to require a more recent gcc, and installing said gcc, works. For other tasks (such as build.sh), this doesn't work. I copied libstdc++.so.9 into libstdc++.so.6 as a workaround. This may be addressed by doing a manual install instead of going through sysinst as well, I am assuming. This will probably be my favored workflow from now on.

Battery is currently detected but battery state (powered/discharging) does not seem to be detected (at least I see no messages in /var/log/messages and mate-desktop doesn't see the change). Would enabling acpi power monitor in the kernel help?

Touchpad options are a bit limited but otherwise work well. I couldn't find how to disable pointer acceleration, though, and I have to reset the scale/maxspeed at every boot for now. Probably missing what config file to use here? Or just .xinitrc or such?

Trying to get emul-linux working, I found that the suse13 packages are quite old and the handful of packages I tried did not seem to like it (my "goal test case" is to get portable palemoon working: it's a relatively complex program but doesn't necessarily require the likes of gtk3, pulseaudio, and such, so hopefully it could have worked with the suse13-based core). Palemoon just segfaults (it's a non-PIE linux x86_64 executable and COMPAT_LINUX and EXEC_ELF64 are active in the running kernel), and trying to run linux-ldd on it also causes linux-ldd to segfault (code 139).
I am currently trying to substitute it with a gentoo stage3 tarball, I pull the extra packages I need from the cloveros BINHOST. I setup the emul filesystems in fstab as instructed, but I'm missing something if I want to chroot into the emul root to have an easier time installing dependencies. I haven't investigated further here, although trying to execute binaries from this gentoo-like system fails due to a missing interpreter (e.g. when trying to execute ldd). Probably path-related? I'm half-aware that what I'm doing is crazy,. If someone has a saner solution, I'd like to hear about it. In any case, this should still more or less be in line with https://man.netbsd.org/compat_linux.8 although I'm going with the whole stage3 because of versioned dependency incompatibilities avalanching through the whole stack.

UPDATE: As expected, this did not help. Both the cloveros binary for palemoon and the upstream binary were tried, both segfaulted on launch. gdb is of no help here because it's not able to work with linux ABI binaries.

The cpu on the device is a kabylake, whose gpu seems to work out of the box. However, it does complain about not being able to find the i915 firmware (kbl_guc_ver9_16.bin and kbl_dec_ver1.bin are both mentioned). Said firmware doesn't actually appear to be available. Can I steal this from linux and put it somewhere? (If so: where? messages tells me this is an internal firmware so it has to be put in the initramfs or in-kernel, but I find no instructions to do either in the manual).

The device also features a touchscreen. It seems this is actually detected by netbsd on boot, however:

uts0: autoconfiguration error: touchscreen has no range support

And it fails to work.

Other than that, I'm surprised about how the rest of the system seems to be working for now. I'm not getting audio jack detection on linux, yet on netbsd it works out of the box. Wifi also worked straight away, and so on.

rvp

electrolytes I can try to help you with

manual install
gcc
battery
touchpad
touchscreen

and, possibly:

To start with:

Boot the kernel with debug messages
At the bootloader screen, get to the prompt then type boot -x. Post the dmesg output.
Output of this command: envstat--2 times; once when the laptop is on AC power and once more when it is on battery.

electrolytes

rvp
Thanks!

envstat

Correctly identifies that it's connected or not. The problem must therefore lie elsewhere.

dmesg

I don't know how to share this with you. The file is too big for any online paste service and I don't want to dump it raw in a post (though I will if there's no other way).

I think I also forgot to note in the OP that about 50% of the time, video fails to properly initializes at boot (screen is permanently turned off after netbsd tries to init the screen).
Additionally, I just noticed that xdm randomly needs to be restarted manually after a boot before it's able to launch anything, otherwise whatever it tries to launch fails, complaining the display can't be opened.

With regard to manual install, I'd like to see a consolidated "walkthrough" of setting up an encrypted system. Since NetBSD doesn't currently support FDE, maybe as CGD 1> LVM 1a> root 2a> home 2> boot as an example. I'm focusing on the other areas for now.
In that case, I assume the workflow looks like this:

Partition
Copy live media
Setup important config (fstab, hosts, etc.)
(optional) chroot, install/update/build.sh...

Questions that immediately come to mind:

What tools can be used to generate the ESP and set it up?
What kernel boot parameters are needed when using CGD and LVM, and how/where are they set?
Is LVM under CGD (with separate boot partition) supported?
What should the boot partition look like?
What needs to be done to ensure boot progresses properly (with regard to chaining or whatnot)?

rvp

electrolytes I don't know how to share this with you.

Send it to my username c/o the SDF Public Access Unix System.

electrolytes I think I also forgot to note in the OP that about 50% of the time, video fails to properly initializes at boot (screen is permanently turned off after netbsd tries to init the screen).

Can you install the latest 9.0-STABLE? Some of these problems may already have been fixed there compared to 9.0-RELEASE. (And, STABLE is also what I run.)

Re: Video problems in general: Older video cards are well supported, the latest ones are not so well supported. On these, you can run X on wsfb as a last resort.

electrolytes With regard to manual install, I'd like to see a consolidated "walkthrough" of setting up an encrypted system.

Never tried a CGD setup before, so I have no clue about this, but, it shouldn't be that difficult to figure things out. Do you want to encrypt only the home partition or all of NetBSD?

electrolytes What tools can be used to generate the ESP and set it up?

ESP is just a VFAT/FAT partition with a specific UUID and a standard directory structure. You create it using gpt and format it using newfs_msdos--standard tools.

All of this has mostly been covered on this board already:

NetBSD - A little guide for newcomers
Manual NetBSD installation on GPT/UEFI
Installing NetBSD UEFI + encrypted file system (pointer to relevant docs.)
Post-installation helper

The list above should get you started. Ask for clarifications here if needed.

electrolytes

rvp

Send it to my username

Sent it by email. Let me know if that worked.

Can you install the latest 9.0-STABLE?

Not currently, it fails with an abort trap during install. I'm on 9.99.74 (i.e. -CURRENT?) for that reason.

the latest ones are not so well supported.

9.0 features support for kabylake specifically so I expected it to be supported. The card is recognized by netbsd as "Intel HD Graphics 620" as per pcictl pci0 list, and netbsd 9.0 was supposed to have

Graphics driver update, matching Linux 4.4, adding support for up to Kaby Lake based Intel graphics devices.

According to an update in 2019.

All of this has mostly been covered on this board already:

Thanks for the links. With regard to ESP, an executable must be placed on it to be able to boot. According to the links, there is no need to generate a specific one for the disk install and it's sufficient to copy the one from the live media.

The 3rd link is basically the same question as me (mine is a variant) and the consensus effectively amounts to "it can't be done/nobody has done it": the post is about the user knowing about the docs but not being able to combine them, and replies specify this is not supported/experimental/etc. with nothing further. The variant in my case is that I'd like to add LVM to the mix.

Your 4th link is a script generator for a bunch of wrappers to unify commands across OSs: for example, mkdir is wrapped in makeDirectory(). It does not seem to have any particularly useful functions, and simply installs a bunch of programs according to the user's favored environment. It does not do anything special (such as fixing hardware, adjusting firmware loading, or anything pre- and intra-installation), so unfortunately it doesn't seem relevant.

The most interesting is the second link. While it does not answer any questions I mentioned, it is using that post that I got a lot of things setup (my questions are exclusively things that are not mentioned in that post or its replies for that reason). For example, it's by finding replies to that post mentioning the need to wipe the first few MBs of the disk that I was able to actually partition the disk without abort traps during install.

Unfortunately, only the first link addresses some of my questions. It's a good "cheatsheet" but does not go over non-"standard" partitioning, for example, unfortunately. It does go over bootloader setup and implies that the ESP boot binary is available in the liveimage which I was wondering.

rvp

electrolytes Sent it by email. Let me know if that worked.

Got it.

electrolytes The variant in my case is that I'd like to add LVM to the mix.

Great! Yet another obscure corner of NetBSD that I have to swot over now. Thanks a bunch, mate!

Describe how you want your NetBSD setup to be, so I can figure out what needs to be done:

Other OSes, or only NetBSD on Laptop?
One partition / holding everything or installation in separate partitions.
Why do you need a LVM?--with that single disk that you have.

electrolytes Your 4th link is a script generator for a bunch of wrappers to unify commands across OSs

Wait till you find yourself installing the same packages over and over again, or repeatedly tweaking some config. file, after yet another install fails... I have a post-inst.sh script myself, and before this little job is done, you will too. Take pointers from that link.

electrolytes The most interesting is the second link

Then I will give that link 1st place... Done.

electrolytes Only the first link addresses some of my questions

Which is why I put it first. (Now relegated to second place.)

electrolytes

rvp

Great! Yet another obscure corner of NetBSD that I have to swot over now. Thanks a bunch, mate!
😉

Describe how you want your NetBSD setup to be, so I can figure out what needs to be done:

NetBSD alone on the device is fine.
I would love to see FDE, but failing that:

Boot partition
At least 2 partitions (one /, one /home) under an encrypted FS
Without LVM, the amount of partitions that can be created on one disk is limited, and LVM additionally allows resizing partitions while the system is running (because they're not real partitions) while maintaining easy migration of, in this example, /home, if needed: install failure or need for a clean upgrade? Just decrypt and overwrite root, rest is good to go. This system needs a larger root? Not a problem either. More of a nice-to-have than a must, at least currently.

rvp

electrolytes I would love to see FDE

We'll leave FDE out for the moment--I've not quite wrapped my head around that.

electrolytes Without LVM, the amount of partitions that can be created on one disk is limited,

Only on MBR disks. You have a EFI/GPT setup--you can have upto 128 partitions.

electrolytes

rvp

We'll leave FDE out for the moment

All I've been reading says it's simply not supported, beside that and given the rest of the setup is understood, I should have no problem setting that up.

Only on MBR disks.

Didn't know that. Nice to know.

I've started testing some of my other peripherals in the meantime: a webcam (activating it instantly froze the system, couldn't even switch to other tty's) and a microphone (did not seem to generate signal).

Messages say:

savecore: check_kmem:412: kvm_read msgbuf: _kvm_kvatop(...)
savecore: reboot after panic: trap

I guess I should try again with boot -x and see what happens.
I also need to check support for them and if relevant, maybe which driver to scavenge from elsewhere.
EDIT: The panic seems to be sporadic, works just right on the reboot.

nia

The good news is that a lot of sysinst fixes made it into 9.1 just before it was tagged. You now get prompted to destroy the gpt partition table just before doing a whole-disk install.

If the webcam is attached via an xhci bus (USB 3), it will have that behaviour. USB Video devices depend on isochronous pipe support in the USB 3 stack, which is only partially implemented in -current. Webcams currently only work on NetBSD when they are attached over USB 2.

For hdaudio devices the microphone generating input will depend on the currently selected ADC. e.g. to use my laptop's internal mic, I have:

$ dmesg | grep ADC
[     1.001790] hdafg0: ADC01 2ch: Mic In [Jack]
[     1.001790] hdafg0: ADC02 2ch: Mic In [Built-In]
$ mixerctl record.source
record.source=ADC02

There's various ways to get something approximating FDE (honestly, that term annoys me... at best it's always partial). However, I'm lazy and always create a small unencrypted / with the installer then add encrypted partitions later. You could also do it the oldshool way, with chroot...

electrolytes

nia

If the webcam is attached via an xhci bus (USB 3), it will have that behaviour.

Yes, this seems to be the case according to usbdevs. This is an in-built webcam so I can't change the connectivity unfortunately. Regardless, as per my edit, no panic after the reboot: it works just right.

For hdaudio devices the microphone generating input will depend on the currently selected ADC. e.g. to use my laptop's internal mic, I have:

I am using your aiomixer to play with the sources and such. I have both a USB microphone and an inbuilt one: I see no additional source when I plug the microphone in (it's a snowball). Regardless of source chosen, I don't get any response from programs (e.g. firefox) nor from audiorecord. I also don't see any source being muted. I can see that mixerctl record.source updates as per aiomixer (as expected).

There's various ways to get something approximating FDE (honestly, that term annoys me... at best it's always partial).

Ultimately, rather than FDE (the means), what matters is the goal: avoiding unauthorized 3rd parties from peeking at the data while the device is turned off (if it's on, all bets are off anyway).

As you surely know, the basic rationale is that:
Unencrypted boot opens the way to evil maid attacks (but not much else, unless more than the bootsystem is exposed on the unencrypted disk). Unencrypted swap is more worrying if the user likes suspend to disk. Additionally, it can also be used in combination with other attacks to push pages into swap to be read by the attacker later. This has also previously been exploited to overwrite swapped-out pages to get arbitrary code execution as the page is restored. The point is that simply encrypted the disk of interest is of limited use in a vacuum, as the unencrypted disks defeat it.
Finally, in practice, all of this is rather unusual concerns except in very peculiar circumstances, or by policy requirements (arguably except the unencrypted swap as untrusted peripherals could leverage this in practice).

Perhaps veriexec is sufficient to defeat evil-maid attacks, and perhaps swap can safely live in the same encrypted device. If so, there is no difference in outcome as far as I know.

In the end, the reason I'm exploring this angle out of interest is that SED (sufficiently encrypted disk, since you don't like FDE) is very easy to setup on, say, linux (well, there are caveats to this notion of 'easy'...) and becomes "free security from volcanoes" -- the user is hardly impacted, beside having to enter a password, connect a device that contains a keyfile, or similar (the performance impact is negligible for almost all use-cases where SED is typically considered), and gains safety from a rare but possibly highly damaging event. SED is therefore usually the favored solution, but it doesn't matter what specific solution does achieve the goal of this setup.

Thus, another way to phrase the question is: what's a good setup to achieve data protection from unauthorized 3rd parties at rest?

You could also do it the oldshool way, with chroot...

Is there documentation somewhere about this?

Thank you for your response.

nia

I am using your aiomixer to play with the sources and such. I have both a USB microphone and an inbuilt one: I see no additional source when I plug the microphone in (it's a snowball). Regardless of source chosen, I don't get any response from programs (e.g. firefox) nor from audiorecord. I also don't see any source being muted. I can see that mixerctl record.source updates as per aiomixer (as expected).

According to duckduckgo the snowball is a USB microphone. USB microphones are not accessed through the primary sound card. Check dmesg...

It will register as a secondary audio and mixer device, i.e. /dev/audio1 and /dev/mixer1. aiomixer works on the primary mixer device by default, you can change that with -d. Same with audiorecord and the primary audio device.

There's an open issue to add a device selection menu to aiomixer but I haven't had the time.

Is there documentation somewhere about this?

There's documentation about doing it the "proper, non-hacky" way. But since it still relies on an unencrypted kernel (and we don't support any "Secure Boot" type stuff), you might not be satisfied. I'm not convinced it's worth doing compared to having a small root and moving the less static parts of the OS to encrypted partitions.

https://wiki.netbsd.org/security/cgdroot/

electrolytes

nia

I'm not convinced it's worth doing compared to having a small root and moving the less static parts of the OS to encrypted partitions.

I wanted to know about the "oldschool way, with chroot", not sure if it responds to the listed problem statement though.

I'm not convinced it's worth doing compared to having a small root and moving the less static parts of the OS to encrypted partitions.

As far as I, personally, am concerned, the whole thing is more of an exploration than a particularly "hard requirement". However, kernel on unencrypted partition should mean the ability to bypass any attempt to use veriexec at boot time even if that is possible, so that method would not work.
Could an initramfs + bootloader be placed on an external device (unencrypted) and setup to decrypt a drive and boot the kernel from there? Such a setup also works (using physical security in the chain). Deterministic mount/boot-time LABEL mounts?

According to duckduckgo the snowball is a USB microphone. USB microphones are not accessed through the primary sound card. Check dmesg...

Ah, thanks for the clarification.
It gets bound to /dev/audio1 and /dev/mixer1 as you said. However, I did not have any success correcting for this: aiomixer -d /dev/audio1 (or /dev/mixer1) shows it's not muted, as expected. audiorecord -d /dev/audio1 (/dev/mixer1 says invalid argument) does not seem to receive anything, then dies with read failed: Undefined error: 0 or read failed: Resource temporarily unavailable. In the latter case, dmesg says audio1: device timeout. The latter error is repeated indefinitely after the first error is encountered and until the device is unplugged and plugged back in.

Additionally, my internal microphone does not work (it should be on the main device). The audio card shows as Intel 200 series HD Audio (mixed mode multimedia, revision 0x21) but I don't see a microphone on usbdevs or on pcictl for the internal. I also don't see anything that seems relevant in dmesg.

electrolytes

After a few restarts, the USB microphone now works with no explanation. The internal microphone still does not appear to work, however.

I've been experiencing quite a few kernel panics. Just now another panic out of the blue (nothing was happening, no core dump). Before, a panic when recording sound, in usbi.c around line 1083 or so due to some queueing code (something about the pointer not being the head of the queue). Before that, another when the webcam was first tested, which was probably also related.

I wanted to try and see if I could figure out what this was about but I had trouble finding good documentation about where to start with kernel debugging. Vague mentions of kgdb (and mentions of requiring a second machine), but nothing else. Is there anything more substantial I could get started with?

20-100

electrolytes I like writing documentation but I haven't got very far into the subject yet - I've only passed the kernel & module building step...
If you're interested, we could use this thread to gather information, from which I could then write a HowTo document. It will not help us, but the next people with the same need will find it easier.

electrolytes

20-100
Sounds good to me, this is pretty much exactly what I intended the thread to be for anyway: gather information related to these adventures, wherever they may go.
Instructions for building are easily accessible, but I didn't get so lucky trying to go deeper, except if I would like to do a lot of trial and error (especially with regard to debugging).