I know there is a shit tone of info on the internet about how to record screen casts using ffmpeg but I am wanting to know how to also capture my systems audio (so I can play a track in the background). I'm just not sure what command to use. An example I was given: `ffmpeg -f pulse -i -ac 2 -thread_queue_size 128 -f x11grab -s $RESOLUTION -i :0.0 -vcodec libx264 -preset ultrafast -threads 0 -loglevel quiet output.mkv` I need to know what the pulse source is on my system for I tried `pactl list short sources` as suggested on ffmpeg's wiki but it doesn't look like `pactl` is installed nor is available... Any idea what I should I use in its place to find the input device?

pulse is the pulseaudio source, if you want to capture pulseaudio's output you need (a) pulseaudio running (b) applications to be outputting to pulseaudio. An alternative on NetBSD is using a psuedo audio device, see https://man.netbsd.org/pad.4

@"nia"#p8024 Ok cool, thanks. can you give me an idea how I can use `pad(4)` in the ffmpeg command instead of pulse to capture my screencast _and_ any music I play from an app like `ncspot` (which i believe uses `portaudio`)? currently my screencapture.sh looks like this: ``` #!/bin/sh RESOLUTION=1366x768 ffmpeg -thread_queue_size 128 -f x11grab -r 30 -s $RESOLUTION -i :0.0 -vcodec libx264 -preset ultrafast -threads 0 -y output.mkv ffmpeg -i output.mkv workflow-${USER}.webm ```

@"[unknown]"#p8051 Hate to use a trope, but WorkForMe(tm). I just have to restart audio playback for some reason, then it's seamless.

Screen recording with ffmpeg

nia

I don't know how to point ncspot at a secondary audio device, maybe @pin can help.

nia I wonder if the easiest way wouldn't be for @pfr to build ncspot with the pulseaudio back-end?

librespot itself supports most audio back-ends but, ncspot supports only alsa, pulse, portaudio and rodio.

JuvenalUrbino

A temporary workaround would be using another player either capable of speaking to /dev/audio directly or through ossaudio(3), then pointing ffmpeg to it by passing -i /dev/audio -f oss. @pfr might be interested in my ffmpeg screencast thread

nia

JuvenalUrbino I think this won't work -- you need the pad in there to actually get the player's output fed back to userspace. If nscpot really doesn't have a way to configure PortAudio's default audio device (portaudio speaks /dev/audio), the pad's audio device could be made the default output using audiocfg, which would make portaudio use it.

pfr

nia I'm still unsure how to call pad from the script. Can you give me an example?

Secondly, what if I just use mpg123 instead of ncspot?

I've tried:

mpg123 -a /dev/audio /home/$USER/Music/track.mp3

ffmpeg -f oss -i /dev/audio -thread_queue_size 128 -f x11grab -r 30 -s $RESOLUTION -i :0.0 -vcodec libx264 -preset ultrafast -threads 0 -y -loglevel quiet output.mkv

However this still doesn't capture the audio.

nia

That's because telling ffmpeg to read from /dev/audio will read from your microphone rather than mpg123.

Maybe something like:

ffmpeg4 -f s16le -ar 44100 -ac 2 -i /dev/pad0 -f x11grab -r 30 -s 640x480 -i :0.0 -vcodec libx264 -preset ultrafast output.mkv
mpg123 -a /dev/audio1 file.mp3

(This is mostly copied from the example in the pad man page, with your x11grab bits added.)

mpg123 needs to run after ffmpeg, only then will ffmpeg start recording. The reason for this is that the fake audio device /dev/audio1 is only created after /dev/pad0 is opened. Once mpg123 starts writing to /dev/audio1, the audio samples become available through the pad that ffmpeg opened.

rvp

nia This is, of course, the standard technique to make screencasts with background music, but, it doesn't work--neither on NetBSD nor on OpenBSD (with its sndio).

The only reliable, workable method on both OSes is to have 2 separate ffmpeg processes running: one for background music and another for MIC input + screen grab, both writing NUT files which are then combined into the final video file.

Of course, this precludes any live screencasting.

nia

rvp Actually, my example works just fine for only recording background music combined with the X11 display, which is what I thought OP was asking about. If you could point out why it doesn't work I'd be interested to know.

ffmpeg does seem to struggle when combining multiple audio sources (pad + mic) with the X11 recording, but perhaps there's a workaround, e.g. pointing it at a shoutcast stream. I'd be tempted just to play some music from speakers in the background. 😃 pad is quite simple because it was designed for the single purpose of streaming NetBSD's audio output to an AirTunes device, but we could also look into extending it for use cases like this.

I'd also be curious about the actual latency of having a separate ffmpeg process combining the streams, tbh.

rvp

nia If you could point out why it doesn't work I'd be interested to know.

It didn't work because the default -thread_queue_size is set too low.

Raise it, and use the sunau device instead of oss, and everything seems to work OK:

#!/bin/sh
set -eu
${1:?"$0: no output file given."}
RATE=30
SIZE=1366x768
ffmpeg \
	-thread_queue_size 2048 -f s16le -ar 44100 -ac 2 -i /dev/pad \
	-f sunau -i /dev/audio \
	-filter_complex '[0] [1] amix' \
	-f x11grab -r $RATE -s $SIZE -i :0 -vcodec libx264 -preset ultrafast \
	"$1" -map 0:a -c:a copy -f sunau /dev/audio

And:

$ env AUDIODEV=/dev/audio1 mpv /mnt/snd/Korean/playlist.m3u

pfr

I tried both your suggestions and get these errors. The ffmpeg

$ env AUDIODEV=/dev/audio1 mpg123 music.mp3     
High Performance MPEG 1.0/2.0/2.5 Audio Player for Layers 1, 2 and 3
        version 1.27.2; written and copyright by Michael Hipp and others
        free software (LGPL) without any warranty but with best wishes
[src/libmpg123/readers.c:INT123_open_stream():1204] error: Cannot open file music.mp3: No such file or directory
main: [src/mpg123.c:open_track():775] error: Cannot open music.mp3: File access error. (code 22)

$ mpg123 -a /dev/audio1 music.mp3  
High Performance MPEG 1.0/2.0/2.5 Audio Player for Layers 1, 2 and 3
        version 1.27.2; written and copyright by Michael Hipp and others
        free software (LGPL) without any warranty but with best wishes

Terminal control enabled, press 'h' for listing of keys and functions.

Playing MPEG stream 1 of 1: music.mp3 ...

MPEG 1.0 L III vbr 44100 j-s

[src/libout123/libout123.c:out123_play():739] error: Error in writing audio, wrote only -1 of 4608 (Input/output error?)!
main: [src/mpg123.c:play_frame():876] error: Deep trouble! Cannot flush to my output anymore!

And from ffmpeg I get this:

...
Guessed Channel Layout for Input Stream #0.0 : stereo
Input #0, s16le, from '/dev/pad0':
  Duration: N/A, bitrate: 1411 kb/s
  Stream #0:0: Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s
Guessed Channel Layout for Input Stream #1.0 : stereo
Input #1, oss, from '/dev/audio':
  Duration: N/A, start: 1623034291.434141, bitrate: 1536 kb/s
  Stream #1:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
At least one output file must be specified
$

Although, @rvp the script only runs if I comment out the line containing -filter_complex '[0] [1] amix' because otherwise I get:

$ sh screenrecord.sh  
screenrecord.sh: 1: parameter not set

I also tried with env AUDIODEV=/dev/audio1 mpv --no-video music.mp3 and it looked as though the track was playing but I heard no sound.

$ env AUDIODEV=/dev/audio1 mpv --no-video music.mp3 
 (+) Audio --aid=1 (mp3 2ch 44100Hz)
AO: [sdl] 44100Hz stereo 2ch s32
audio: Input/output error%)
A: 00:00:04 / 00:03:55 (2%)

Exiting... (Quit)
$

😕
I'm fully aware this is probably due to my lack of understanding of the script itself, but I tried a bunch of different things before posting this comment and my brain capacity is maxed out 😆

rvp

pfr
At least one output file must be specified
screenrecord.sh: 1: parameter not set
error: Deep trouble! Cannot flush to my output anymore!

Same cause for each: you've not specified an output file to the script: screenrecord.sh foo.mp4

pfr $ env AUDIODEV=/dev/audio1 mpg123 music.mp3

The AUDIODEV env. variable is for SDL, which is what mpv uses for audio output. For mpg123, specify alternate output device with -a device

pfr it looked as though the track was playing but I heard no sound.

Yes, so I've noticed; because, I think, we're playing to /dev/audio1, which is only hooked up to pad0-- i.e. not to the actual output HW. But, no worries, the playback and mic audio do get mixed in properly though.

nia

rvp

Yes, so I've noticed; because, I think, we're playing to /dev/audio1, which is only hooked up to pad0-- i.e. not to the actual output HW. But, no worries, the playback and mic audio do get mixed in properly though.

In the example from here
http://netbsd.org/docs/guide/en/chap-audio.html#chap-audio-pad

I use ffmpeg to output back to the primary audio device by specifying it as an output at the end of the command line i.e. -f oss /dev/audio

rvp

nia use ffmpeg to output back to the primary audio device

Yes, that's much more sensible than what I was contemplating: using hdaudioctl to split out speaker+HP into separate speaker & HP devices (and, not sure here if will get 2 devices); or do the split earlier at the source.

This should've worked, but, the "monitor" playback is very choppy:

ffmpeg -thread_queue_size 2048 \
        -f s16le -ar 44100 -ac 2 -i /dev/pad \
        -f oss -i /dev/audio \
        -filter_complex '[0] [1] amix' \ 
        -f x11grab -r 30 -s 1366x768 -i :0 -vcodec libx264 -preset ultrafast \
        output.mp4 -map 0:a -c:a copy -f oss /dev/audio

And, the example in the guide doesn't work at all.

nia

rvp I think the thread_queue_size argument was not necessary when I wrote it... dunno what's changed.

rvp

nia dunno what's changed.

This does the trick: add in /etc/sysctl.conf:

hw.audio0.blk_ms=1000

nia

rvp I'd... rather not.

rvp

nia i can find no other workaround for that choppy playback.

pfr

nia I'd... rather not.

Can I ask why? What are the side affects of doing this?

rvp

rvp i can find no other workaround for that choppy playback.

There is one: use the sunau driver instead of oss and everything looks OK--so far. This may even work for live streaming now, I think.

Post #13 updated.

nia

pfr Increases the system-wide audio latency from 10 milliseconds to 1 second.

Also... https://freshbsd.org/netbsd/src/commit/YXkYhr6TqxmFajWC

rvp

nia Yeah, I noticed that.

pad has other issues: eg. one has to stop the screen-capture first, and only then stop the background audio playback: ffmpeg will choke if you switch this order. Caveat emptor, etc...