Please share your BSD Tips and Tricks and we can publish a selection of them on the aggregator page: https://www.moritz.systems/bsd-tips-and-tricks/ Enjoy!

"Find the largest files in the current directory" by Kamil Rytarowski ``` find . -type f -exec stat -f "%z %N" {} \; | sort -r -n -k1 ``` https://www.moritz.systems/bsd-tips-and-tricks/find-largest-files-in-the-current-directory/ #FreeBSD #NetBSD #OpenBSD

@"kamil"#178 ...and with some eye-candy ;) [img]https://i.postimg.cc/zv5Y5B8D/2020-11-09-164728-1366x768-scrot.png[/img] I'm really liking this `broot` tool.

EDIT: removed stray `;` in the `c` function. @"kamil"#p5118 3 selections: 1. Remove duplicate lines in files (eg. rc.conf files, shell history files): ```sh $ cat ~/bin/nuniq.sh #!/bin/sh # # nuniq.sh: Remove duplicate lines, leaving only the last one. # (based on histsort.awk from "GAWK: Effective AWK Programming") exec awk ' { lines[NR] = $0 if ($0 in array) delete lines[array[$0]] array[$0] = NR } END { for (i = 1; i

To reduce the cycle of asking people for permission. Pasting here the tips and tricks means that I can publish them on the website freely, without asking and with the credit for submitter.

A tip which is trivial, but overlooked: Use -C to specify the directory in which `make` should run, as in: ``` make -C /usr/pkgsrc/devel/git-base install clean clean-depends ```

### MVMPIYTGA! Make viewing man pages in your terminal great again! `alias man='w3mman'`

@"rvp"#p5125 Is this form OK: `alias clr='printf "\033[H\033[J"'` ? I needed to delete the semicolon character as otherwise it was printed in the termnial (NetBSD, ksh).

@"[unknown]"#p5136 Use `clear` instead of hardcoding the sequence to stay terminal independent. If you need it multiple times and don't want to call en external binary each time: ``` $ CLEAR=$(clear) # or tput clear $ printf "$CLEAR" | hexdump -c 0000000 033 [ H 033 [ J 0000006 ```

> @"kamil"#p5136 I needed to delete the semicolon character I swear that wasn't there when I typed it in. It's these semicolons I tell you--they sneak right in somehow. Thank you for eradicating the beast. I made the `clear` work-alike a function because a) I prefer functions, and b) some shells don't have an `alias` command. However, `alias` is in POSIX, so go right ahead with them. As for permissions, etc: All code/scripts I write and post on this board can be considered to be in the public-domain. Use as you wish. Code & scripts I did not write, I will try to attribute correctly, and for these, you'll have to use your own judgement. Some sed(1) one-liners: 1. The sed(1) "list" command is very useful when dealing with unprintable (and invisible!) chars. I use it all the time because the other methods of dealing with these pesky characters is not very convenient: hexdump(1) is great, but, it doesn't preserve line endings; and vis(1) is a BSD-only thing. ```sh $ printf 'MS-DOS line ending\r\n' | sed -n l MS-DOS line ending\r$ $ printf 'Non-breaking space: '$'\u00A0''\n' | LC_ALL=C sed -n l Non-breaking space: \302\240$ ``` 2. Insert thousands-separators into numbers. The `-h` option to commands like du(1), df(1), etc. are nice and all, but, sometimes I want to see whole numbers in outputs. Then separators are useful to make sense of large numbers. ```sh #!/usr/bin/sed -Ef # # Insert US-style thousands-separators (commas) into numbers. : loop s/(^|[[:blank:]]+)([0-9]{1,})([0-9]{3})([0-9,]*)([[:blank:]]+|$)/\1\2,\3\4\5/ t loop ``` Use as: `du -a /some/dir | comma`, or `netstat -b -w 1 -I alc0 | comma`, etc. 3. Number lines in file. ```sh #!/bin/sh # # Simulate nl(1) or cat(1) -n exec sed -e = -- ${1+"$1"} | sed -nEe 'N;s/([[:digit:]]+)./\1'$'\t''/;p' ``` 4. NetBSD doesn't come with a tree(1) command so here's one in sed(1): ```sh #!/bin/sh # # Print directory tree exec find -- ${1:-.} | sed -Ee '2,$s:[^/]*/: :g; s: ([^ ]+)$:|_ \1:' ``` Note: it doesn't draw a fully-connected tree. Drawing a full tree is very hairy--and, completely out of our self-imposed one-liners category. 5. Reverse-print a file--aka tac(1): ```sh #!/bin/sh # # Implement tac(1) using sed(1). exec sed -ne '1h; 2,${x;H;}; ${x;p;}' -- "$@" ``` 6. Same thing in ed(1); using the standard idiom: ```sh #!/bin/sh # # Implement tac(1) using ed(1). : ${1?"Usage: ${0##*/} FILE"} exec ed -s -- "$1"

> @"qik"#p5133 A tip which is trivial, but overlooked: Use -C to specify the directory in which make should run Yes, just like `patch -d /some/dir ...`

Moritz Systems | BSD Tips and Tricks

pfr

oui I highly recommend this little gold nugget

Oh gee, this is an excellent walk through for someone who's never used ed (me!). Thanks for sharing 🙏

JuvenalUrbino

List 20 largest files (larger than 5 MB) sorted by megabytes:

$ find / -type f -size +10000 -print0 | xargs -0 du -m | sort -nr | head -20

JuvenalUrbino Or, if you use nushell

ls -la | where size > 5Mb 😉

rvp

JuvenalUrbino Can be shortened somewhat (and, a process or two saved):

$ find / -size +5M -exec du -m {} + | sort -nr | head -n20

JuvenalUrbino

pin looks like the systemd of command-line interpreters 😆

JuvenalUrbino have a look at the book and see what you can do, https://www.nushell.sh/book/

JuvenalUrbino

pin captivating. That's a really cool project. The explicit commands, the syntax, and their slightly imperative approach remind me a bit of OpenVMS DCL:

$ ll :== dir /size/date=mod
$ ll

Directory SYS$SYSDEVICE:[USER.ANANKE]

$MAIN.TPU$JOURNAL;1
                           1   8-NOV-2021 11:45:10.23
HELLO.COM;1                1   2-FEB-2022 13:11:23.41
LOGIN.COM;1                3   2-FEB-2022 13:14:52.13
LOOP.COM;1                 1   2-FEB-2022 12:56:39.76
LOOP.LOG;1                 1   2-FEB-2022 12:58:48.90
MAIL.MAI;1               360   2-FEB-2022 13:11:00.33
SYSPRIV.COM;1              1   2-FEB-2022 12:59:47.19

rvp

Sometimes, it's very convenient to make the mod. time of a directory match the latest file in that directory: For instance, on FTP servers, just check the mod. time on a directory to bypass checking for the latest file in that directory.

I use this script:

rtouch.sh

#!/bin/sh
#
# Change modification time of directory to match
# the latest file in that directory.
#
# Recursive version:
# find DIR -depth -type d -exec rtouch.sh {} \;

set -eu
cd -- "$1"
F=$(ls -1ABt | head -n1)
[ -n "$F" ] || exit 0
F=$(echo "$F"_ | unvis)
F=${F%_}
exec touch -r "$F" .

Usage is:

$ rtouch.sh DIR

And, recursively:

$ find DIR -depth -type d -exec rtouch.sh {} \;

Should work even if your files have weirdo chars. in them. If not, tell me--I have other versions. (This is one of those scripts that I end-up re-creating 'cuz I forget that I already have it lying around in my other OS's ~/bin directories.)

rvp

Convert MS-DOS line-endings to Unix line-endings. (Because, many things--ftp xfers in ASCII mode being one--can change text causing patch files, for instance, to not apply cleanly). It is tempting to use trfor this:

$ tr -d '\r' < IN > OUT

but, I prefer to remove \r only when they constitute a line-ending:

dos2unix.sh

#!/bin/sh

# short ver. (no stdin)
# exec sed -e $'s/\r$//' -i '' "$@"

set -eu
(set -o pipefail 2>/dev/null) && set -o pipefail
me=${0##*/}

EOL=$(printf '\r$')	# MS-DOS EOL (\r\n)

[ $# -eq 0 ] && exec sed -e "s/$EOL//"

for f
do	if head -n1 "$f" | grep -q "$EOL"
	then	sed -e "s/$EOL//" -- "$f" > "$f.$$"
		touch -r "$f" "$f.$$"
		if mv "$f.$$" "$f"
		then	echo "$f"
		else	echo "$me: $f: failed" >&2
			exit 1
		fi
	fi
done

Just uncomment the short version if you prefer that (after adjusting -i '' for your sed).

rvp

Transferring multiple files using netcat:

nc-send.sh

#!/bin/sh
set -eu -o pipefail
for f
do	test -h "$f" -o ! -f "$f" -o ! -r "$f" -o ! -s "$f" && continue
	echo $(stat -f %z "$f") "${f##*/}"      # | tee /dev/stderr
	cat "$f"
done | nc -N localhost 9000

nc-recv.sh

#!/bin/sh
set -eu -o pipefail
exec nc -l 9000 |
while read SIZ FN
# do	dd bs=1 count=$SIZ of="$FN" 2>/dev/null
do	dd iflag=fullblock bs=$SIZ count=1 of="$FN" 2>/dev/null
done

Usage

Start nc-recv.sh somewhere first, then run nc-send.sh file1 file2 ...--after changing the server hostname in the script.
If your dd doesn't understand iflag=fullblock, then uncomment the other one (portable, but slow).
But really, should just use tar for this.

rvp

If you have a single C source file, then simply use make to compile it for you:

$ make foo
cc -O2 -pipe  foo.c  -o foo
$

Of course, you can customize the build (for debug let's say):

$ make CFLAGS=-g LDFLAGS= foo
cc -g  foo.c  -o foo
$

rvp

Draw a ruler at the bottom of the screen. Useful for a variety of reasons.
If some program erases the scrolling-region ruler, simply do ruler on again to restore.
If you have xterm >= 371, then you get a persistent status-line ruler.
Turn off using ruler off

ruler

#!/bin/sh
#
# ruler: show a ruler at the bottom of the screen.
# Usage: ruler on|off
#
# Do `ruler on' again if some program erases the scrolling-region ruler.
# The status-line ruler currently needs xterm >= 371

set -eu
me=${0##*/}

# Usage
#
usage() {
	echo "Show a ruler at the bottom of the screen."
	echo "Usage: $me on|off"
}

# Make a ruler line
#
mkruler() {
	local s=".........1.........2.........3.........4.........5.........6.........7.........8.........9.........0"
	local c=$1 i=0 n

	while [ $i -lt $c ]
	do	n=$(( (c-i) <= ${#s} ? (c-i) : ${#s} ))
		printf "%.*s" $n $s
		i=$((i + n))
	done
}

# Save current attributes and cursor position
#
save() {
	tput sc
}

# Restore saved attributes and cursor position
#
rest() {
	tput rc
}

# Move cursor to specified ROW ($1) COL ($2)
#
move() {
	$tcap && { tput cm $(($2 - 1)) $(($1 - 1)); return $?; } ||
		 { tput cup $(($1 - 1)) $(($2 - 1)); return $?; }
}

# Set bold mode
#
bold() {
	$tcap && { tput md; return $?; } || { tput bold; return $?; }
}

# Set scrolling region from START ($1) to END ($2) lines
#
csr() {
	$tcap && { tput cs $(($2 - 1)) $(($1 - 1)); return $?; } ||
		 { tput csr $(($1 - 1)) $(($2 - 1)); return $?; }
}

# Clear to end of line
#
ceol() {
	$tcap && { tput ce; return $?; } || { tput el; return $?; }
}

# Make user-defined status-line (DECSSDT/DECSASD)
# (xterm >= 371)
#
mkdecsl() {
	printf '\033[2$~'		# set user-defined status-line
	printf '\033[1$}'		# switch to status-line
	bold
	mkruler $COLS
	printf '\033[0$}'		# switch to main display
}

# Remove status-line
#
rmdecsl() {
	printf '\033[0$~'		# no status-line
}

# Make scrolling-region-based status-line (DECSTBM)
# (almost all VT100-compatible terminals)
#
mkcsrsl() {
	move $ROWS 1			# move to last line
	bold
	mkruler $COLS
	csr 1 $((ROWS-1))		# make scrolling-region excl. last line
}

# Remove scrolling-region-based status-line
#
rmcsrsl() {
	move $ROWS 1			# move to last line
	ceol				# clear ruler
	csr 1 $ROWS			# put back default scrolling-region
}

# Do a DECRQSS query and return response
#
rqss() {
	local B='\' E=$(printf '\033') S="" ch esc=false run=true ttyst

	ttyst=$(stty -g)		# save term state
	stty cbreak -echo min 0 time 1
	printf %s "$1" >/dev/tty
	while $run			# read until string-terminator: ESC \
	do	ch=$(dd bs=1 count=1 2>/dev/null || true)
		case "$ch" in
		"")	break ;;	# timeout
		"$E")	esc=true ;;
		"$B")	$esc && run=false ;;
		*)	esc=false ;;
		esac
		S="$S$ch"		# collect response
	done
	stty "$ttyst"			# restore saved state

	printf %s "$S"
}

# Check if terminal has DECSSDT/DECSASD, or, if we must use DECSTBM
# by using DECRQSS
#
okdecsl() {
#	local DECSTBM=$(rqss "$(printf '\033P$qr\033\\')")
	local DECSSDT=$(rqss "$(printf '\033P$q$}\033\\')")
	local DECSASD=$(rqss "$(printf '\033P$q$~\033\\')")

#	echo "$DECSTBM" | egrep -q "$(printf '^\033P1\\$r.+r\033\\\\$')" \
#		&& DECSTBM=true || DECSTBM=false
	echo "$DECSSDT" | egrep -q "$(printf '^\033P1\\$r.+\\$\\}\033\\\\$')" \
		&& DECSSDT=true || DECSSDT=false
	echo "$DECSASD" | egrep -q "$(printf '^\033P1\\$r.+\\$~\033\\\\$')" \
		&& DECSASD=true || DECSASD=false

	if $DECSSDT && $DECSASD
	then	return 0		# has status-line capability
	else	return 1		# assume has scrolling-region
	fi
}

# Check if we need to use termcap (FreeBSD) or terminfo (NetBSD/Linux)
#
oktinfo() {
	local rc

	save
	tput cup 0 0 >/dev/null
	rc=$?
	rest
	return $rc
}




# S T A R T
#
if [ $# -eq 0 ]
then	usage 1>&2
	exit 1 
fi

okdecsl && sl=true || sl=false
oktinfo && tcap=false || tcap=true
arg=$1
set -- $(stty size)
ROWS=$1
COLS=$2

case $arg in
on)	save
	$sl && mkdecsl || mkcsrsl
	rest
	;;
off)	save
	$sl && rmdecsl || rmcsrsl
	rest
	;;
-h)	usage
	exit 0
	;;
*)	echo >&2 "$me: don't know \`$arg'. Try -h."
	exit 1
	;;
esac

Xterm >= 371 implements the DEC status-line escape-sequences (if compiled).

Enable the default status-line using:

printf '\033[1$~'

Disable it using:

printf '\033[0$~'

or ruler off

JuvenalUrbino

@kamil what's the current state of sailor? I was thinking about running some services in a dedicated container.

rvp

Some nc tricks:

1. Remote shell

Start server

mkfifo f
<f sh -i 2>&1 | nc -l 9998 >f

Run client

nc -N localhost 9998

This example is from the Linux man-page. (Missing in the *BSDs.)

Note:

You won't get any command-line editing, nor be able to run full-screen programs like vi.
(But, everyone here already knows how to use ed, right?)

2. SOCKS5 proxy for non-SOCKS clients

Standard lynx doesn't support the SOCKS protocol, so if you want to use TOR to connect to, let's say the NNTP server at retrobsd.ddns.net, then you'd usually have to install socat. However, since nc is already in base, let's use that.
Assuming your TOR server runs on localhost:9050:

Start proxy forwarder

mkfifo f
<f nc -xlocalhost:9050 retrobsd.ddns.net 119 | nc -l 9999 >f

Run client

lynx nntp://localhost:9999/

and you should be connected to the fine NNTP server at retrobsd.ddns.net via TOR.

Note:

The forwarder will quit when the client exits. Run the command in a loop if you want to service multiple (serial, not simultaneous) clients.
This technique isn't suitable for protocols like HTTP or Gopher where the client send a request, the server responds and then closes the connection. Due to the way nc works, it won't quit until the local client (like lynx) tries to send another request (which won't happen).
Add a -w1 before -x... for "fix" this issue; but it's an ugly hack nonetheless.
Another reason why this is not suitable for HTTP, Gopher, Gemini etc. is that these exist to provide links to other, external, sites. Links to the same site will use the forwarder, but, links to a different site will cause the browser to make a direct connection. Ie. this technique is not a general-purpose $http_proxy

rvp

Display text in the middle of the screen (pass -c to centre every line). Useful for reading e-books, man-pages (hint: col -bx) etc.

middle.awk

#!/usr/bin/awk -f
#
# Display text of each FILE in the middle of the screen.
# With `-c', centre each line.
#
# Usage: middle.awk [-- -c] [FILE...]

function getcols(	a, cmd, col, s) {
	col = ENVIRON["COLUMNS"] + 0
	if (col > 0)			# user override
		return col
	cmd = "stty -f /dev/tty size 2>/dev/null"
	cmd | getline s
	close(cmd)
	split(s, a)
	col = a[2] + 0
	return (col > 0) ? col : 80
}

function strip(str,	s) {
	s = str
	sub(/^[[:blank:]]+/, "", s)	# leading blanks
	sub(/[[:blank:]]+$/, "", s)	# trailing  "
	return s
}

function len(str,	a, i, ln, n) {	# compute length of str,
					#  accounting for backspace.
	n = split(str, a, "")
	for (i = 1; i <= n; i++)
		(a[i] == "\b") ? ln-- : ln++
	if (ln < 0) ln = 0
	return ln
}

function pr(L, N,	i, n) {
	for (i = 1; i <= N; i++) {
		n = COLS/2 - LL/2
		if (n < 0) n = 0	# hopeless: line len. > scr. width
		printf "%*s%s\n", n, "", L[i]
	}
}

BEGIN {
	STYLE = "middle"		# default
	if (ARGV[1] == "-c") {
		STYLE = "centre"
		ARGV[1] = ""
	}
	COLS = getcols()
}

{
	if (STYLE == "middle") {
		if (FN != FILENAME) {	# new file
			FN = FILENAME
			pr(L, N)	# print saved prev. file
			LL = N = 0	# reset
		}
		if ((n = len($0)) > LL) LL = n
		L[++N] = $0
	} else {			# STYLE == "centre"
		s = strip($0)
		n = COLS/2 + length(s)/2
		printf "%*s\n", n, s
	}
}

END {
	if (STYLE == "middle")
		pr(L, N)
}

rvp

Colours in less (nice when viewing man-pages). For example:

$ export LESS='--use-color -DN240$Dd+w$Du+C$'
$ man 2 open

will show line-numbers (if enabled with -N) in grey, bold text as bold+white, and underlined text as underlined bold+cyan (as Slackware used to back in the day).

See the description of the -D flag in the man-page for more details.

pfr

rvp

~ λ export LESS='--use-color -DN240$Dd+w$Du+C$'
~ λ cat nsxiv-play.sh | less
There is no use-color -DN240$Dd+w$Du+C$ option ("less --help" for help)

rvp

Batch rename files using sed

bren.sh

#!/bin/sh

set -eu
me=${0##*/}

usage() {
	cat <<EoF
$me: Batch rename files using sed(1).
Generates new name by applying SED-CMD to each FILE.

Usage: $me [-nv] SED-CMD FILE...
  -n	no execute (implies -v)
  -v	verbose operation

Eg: $me 's/jpg$/png/' *.jpg

EoF
}

# Defaults
#
noexec=false
verbose=false

while getopts nv opt
do	case $opt in
	n)	noexec=true
		verbose=true
		;;
	v)	verbose=true
		;;
	\?)	usage >&2
		exit 1
		;;
	esac
done
shift $((OPTIND - 1))

[ $# -lt 2 ] && { usage >&2; exit 1; }

cmd=$1		# sed cmd. usually, s/BRE/repl/
shift

for f
do	F=$(basename "$f")	# we work only on filenames,
	D=$(dirname "$f")	# and use the same directory.
	new=$(echo "$F" | sed "$cmd")	# dis be it, mon.
	[ "$F" = "$new" ] && continue	# no match--skip
	F="$D/$new"
	if [ -f "$F" ]
	then	echo >&2 "$me: $F: exists--skipping"
		continue
	fi
	$verbose && echo /bin/mv -i "$f" "$F"
	$noexec || /bin/mv -i "$f" "$F"
done

rvp

pfr You need less >= 581. Compile it yourself: the one in pkgsrc is also pretty old (@pin: please update?)

misc/less has a maintainer.

# $NetBSD: Makefile,v 1.29 2021/01/06 14:29:30 leot Exp $

DISTNAME=	less-563
CATEGORIES=	misc
MASTER_SITES=	http://www.greenwoodsoftware.com/less/

MAINTAINER=	leot@NetBSD.org
...

rvp

pin misc/less has a maintainer.

And now, @pfr knows the proper protocol.

« Previous Page Next Page »