pkg_comp is a utility to build pkgsrc packages in a fully automated and self-contained manner. It can be seen as a wrapper over cvs/git, the pbulk infrastructure provided by pkgsrc, and sandboxctl(8).
This turns out particularly useful for whenever official binaries are not available: packages in sync with latest pkgsrc-current changes, packages linked against a -current userland, packages for tier2 archs or archs added recently and only supported in current (pkg_comp used to be my go to on aarch64), packages with licenses preventing them to be freely redistributed as binary.
pkg_comp is also useful to set up a small repo to serve on the local network (a trivial task using ftpd(8) or bozohttpd(8)), if bandwidth and download speed are problem.
A third interesting use case is compiling packages with custom build options (and prevent them from being refreshed/replaced by a pkgin fug
), as well as applying local patches.
preliminary tasks
Get the sysrc and xsrc source sets matching your system version and unpack them to /usr, for packages needing to be linked against them.
See Chapter 32. Obtaining the sources.
Fetch the release binary sets matching your system version. You can use sysupgrade fetch
, it will store them automatically at /var/cache/sysupgrade.
Set up the release dir which sandboxctl shall use to create the sandbox. As of today, sandboxctl doesn't seem to allow specifying the archive extention, hence .tgz is expected. In NetBSD-9, I got around this by using the following xz2gz
script, which will convert the archives to gzip and move them to /release/binary/sets
#!/bin/sh
tgzdir='/release/binary/sets'
xzdir='/var/cache/sysupgrade'
if [ ! -d $tgzdir ]; then
mkdir -p $tgzdir
fi
for set in ${xzdir}/*.xz
do
xz -d $set
gzip ${xzdir}/*.tar
done
for set in ${xzdir}/*.tar.gz
do
mv $set $(basename $set .tar.gz).tgz
done
for set in ${xzdir}/*.tgz
do
mv $set $tgzdir
done
configure pkg_comp
Configuration files for pkg_comp and its sandbox are stored by default at /usr/pkg/etc/pkg_comp
- sandbox.conf
# The `netbsd-release' sandbox type sets up a NetBSD sandbox on a NetBSD
# host using release sets. Sandboxes created in this manner are "clean" in
# the sense that they reproduce a freshly installed system and are
# completely decoupled from the host system
SANDBOX_TYPE=netbsd-release
# sandbox path
SANDBOX_ROOT=/var/tmp/sandbox
# Parameters for a netbsd-release sandbox
NETBSD_RELEASE_RELEASEDIR=/release
NETBSD_RELEASE_SETS="base comp etc games man misc modules rescue tests text xbase xcomp xetc xfont xserver"
# User-provided routines required for building packages that use X libraries
# or need to build against system and/or X sources
pkg_comp_post_mount_hook() {
#
# Required to be able to build any packages that uses X and its libraries
#
if [ -d /usr/X11R7 ]; then
mkdir -p ${SANDBOX_ROOT}/usr/X11R7
sandbox_bindfs -o ro /usr/X11R7 ${SANDBOX_ROOT}/usr/X11R7
fi
#
# Optional, but needed for any packages which need access to system or X sources
#
if [ -d /usr/src ]; then
mkdir -p ${SANDBOX_ROOT}/usr/src
sandbox_bindfs -o ro /usr/src ${SANDBOX_ROOT}/usr/src
fi
if [ -d /usr/xsrc ]; then
mkdir -p ${SANDBOX_ROOT}/usr/xsrc
sandbox_bindfs -o ro /usr/xsrc ${SANDBOX_ROOT}/usr/xsrc
fi
exit 0
}
- local.conf , our local pkg_comp configuration
# update your pkgsrc tree whenever invoking a pkg_comp build.
# not strictly necessary on quartery branches
UPDATE_SOURCES=true
# Additional configuration files to support this setup.
EXTRA_MKCONF="/usr/pkg/etc/pkg_comp/extra.mk.conf"
SANDBOX_CONFFILE="/usr/pkg/etc/pkg_comp/sandbox.conf"
# Remote VCS configuration.
FETCH_VCS=cvs
CVS_ROOT=:ext:anoncvs@anoncvs.NetBSD.org:/cvsroot
CVS_TAG=pkgsrc
#GIT_URL=https://github.com/NetBSD/pkgsrc.git
#GIT_BRANCH=trunk
# Host file layout
# These directories point to trees managed outside of the
# sandbox and are exposed within the sandbox via null mounts.
PKGSRCDIR="/usr/pkgsrc"
DISTDIR="/usr/pkgsrc/distfiles"
PACKAGES="${PKGSRCDIR}/packages"
PBULK_LOG="${PACKAGES}/log"
PBULK_PACKAGES="${PACKAGES}/pbulk"
# Target file layout
LOCALBASE=/usr/pkg
PKG_DBDIR=/var/db/pkg
SYSCONFDIR=/usr/pkg/etc
VARBASE=/var
# Enable interactive pkgsrc development via sandbox-shell.
PKG_DEVELOPER=yes
# List of packages to build during automatic execution.
AUTO_PACKAGES="$(grep -v '^#' '/usr/pkg/etc/pkg_comp/pbulk.list')"
- extra.mk.conf, additional pkgsrc settings to be used with pkg_comp.
# extra
# sanity checks for shared libs
PKG_DEVELOPER=yes
# compatibility pkgdbdir
PKG_DBDIR=/var/db/pkg
# build on tmpfs
WRKOBJDIR=/var/tmp/pkg
# cflags
#CPUFLAGS+=-O2 -pipe -fomit-frame-pointer -mfpmath=sse -msse2 -march=native
#CFLAGS+=-Wfatal-errors
# ccache
#PKGSRC_COMPILER=ccache gcc
#CCACHE_DIR=/var/tmp/ccache
# make jobs
MAKE_JOBS=$(/sbin/sysctl -n hw.ncpuonline)
# PKG_ALTERNATIVES
PYTHON_VERSION_DEFAULT=39
LUA_VERSION_DEFAULT=54
PHP_VERSION_DEFAULT=80
RUBY_VERSION_DEFAULT=30
# avoid linking against outdated libs
PREFER_PKGSRC?=MesaLib Xft2 Xrandr Xrender expat fontconfig freetype2 glu openssl pixman xcursor
# user patches
LOCALPATCHES=/usr/pkg/etc/pkg_comp/patches
# other
SKIP_LICENSE_CHECK=yes
#NO_CHECKSUM=yes
#ALLOW_VULNERABLE_PACKAGES=yes
# PKG_OPTIONS
PKG_OPTIONS.seamonkey=dbus webrtc
PKG_OPTIONS.packagename=option1 option2 -disabledoption3
- pbulk.list, list of packages to build.
# specify a list of packages to build, one per line
# with a category prefix, as they are found in the
# pkgsrc tree.
wm/ctwm
editors/texmaker
print/zathura
converters/pandoc
graphics/sxiv
...
start a pkg_comp build
$ pkg_comp -c local auto
That's pretty much all you need. The local
configuration file, specified without the .conf extension, is expected to be found in the standard /usr/pkg/etc/pkg_comp directory.
The auto
command will tell pkg_comp to:
- optionally checkout or update a pkgsrc tree (if
UPDATE_SOURCE=true
);
- create the sandbox (using sandboxctl);
- bootstrap pkgsrc and pbulk;
- use pbulk to build the given packages;
- destroy the sandbox.
And is equivalent to:
$ pkg_comp -c local fetch
$ pkg_comp -c local sandbox-create
$ pkg_comp -c local bootstrap
$ pkg_comp -c local build <pbulk.list>
$ pkg_comp -c local sandbox-destroy
The stages can be invoked separately (e.g. you may just create/destroy the sandbox and build the packages in the middle).
The sandbox-shell
command will chroot you into the sandbox, this is useful if anything goes wrong with the build.
The -o UPDATE_SOURCE=false
can be passed interactively (as any other setting) to skip updating the pkgsrc tree.
If you'd like to restrict the set of packages to build during a manually-triggered build, provide those as additional arguments to auto
(i.e. 'auto pkgin'). This will override the contents of AUTOPACKAGES
(which was in turn derived from your pbulk.list file).
Nevertheless, you may you may also tell pkgcomp to just build individual packages or a series of packages:
$ pkg_comp -c local build package1 package2 package3 ...
installing packages
Any successfully built package will be stored in the directory set in PACKAGES
, usually /usr/pkgsrc/packages/All. The pkg_summary files are also going to be updated, hence the directory can be used without further changes by pkgin to install or upgrade the packages on your host.
Now, all you need to do to use them via
pkg_add(8) is:
$ PKG_PATH=file:///usr/pkgsrc/packages/All; export PKG_PATH
$ pkg_add pkgname
Or add the repository to repositories.conf in order to use it with pkgin.