I solved it, by substituting in /etc/man.conf the priority precedence to first look into /usr/pkg instead of /usr/X11R7.
Basically I changed this line:
_default /usr/{share,X11R7,pkg,local}/man/
to:
_default /usr/{local,pkg,share,X11R7,X11R6}/man/
It fixed the problem, but still it got me wondering, is there a way to remove stuff installed with the base system on netbsd?
I mean how do I keep track of packages which are pre-installed if they do not come up in pkgin ?