• NetBSD
  • pkgin stats|lists return seems false

Hi

On my fresh install 9.2 RELEASE, on my laptop, I've installed some packages with pkgin tool, like nano mc htop xfce4 (and its extras, and their dependencies).

# pkgin stats
Local package database:
             Installed packages: 2
             Disk space occupied: 1704K

Remote package database(s):
             Number of repositories: 1
             Packages available: 24683
             Total size of packages: 38G

# pkgin list
pkg_install-20210410 Package managment and administration tools for pkgsrc
pkgin-20.12.1nb1       Apt / yum like tool for managing pkgsrc binary packages

I am surprised that it only returns two installed files. What about the others?
What did I not understand?

  • pin replied to this.

    CiotBSD and this will be certainly my last message, since it appears that some people are starting to get annoyed with my returns

    I don't know where you got this from.
    Please, just so I understand your system, where is pkgdb located?

    Inside /usr/pkg or inside /var/db/pkg?

      CiotBSD Yes, I can see the override flags but, after you installed some packages? What are the contents of /var/db? If there's a /var/db/pkg directory, what does it contain?

        pin What are the contents of /var/db?

        $ ls -al /var/db/
        total 92648
        drwxr-xr-x   9 root     wheel         512 Aug 26 21:02 .
        drwxr-xr-x  26 root     wheel         512 May 12 15:15 ..
        drwxr-xr-x   2 root     wheel         512 May 12 15:15 dhcpcd
        -rw-------   1 root     wheel         536 Aug 26 21:02 entropy-file
        -r--r--r--   1 root     wheel      392535 May 12 15:15 libc.tags
        -rw-r--r--   1 root     wheel           0 May 12 15:15 locate.database
        -rw-r--r--   1 root     wheel    46620672 Aug 26 12:12 man.db
        drwxr-xr-x   2 root     wheel         512 May 12 15:15 ns
        drwxr-xr-x   2 root     wheel         512 Aug 25 17:03 obsolete
        drwxr-xr-x   3 root     wheel         512 Aug 25 17:23 pkgin
        drwxr-xr-x   2 postfix  postfix       512 May 12 15:15 postfix
        -rw-r--r--   1 root     wheel      339325 Aug 20 19:18 services.cdb
        drwxr-xr-x   3 root     wheel         512 May 12 15:15 xdm
        drwxr-xr-x   2 root     wheel         512 Aug 26 21:02 xkb

        If there's a /var/db/pkg directory, what does it contain?
        As you can see, there is no such directory, but /var/db/pkgin!

        $ ls -al /var/db/pkgin/
        total 59648
        drwxr-xr-x  3 root  wheel       512 Aug 25 17:23 .
        drwxr-xr-x  9 root  wheel       512 Aug 26 21:02 ..
        drwxr-xr-x  2 root  wheel      5632 Aug 26 11:50 cache
        -rw-r--r--  1 root  wheel     43787 Aug 26 11:54 pkg_install-err.log
        -rw-r--r--  1 root  wheel  30416896 Aug 25 17:23 pkgin.db
        -rw-r--r--  1 root  wheel       276 Aug 20 17:21 sql.log

        ⇒ content of pkg_install-err.log: https://termbin.com/arzr

        ⇒ content of sql.log: https://termbin.com/qpfz

        Maybe those logs will be usefull?!

        • pin replied to this.

          CiotBSD Something is very wrong 🤔
          There're loads of these

          ...
          /usr/pkg/pkgdb: +�*.*�*..,�*: not found
          ---Aug 26 11:51:42: installing xfce4-wm-themes-4.10.0nb16...
          /usr/pkg/pkgdb: +�*.*�*..,�*: not found
          ...

          I wonder what could be the reason for it. Apparently you don't have a pkgdb at all...?!

            another weird thing:

            # pkgin remove nano
            no such installed package nano
            no packages to delete

            ???


            pin Apparently you don't have a pkgdb at all...?!

            # ls -al /usr/pkg/pkgdb/
            total 32
            drwxr-xr-x   5 root  wheel   512 Aug 23 16:40 .
            drwxr-xr-x  20 root  wheel   512 Aug 27 07:20 ..
            drwxr-xr-x  10 root  wheel  6656 Aug 23 08:27 pkg
            drwxr-xr-x   2 root  wheel   512 Aug 23 23:03 pkg_install-20210410
            -rw-r--r--   1 root  wheel  8192 Aug 23 16:40 pkgdb.byfile.db
            drwxr-xr-x   2 root  wheel   512 Aug 20 19:10 pkgin-20.12.1nb1

            apparently, it seems present!

            • pin replied to this.

              pin /usr/pkg/pkgdb: +�.�..,�: not found
              ---Aug 26 11:51:42: installing xfce4-wm-themes-4.10.0nb16...
              /usr/pkg/pkgdb: +�.�..,�: not found

              Again, I remember those random unicode chars printed to stderr only from a corrupted pkgdb. At the time, since the system phisically resided on a SD card, I was able to mount that on my daily driver and fix it manually.
              @CiotBSD what was that machine running before NetBSD? Were you experiencing any problem with it?
              Did you run into power losses while performing a package upgrade?
              Have you been mixing binary packages with those built from source?
              But even more importantly: Have you factually reinstalled the system as you mentioned in the other thread or are you still coping with that apparently very messed up situation?

                JuvenalUrbino only from a corrupted pkgdb.

                Yeah, his DB is corrupted for sure, but, the question is why?

                OP: can you boot using the install CD/USB stick and run a full fsck (fsck -f /dev/...) on the root FS?

                  rvp OP: can you boot using the install CD/USB stick and run a full fsck (fsck -f /dev/...) on the root FS?

                  I was about to come here and suggest the same. nice 🙂. @CiotBSD after that, you may want to enable journaling. I haven't had to fsck a drive in ages since discovering WAPBL.

                    CiotBSD apparently, it seems present!

                    Yes but, for some reason it contains only 2 packages and anything you're trying to install does not get registered in the database.

                      JuvenalUrbino what was that machine running before NetBSD

                      It's an oldier Dell M1330 First Gen.

                      JuvenalUrbino Were you experiencing any problem with it

                      no, never!

                      JuvenalUrbino Did you run into power losses while performing

                      no, always connected on electric network, because the battery is very old.

                      JuvenalUrbino Have you been mixing binary packages with those built from source

                      no. In first, I installed pkgin with pkg after set environment var $PKG_PATH, and after installed I use only pkgin!
                      (like as wroted on NetBSD Guide, and assume too, segun wrote on other post on this forum)

                      JuvenalUrbino Have you factually reinstalled the system as you mentioned in the other thread or are you still coping with that apparently very messed up situation?

                      Yes, even I flush/erase both slices with tool on the ad hoc menu, before reinstall!

                      JuvenalUrbino are you still coping with that apparently very messed up situation?

                      No, I copy all informations after reinstall; I'm not a lier, if I say I do something, I do something.
                      I am known to be serious ### and honest, and I feel very strongly about it.


                      rvp can you boot using the install CD/USB stick and run a full fsck (fsck -f /dev/...) on the root FS?

                      OK, I will do that! But later, because now, I'm really not well; yesterday I had the second shot of the Pfizer vaccine, and today it's like having the flu, stiffness, pain, headache, etc..


                      OK. I understand.

                      pin anything you're trying to install does not get registered in the database.

                      Yes, I think you understand the situation.

                      • rvp replied to this.

                        JuvenalUrbino I was about to come here and suggest the same.

                        Ya. A clean fsck would point to a user-space problem (in which case I'll have OP do an mtree check); otherwise, a HW/kernel issue.

                        CiotBSD I'm not a lier, if I say I do something, I do something.

                        Relax mate, we're just trying to collect all possible data here. Remember, we're pretty much in the dark about your setup at present.

                          rvp OP

                          What is this means?

                          rvp run a full fsck (fsck -f /dev/...) on the root FS?

                          As you can see on this photo, results no problem:

                          rvp Remember, we're pretty much in the dark about your setup at present.

                          Sure. And sorry. Very basic, just two slices, one for /, other for swap; just change the layout to FR, and install OS, and finishing with set parameters for root, system and my personal account.

                          $ cat /etc/fstab 
                          # NetBSD /etc/fstab
                          # See /usr/share/examples/fstab/ for more examples.
                          /dev/wd0a               /       ffs     rw               1 1
                          /dev/wd0b               none    swap    sw,dp            0 0
                          kernfs          /kern   kernfs  rw
                          ptyfs           /dev/pts        ptyfs   rw
                          procfs          /proc   procfs  rw
                          /dev/cd0a               /cdrom  cd9660  ro,noauto
                          tmpfs           /var/shm        tmpfs   rw,-m1777,-sram%25

                          I'm going to add log option into fstab file for / and reboot.

                          # grep wd0a /etc/fstab 
                          /dev/wd0a               /       ffs     log,noatime,rw           1 1
                          
                          # mount -u -o log,noatime,rw /

                          And, now? what can I do?

                            CiotBSD And, now? what can I do?

                            1. Can you post the output of these commands:

                              fdisk -vv wd0
                              disklabel wd0
                              gpt show wd0
                              df -h
                              ls -ld /tmp /var/tmp
                              cat /etc/sysctl.conf
                            2. Post also the output of this command:

                              cat /etc/mtree/set.[bcf-x]* | mtree -e -p /
                            3. From your post, above:

                              # ls -al /usr/pkg/pkgdb/
                              total 32
                              drwxr-xr-x   5 root  wheel   512 Aug 23 16:40 .
                              drwxr-xr-x  20 root  wheel   512 Aug 27 07:20 ..
                              drwxr-xr-x  10 root  wheel  6656 Aug 23 08:27 pkg               <---HERE
                              drwxr-xr-x   2 root  wheel   512 Aug 23 23:03 pkg_install-20210410
                              -rw-r--r--   1 root  wheel  8192 Aug 23 16:40 pkgdb.byfile.db
                              drwxr-xr-x   2 root  wheel   512 Aug 20 19:10 pkgin-20.12.1

                              This does not look right, there's an extra pkg directory there. What does that contain? And, the DB size also looks too small to be correct. (Since only 2 packages are correctly registered, 8K is correct.)

                            And, OP = Original Poster (person who started a thread on a forum)

                            CiotBSD The first set of commands look OK, except for the df output:

                            # df -h
                            Filesystem         Size       Used      Avail %Cap Mounted on
                            /dev/wd0a          913G       4.0G       863G   0% /

                            4G is used, but, that's not correct for the no. of packages seen in /usr/pkg/pkgdb or in /usr/pkg/pkgdb/pkg. With so few packages registered only about 1.5G should've been used. Have you installed the source code too?

                            # cat /etc/mtree/set.[bcf-x]* | mtree -e -p /
                            etc/X11/xdm/Xsetup_0: 
                                    size (88, 101)
                                    sha256 (0xe928997897a6a7778cd9a04f2602a28526a1e1bef569c92cd7b85cb5eeff34e4, 0xaf7f193051c7b843cd1beed6cf23b48e3d4a500795a24c0af35e82a78f209fc1)
                            usr/lib/i386/libkadm5srv.so.15.0: 
                                    sha256 (0xf2f4000bb322ac102e8c1f6f411f2b9d4200fd2675739e91867c2c13b1644214, 0xc347f1b233a2e1a84d41d2143f414716bfe4a159b079914306d36e36a77dd56f)
                            ./var/run/named missing

                            The checksum mismatch for /usr/lib/i386/libkadm5srv.so.15.0 I see on the latest 9.2_STABLE also but should be reported. The other 2 issues reported are OK.

                            Alright, let's try again since some screw-up has certainly happened to create a /usr/pkg/pkgdb/pkg.
                            Exit Xorg, then as root on the system console:

                            1. Bring system back to newly-installed state:

                              rm -rf /etc/mk.conf /etc/pkg_install.conf /var/db/pkgin/ /var/db/pkg/ /usr/pkg/
                              
                              df -h       # This should report ~1.5G if `/` used (without source).
                            2. Run the pkgin boot-strap script (please do not type). Post complete output (use script(1)):

                              /tmp/pkg-setup.sh
                            3. Do a check of the 2 initial packages:

                              pkg_admin check
                            4. If check passes, install your packages one-by-one, doing a pkg_admin check after each:

                              pkgin -y in bash
                              pkg_admin check
                              
                              pkgin -y in mc
                              pkg_admin check
                              
                              pkgin -y in firefox
                              pkg_admin check
                              
                              pkgin -y in xfce4
                              pkg_admin check
                              
                              ...etc

                              Stop and report if there are any errors.

                              Hi all.

                              OK, let's go continue! :p

                              rvp Have you installed the source code too?

                              NO!
                              as I wroted previously, I installed in a very basic way without doing anything else:

                              CiotBSD Very basic, just two slices, one for /, other for swap; just change the layout to FR, and install OS, and finishing with set parameters for root, system and my personal account.


                              1.

                              localhost# rm -rf /etc/mk.conf /etc/pkg_install.conf /var/db/pkgin/ /usr/pkg/
                              localhost# mkdir /var/db/pkg
                              localhost# df -h
                              Filesystem         Size       Used      Avail %Cap Mounted on
                              /dev/wd0a          913G       2.5G       865G   0% /
                              kernfs             1.0K       1.0K         0B 100% /kern
                              ptyfs              1.0K       1.0K         0B 100% /dev/pts
                              procfs             4.0K       4.0K         0B 100% /proc
                              tmpfs              1.0G         0B       1.0G   0% /var/shm

                              2.5Go on /!
                              I do not why the reason!?

                              2.

                              # script 
                              Script started, output file is typescript
                              localhost# /tmp/pkgin-setup.sh 
                              + D=PKG_DBDIR=/usr/pkg/pkgdb
                              + uname -m
                              + M=amd64
                              + uname -r
                              + R=9.2
                              + R=9.2
                              + test '!' -r /etc/mk.conf
                              + test '!' -r /etc/pkg_install.conf
                              + test '!' -d /usr/pkg/etc
                              + echo PKG_DBDIR=/usr/pkg/pkgdb >>/etc/mk.conf
                              + echo PKG_DBDIR=/usr/pkg/pkgdb >>/etc/pkg_install.conf
                              + mkdir -p /usr/pkg/etc
                              + echo PKG_DBDIR=/usr/pkg/pkgdb >>/usr/pkg/etc/pkg_install.conf
                              + PKG_PATH='https://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/amd64/9.2/All'
                              + export PKG_PATH
                              + pkg_add pkgin
                              pkg_add: Warning: package `pkgin-20.12.1nb1' was built for a platform:
                              pkg_add: NetBSD/x86_64 9.0 (pkg) vs. NetBSD/x86_64 9.2 (this host)
                              pkg_add: Warning: package `pkg_install-20210410' was built for a platform:
                              pkg_add: NetBSD/x86_64 9.0 (pkg) vs. NetBSD/x86_64 9.2 (this host)
                              pkgin-20.12.1nb1: copying /usr/pkg/share/examples/pkgin/repositories.conf.example to /usr/pkg/etc/pkgin/repositories.conf
                              + pkgin update
                              processing remote summary (https://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/x86_64/9.0/All)...
                              pkg_summary.bz2                                                                                                                                             100% 3629KB 362.9KB/s   00:10    
                              + rmdir /var/db/pkg
                              rmdir: /var/db/pkg: Directory not empty
                              + echo '/tmp/pkgin-setup.sh: procedure failed.'
                              /tmp/pkgin-setup.sh: procedure failed.
                              + exit 1
                              localhost# 

                              The process fails!
                              Ohh, ok, it fails to delete /var/db/pkg.

                              3.

                              localhost# pkg_admin check
                              ..
                              Checked 21 files from 2 packages.
                              Done.
                              1. Do I continue with install others mentionned packages?!
                              • rvp replied to this.

                                CiotBSD 2.5Go on /!
                                I do not why the reason!?

                                Could be larger block sizes being used for your very large 900G partition. We'll see:

                                ls -ilR / | bzip2 -9 > /tmp/ls.bz2

                                Email or post ls.bz2somewhere.

                                CiotBSD The process fails!
                                Ohh, ok, it fails to delete /var/db/pkg.

                                This shouldn't happen. /var/db/pkg should be empty at this point. What's the output of:

                                ls -la /var/db/pkg /usr/pkg/pkgin

                                  Hi, all.

                                  rvp ls -ilR / | bzip2 -9 > /tmp/ls.bz2

                                  # ls -ilR / | bzip2 -9 > /tmp/ls.bz2
                                  
                                  ls: fd/6: No such file or directory
                                  #

                                  Here the archive: http://0x0.st/-vXi.bz2

                                  rvp ls -la /var/db/pkg /usr/pkg/pkgin

                                  # ls -la /var/db/pkg /usr/pkg/pkgin
                                  ls: /usr/pkg/pkgin: No such file or directory
                                  /var/db/pkg:
                                  total 24
                                  drwxr-xr-x   4 root  wheel   512 Aug 30 15:59 .
                                  drwxr-xr-x  11 root  wheel   512 Aug 31 09:14 ..
                                  drwxr-xr-x   2 root  wheel   512 Aug 30 15:59 pkg_install-20210410
                                  -rw-r--r--   1 root  wheel  8192 Aug 30 15:59 pkgdb.byfile.db
                                  drwxr-xr-x   2 root  wheel   512 Aug 30 15:59 pkgin-20.12.1nb1
                                  • rvp replied to this.