The Cloistered Monkey

Fixing NVidia With Wayland on Ubuntu 24.04 (Noble Numbat)

Cloistered Monkey — Sat, 03 May 2025 16:48:54 GMT

The Problem

Switching from X11 to Wayland (on Ubuntu 24.04 (Noble Numbat)) initially prevented launching the Settings so I went on a hunt to fix it (and here are my notes).

The Problems

WayDroid and X11

I was poking around and saw that there was a project to run Android on PCs (among other things) called Waydroid so I installed it, but whenever I tried to enter the Waydroid shell

sudo waydroid shell

I got an error saying that the waydroid session is stopped. This isn't a post about waydroid so I'll skip the troubleshooting I did, but ultimately I found an AskUbuntu post from someone who ran into the same problem and found from one of the comments that:

it turns out that waydroid is not compatible with X11.

Maybe it's called Waydroid because it needs Wayland? Anyway I checked if I was using Wayland:

echo $XDG_SESSION_TYPE

Which didn't show that I was (I can't remember if it showed X11 or was just empty, but it wasn't anything like "Wayland" anyway). It turned out I had chosen "Ubuntu" at the login screen, not "Ubuntu Wayland" so I logged out, switched to Wayland and then logged in. This seemed to fix the initial WayDroid problem (I got the GUI running but I still haven't finished the installation) but then another problem came up.

The Next Problem - Sleep and Logging out

Having switched to Wayland I decided I needed a break. While I was away my computer went to the lock screen, as it usually does, but when I opened it back up I found that instead of just locking the screen it had actually logged me out, killing all the applications I left open and closing an encrypted mount point I had open. More searching ensued.

Once again, someone posted a question on AskUbuntu that seemed to be about the same problem, the suggested answer being to disable the Screen Lock, which didn't really seem to be a good answer, but it was an answer, which was one more than I started with so I decided to give it a try. But then the next problem appeared.

No Settings

When I tried to launch the settings an icon came up in the launcher like normal but the settings wouldn't come up. Okay. I'm skipping some of the intermediate searches but eventually I tried running the settings in the terminal to look for the errors.

gnome-control-center

This gave an error which included "failed to create dri2 screen" so I went on a search again and found another AskUbuntu post with a suggested answer - install a package:

sudo apt install libnvidia-egl-wayland1

After logging out and logging back in the settings worked and the lock screen no longer logged me out so this seems to have been the fix (for these problems, there's still some weird things going on but nothing I can't live with yet.)

Conclusion

There's an open bug on Launchpad (Gnome apps segfault in Nvidia (535) Wayland sessions on Noble, but 550 works) which looks like it might be related to the problem I ran into. It says that upgrading the NVidia driver to 550 fixes the Gnome Settings problem but the upgrade then introduces some other errors so installing libnvidia-egl-wayland1 looks like the right fix. The bug was opened on April 26, 2024 so I would imagine it would effect more people, making this a high priority. Perhaps Wayland isn't widely used, at least not among nvidia users. I'm typing this in emacs and portions of the buffer are messed up, making it dangerous to use (insofar as I can't tell if the text actually exists in the document in the same place where it appears on the screen) so I'll probably go back to X11 until they get this stuff worked out, but at least it's working a little better than it was this morning.

ProtonPass: Finding the Suggested Password

Cloistered Monkey — Sat, 19 Apr 2025 22:57:25 GMT

What This Is About

I'm using Proton Pass as one of my password managers and noticed today that it thought my Hoopla password was weak so I decided to change it. While I was changing my password ProtonPass stepped in and gave me a suggested password to use so I figured I'd try it out. Unfortunately their implementation is unintuitive to me and I spent a little while thinking I had just locked myself out of my account and was going to have to go through a password reset. Hoopla's password reset process isn't particularly hard, but it seemed like a good idea to figure out what was going on before I locked myself out of a more critical site, so here's my notes for getting the password from ProtonPass.

The Start Of the Trouble

When I went to change my password on Hoopla ProtonPass popped up a prompt giving me a new random password like in the screenshot below.

I usually ignore it but for some reason I tried it out this time, clicking on the box to enter the password and then saving it (I took this screenshot later so that isn't my password, in case you're wondering). I confirmed that my old password no longer worked, but when I tried to use Proton Pass to fill in the new password it turned out that it hadn't updated its record with the new one it gave me so it appeared that I had changed my password and then thrown the new one away.

This seemed like an illogical thing to do so I did a little search on the web and found this response to a Reddit post confirming that this is, in fact, the way Proton Pass works, but that you can get the new password after the fact from the Firefox extension.

Getting It Out Of Proton Pass

The Firefox Extension

I'm using Firefox (on Ubuntu) and the ProtonPass extension, so I clicked on the little purple diamond to bring it up.

Add New Item

This is what the extension showed (with some of my information blocked out). To get to the password you have to (rather unintuitively) click on the purple "+" symbol next to the search box which is the "Add New Item" button.

This brings up a menu, the last item of which is the "Password" option that we want.

Password Generator

Clicking on "Password" brings up a dialog that generates new passwords for you (here it's offering us Joylessly9-Disjoin8-Dynasty4-Delegator0-Mobilize5 - a supposedly memorable one for us to use). At the bottom of that screen, underneath the "Advanced Options" button, is the "Password history" button that we want.

Password History

This shows us the passwords we've generated in the last two weeks (I never generated one before this case so there's only the one entry). Oddly, the post on Reddit from the "ProtonSupportTeam" says that they are deleted after 1 day, not two weeks. It also says that they are working on "improving the auto-save behavior," but the post is two years old (as of this writing) so I guess they decided that lengthening the time to deletion was the improvement we needed.

Clicking on the icon for "hoopladigital.com" copied the password I generated for that site to the clipboard which I then used to update the Proton Pass entry for Hoopla so that I could get back into the site.

Conclusion

Updating the password isn't fantastically difficult, once you know what you have to do, but I don't imagine I'd have figured it out for myself. Hopefully even if I forget how to do it next time I'll at least remember that I wrote down what to do.

Noble Numbat and the Broken Virtual Environments

Cloistered Monkey — Mon, 27 May 2024 23:21:26 GMT

What is this about?

These are notes I made when I was fixing some errors that occurred after updating Ubuntu to 24.04. It's been over a year so I don't really remember writing this, but, here it is. The fixes are specific to virtualfish, pipx, and nikola.

What Happened

I updated Ubuntu
Something raised an obscure error when I changed into one of my git repositories
It turned out that the upgrade broke the python virtual environment
Re-setting-up using the newer version of python that came with the upgrade fixed things.

Short Version

Upgrading to Ubuntu 24.04 made python changes that broke all my virtual enviroments
Reinstalled pipx using apt
vf ls --details shows the python versions for all (virtualfish created) virtual environments (and which are broken)
vf upgrade --rebuild will upgrade the currently active virtual environment (may take more than one run to get past errors)
pipx reinstall-all fixed the missing nikola and pygmentize errors (and probably others)
fixing pygments doesn't fix the errors in posts that were alread built so do a nikola clean and nikola build
pip install setuptools --upgrade in the environment re-setup distutils which was removed from python 3.12 (which Ubuntu Installed)

The First Hint (pipx)

I don't remember exactly what cued me in to what was going wrong, but somewhere in trying to figure out what was wrong I found that running legit after updating to Ubuntu 24.04 returned an error saying that legit wasn't found. It took a while to figure out where legit was supposed to be coming from, but it turned out I had installed it with pipx and had to re-install it to get it working.

I don't remember exactly what I did anymore, but based on the command history it looks like:

Re-install pipx (I purged the original installation that had been installed using pip and instead used apt)
Re-install legit using the updated pipx

So, was pipx the problem? Well, partly.

The Broken Virtual Environments

The next problem came when I tried to build the HTML for this site.

nikola build

Traceback (most recent call last):
  File "/home/hades/.virtualenvs/necromuralist.github.io/bin/nikola", line 5, in <module>
    from nikola.__main__ import main
ModuleNotFoundError: No module named 'nikola'

For some reason the nikola executable was missing. Since I was in a virtual environment I tried to re-install nikola but using pip gave another error.

pip --version

Traceback (most recent call last):
  File "/home/hades/.virtualenvs/necromuralist.github.io/bin/pip", line 5, in <module>
    from pip._internal.cli.main import main
ModuleNotFoundError: No module named 'pip'

So, not only was nikola missing but so was pip, which I thought was a part of the python installation… I looked at what was in the virtualenvironment -

ls ~/.virtualenvs/necromuralist.github.io/lib

And saw that it had this python version:

python3.11/

But when I checked the version of the python I was using it gave this.

python --version

Python 3.12.3

So it looked like the ubuntu update created a bit of a mismatch. It turns out that virtualfish has a command to list what virtual environments you have set up (vf ls) and if you pass it the --details= option it will show you the python version the environment is using.

vf ls --details

It listed all my environments but in particular it showed this for the environment for this site's repository.

necromuralist.github.io (broken)

virtualfish also has a command to upgrade the currently active environment so I tried running it.

Running this with the necromuralist environment active:

vf upgrade --rebuild

Gave this error.

Collecting PyYAML==6.0
  Downloading PyYAML-6.0.tar.gz (124 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 125.0/125.0 kB 3.5 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [54 lines of output]
      running egg_info
      writing lib/PyYAML.egg-info/PKG-INFO
      writing dependency_links to lib/PyYAML.egg-info/dependency_links.txt
      writing top-level names to lib/PyYAML.egg-info/top_level.txt
      Traceback (most recent call last):
        File "/home/hades/.virtualenvs/necromuralist.github.io/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/home/hades/.virtualenvs/necromuralist.github.io/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/hades/.virtualenvs/necromuralist.github.io/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1ai8jd66/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1ai8jd66/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "/tmp/pip-build-env-1ai8jd66/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 288, in <module>
        File "/tmp/pip-build-env-1ai8jd66/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 103, in setup
          return distutils.core.setup(**attrs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1ai8jd66/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 184, in setup
          return run_commands(dist)
                 ^^^^^^^^^^^^^^^^^^
        File "/tmp/pip-build-env-1ai8jd66/overlay/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 200, in run_commands
          dist.run_commands()
        File "/tmp/pip-build-env-1ai8jd66/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
          self.run_command(cmd)
        File "/tmp/pip-build-env-1ai8jd66/overlay/lib/python3.12/site-packages/setuptools/dist.py", line 968, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-1ai8jd66/overlay/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-1ai8jd66/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 321, in run
          self.find_sources()
        File "/tmp/pip-build-env-1ai8jd66/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 329, in find_sources
          mm.run()
        File "/tmp/pip-build-env-1ai8jd66/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 550, in run
          self.add_defaults()
        File "/tmp/pip-build-env-1ai8jd66/overlay/lib/python3.12/site-packages/setuptools/command/egg_info.py", line 588, in add_defaults
          sdist.add_defaults(self)
        File "/tmp/pip-build-env-1ai8jd66/overlay/lib/python3.12/site-packages/setuptools/command/sdist.py", line 102, in add_defaults
          super().add_defaults()
        File "/tmp/pip-build-env-1ai8jd66/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 250, in add_defaults
          self._add_defaults_ext()
        File "/tmp/pip-build-env-1ai8jd66/overlay/lib/python3.12/site-packages/setuptools/_distutils/command/sdist.py", line 335, in _add_defaults_ext
          self.filelist.extend(build_ext.get_source_files())
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "<string>", line 204, in get_source_files
        File "/tmp/pip-build-env-1ai8jd66/overlay/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 107, in __getattr__
          raise AttributeError(attr)
      AttributeError: cython_sources
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.

Which looked like I was in trouble, but it turned out that re-running the upgrade fixed it. One of those magic fixes that would be scary if this was a critical site, I suppose, but which is instead, just another bit of python mystery.

Then a Nikola Error

So, with the virtual environment updated to the current python, I waded back into building the site.

nikola build

First this produced a nikola not found error (updating the virtual environment apparently doesn't fix everything). But since the environment itself was fixed pip was now working so I installed it again, but then building the site produced a different error.

DependencyError - taskid:render_posts:cache/posts/2ac-updating-a-nikola-shortcode-plugin.html  
ERROR: Task 'render_posts:cache/posts/2ac-updating-a-nikola-shortcode-plugin.html' saving success: Dependent file '/home/hades/.virtualenvs/necromuralist.github.io/lib/python3.11/site-packa
ges/nikola/data/shortcodes/mako/raw.tmpl' does not exist.

Re-reinstalling nikola and re-running the build made the error go away, but in the output I noticed some disturbing messages that looked like these.

.  render_pages:output/posts/destroying-tags-with-beautiful-soup/index.html
line 135 column 56 - Error: <module> is not recognized!
line 144 column 56 - Error: <module> is not recognized!
line 154 column 56 - Error: <module> is not recognized!
This document has errors that must be fixed before
using HTML Tidy to generate a tidied up version.

There were are a lot of these errors (which didn't cause nikola to quit so I wouldn't have seen them if I wasn't watching the build messages).

This turned out to be because nikola was encountering errors, trapping them and not failing and then embedding the python error messages into the HTML (and the error messages were causing HTML tidy to fail and then raise a different error message).

This is an example of an error message embedded in the HTML nikola created:

Traceback (most recent call last):
  File "/home/hades/.local/bin/pygmentize", line 5, in <module>
    from pygments.cmdline import main
ModuleNotFoundError: No module named 'pygments'

The cause for this one was a little more mysterious, but it turned out that I had installed pygmentize using pipx so it needed to be updated separately. I decided to use pipx reinstall-all to get everything updated.

Unfortunately, there were a lot of posts with the pygmentize errors, and since they aren't registered as errors by nikola, just building the site didn't fix them so I wiped it first and then re-built it.

nikola clean
nikola build

And since there weren't anymore errors I decided to cross my fingers and move on.

Note to future self: This happened again with the update to Ubuntu 25.04 (Plucky Puffin) and the error messages from not cleaning the old build were confusing/misleading enough to cause me to waste time troubleshooting them. A clean build is a really good idea.

An Ape-Iron Problem (no distutils)

The vf ls --details command I used to check on the virtual environment for this site also showed that all my other virtual environments were broken so I decided to try and fix the Ape-Iron repository while it was fresh on my mind. First I rebuilt the environment.

vf upgrade --rebuild Ape-Iron

And of course, it gave me errors.

Collecting numpy==1.24.3
  Downloading numpy-1.24.3.tar.gz (10.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.9/10.9 MB 7.7 MB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
ERROR: Exception:
Traceback (most recent call last):
  File "/home/hades/.virtualenvs/Ape-Iron/lib/python3.12/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper

File "/tmp/pip-build-env-f7h7lvcc/overlay/lib/python3.12/site-packages/setuptools/__init__.py", line 10, in <module>
    import distutils.core
ModuleNotFoundError: No module named 'distutils'

According to this answer to a question on StackOverflow distutils was removed from python 3.12, which is the default with noble-numbat. The answer suggests installing setuptools.

So, I tried that.

pip install setuptools --upgrade

Amazingly, the upgrade of Ape-Iron seemed to work.

vf upgrade --rebuild Ape-Iron

At this point I think I just called it a day.

Fish Links

Cloistered Monkey — Sat, 17 Jun 2023 21:30:02 GMT

awsm.fish: a github readme with links to fish-related things

Apt-Fast In Sparky Linux

Cloistered Monkey — Fri, 16 Jun 2023 20:15:45 GMT

Background

I have an old eeePC netbook that I thought I'd revive by loading Sparky Linux onto it. One of the things I set up on it is apt-fast, which the README on the github repository describes like this:

apt-fast is a shellscript wrapper for apt-get and aptitude that can drastically improve apt download times by downloading packages in parallel, with multiple connections per package.

I've used it for a while on ubuntu but Sparky Linux didn't have it in the repositories. The apt-fast documentation has instructions for installing it on debian (and derivatives) and since Sparky Linux is based on debian (the current version, SparkyLinux 6.7 (Po-Tolo) is based on debian bullseye) I decided to try that. Ultimately I got it working, but as is often the case, it wasn't quite so straightforward as I would like for it to have been.

Unusual Ingredients List:

SparkyLinux 6.7 (Po-Tolo)
fish shell

The Intructions

Although the PPA system is built for Ubuntu, the recommendation from apt-fast is to use it with debian-based systems too (apt-fast is just a shell-script that runs aria2 and apt (or apt-get, etc.) so it's not like there's a lot of dependencies that might conflict). This is what they say to do.

Create An Apt Entry

First I created a file for the sources at /etc/apt/sources.list.d/apt-fast.list and put these lines in it.

deb http://ppa.launchpad.net/apt-fast/stable/ubuntu bionic main 
deb-src http://ppa.launchpad.net/apt-fast/stable/ubuntu bionic main

Bionic came out in 2018 so they maybe haven't updated the instructions in a while.

Add the Keyring and Install

Once the file was in place I ran the commands.

apt-key adv --keyserver keyserver.ubuntu.com --recv-keys A2166B8DE8BDC3367D1901C11EE2FF37CA8DA16B
apt-get update
apt-get install apt-fast

The first output I saw was a warning:

Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).

This is something I've seen on ubuntu as well so fixing it seemed like a useful thing to do, especially since at the end of the regular output I got an error.

Between the warning and the final error there was the usual output that I've seen:

Executing: /tmp/apt-key-gpghome.YL04bWmGAF/gpg.1.sh --keyserver keyserver.ubuntu.com --recv-keys A2166B8DE8BDC3367D1901C11EE2FF37CA8DA16B
gpg: key 1EE2FF37CA8DA16B: public key "Launchpad PPA for apt-fast" imported
gpg: Total number processed: 1
gpg:               imported: 1

Then came this, the error I mentioned, which is not what I usually see:

gpg: no writable keyring found: Not found
gpg: error reading '[stdin]': General error
gpg: import from '[stdin]' failed: General error
gpg: Total number processed: 0

I don't know if this is a debian problem or a sparky linux problem, but since you're not supposed to be using this method anyway, I went looking for a different solution.

The apt-key Solution

The Start of the Solution

The first part of the solution was pointed to by this Stack Overflow Answer. There was a problem with the answer, though, in that the person asking the question was using a URL that pointed to a gpg file, so the answers all assumed you could download it with curl, wget, etc. (all the answers that I could understand, anyway). So now that I had the answer I had a new problem - how do you get the file from the keyserver?

Getting the GPG File

Once again to the web, this time this answer from SuperUser seemed to work.

First, I made a temporary directory and pointed GNUPGHOME to it so that I wasn't adding anything to my actual gpg setup (this is fish-shell syntax).

set -x GNUPGHOME $(mktemp -d)

echo $GNUPGHOME showed that this created a file at /tmp/tmp.dUDUEgFQ0x (but I didn't actually need to know that, I'm just mentioning it).

Taking the --recv-keys argument from the instructions above (apt-key adv --keyserver keyserver.ubuntu.com --recv-keys A2166B8DE8BDC3367D1901C11EE2FF37CA8DA16B) I added the key.

gpg --keyserver keyserver.ubuntu.com --recv-keys A2166B8DE8BDC3367D1901C11EE2FF37CA8DA16B

Next I made a place to put the gpg file.

sudo mkdir /etc/apt/keyrings

Then I output the file in my home directory (the GNUPGHOME environment variable is only available to my user, so I put the file somewhere that I didn't need to be root, i.e. my home directory).

gpg -o A2166B8DE8BDC3367D1901C11EE2FF37CA8DA16B.gpg --export A2166B8DE8BDC3367D1901C11EE2FF37CA8DA16B

Then I moved the file into the directory I created for it.

sudo mv A2166B8DE8BDC3367D1901C11EE2FF37CA8DA16B.gpg /etc/apt/keyrings/

The SuperUser answer I linked to used gpg -ao but the a option makes it an "armored" file, and part of the Stack Overflow answer for setting up the key is about de-armoring it so I just left that option out.

Now Back to Setting it Up

So now that we have the keyring we need to edit the /etc/apt/sources.list.d/apt-fast.list file that we created at the beginning of this.

This is what I started with.

deb http://ppa.launchpad.net/apt-fast/stable/ubuntu bionic main 
deb-src http://ppa.launchpad.net/apt-fast/stable/ubuntu bionic main

And I changed it to refer to the gpg file that I created.

deb [signed-by=/etc/apt/keyrings/A2166B8DE8BDC3367D1901C11EE2FF37CA8DA16B.gpg] http://ppa.launchpad.net/apt-fast/stable/ubuntu bionic main
deb-src  [signed-by=/etc/apt/keyrings/A2166B8DE8BDC3367D1901C11EE2FF37CA8DA16B.gpg]  http://ppa.launchpad.net/apt-fast/stable/ubuntu bionic main

Update and Install

So then I updated apt and installed it.

sudo apt update
sudo apt install apt-fast

And it worked.

And Now, Another Problem

This got me to a working apt-fast installation but the fact that I was using bionic seemed off to me so I decided to update the apt-fast.list. Under the instructions for adding the PPA is this note.

Note that the PPA version bionic might need to be updated with the recent Ubuntu LTS codename to stay up-to-date.

So I went and looked up the Ubuntu Release Cycle and saw that "jammy" is the most recent version so I updated the apt-fast.list file to match.

deb [signed-by=/etc/apt/keyrings/A2166B8DE8BDC3367D1901C11EE2FF37CA8DA16B.gpg] http://ppa.launchpad.net/apt-fast/stable/ubuntu jammy main
deb-src  [signed-by=/etc/apt/keyrings/A2166B8DE8BDC3367D1901C11EE2FF37CA8DA16B.gpg]  http://ppa.launchpad.net/apt-fast/stable/ubuntu jammy main

And then I installed the newer version.

sudo apt update
sudo apt install apt-fast

And I got a nice long stack-trace and error message at the bottom of which was this:

dpkg-deb: error: archive '/var/cache/apt/archives/apt-fast_1.9.12-1~ubuntu22.04.1_all.deb' uses unknown compression for member 'control.tar.zst', giving up
dpkg: error processing archive /var/cache/apt/archives/apt-fast_1.9.12-1~ubuntu22.04.1_all.deb (--unpack):
 dpkg-deb --control subprocess returned error exit status 2
Errors were encountered while processing:
 /var/cache/apt/archives/apt-fast_1.9.12-1~ubuntu22.04.1_all.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

The part of it that seemed like it might matter the most was the fragment:

uses unknown compression for member 'control.tar.zst', giving up~

What is a zst file? According to wikipedia it's a "Zstandard" file and Debian and Ubuntu added support for using it to compress deb packages back in 2018. There is a package listed in apt called zstd that it says supports zst compression so I installed it but the error remained.

Once again, someone ran into this and asked about it on Stack Exchange. One of the answers said:

Debian’s dpkg package didn’t support zstd compression prior to version 1.21.18. Support was added just in time for Debian 12.

Since the SparkyLinux install is based on Debian 11 that seemed like it might be the problem. I checked the dpkg-version and got back:

Debian 'dpkg' package management program version 1.20.12 (i386).

So that seemed like the likely culprit. The ubuntu release dates page noted that there was a LTS version between "bionic" and "jammy" called "focal" so I edited the apt-fast.list file again, replacing "jammy" with "focal" and re-ran the installation and so far… it works.

What Have We Learned Today Children?

Further down in the installation instructions it says that you can just download the files and install them along with the aria2 package, so going through this whole thing was kind of unnecessary, but getting around the apt-key problem was something that I'd wondered about before, so it might be useful in the future, if PPA creators keep using it and they don't come up with an automatic fix for it.

I guess the main thing I learned is that I should have read to the end of the instructions and picked the easy way out instead of trying to force the old familiar way to work.

Links Collected

FWDekker. Answer to “Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead” [Internet]. Stack Overflow. 2022 [cited 2023 Jun 16]. Available from: https://stackoverflow.com/a/71384057
Kitt S. Answer to “zst compression not supported by apt/dpkg” [Internet]. Unix & Linux Stack Exchange. 2021 [cited 2023 Jun 16]. Available from: https://unix.stackexchange.com/a/669008
fuzzydrawrings. Answer to “gpg: only download a key from a keyserver” [Internet]. Super User. 2021 [cited 2023 Jun 16]. Available from: https://superuser.com/a/1643115
ilikenwf/apt-fast: apt-fast: A shellscript wrapper for apt that speeds up downloading of packages. [Internet]. [cited 2023 Jun 16]. Available from: https://github.com/ilikenwf/apt-fast
zstd. In: Wikipedia [Internet]. 2023 [cited 2023 Jun 16]. Available from: https://en.wikipedia.org/w/index.php?title=Zstd&oldid=1157564498

Fish, Mocha, Chai - A Local Global Installation In Ubuntu

Cloistered Monkey — Thu, 08 Jun 2023 23:29:32 GMT

What This Is About

I've been getting back into p5.js lately and thought that I should add some testing so I went to their site and found that they had a tutorial page called "Unit Testing and Test Driven Development" which I decided to follow along with to get re-acquainted with testing javascript but then I ran into a problem running mocha, or more specifically, running mocha crashed because it couldn't find chai even though I'd followed the instructions to install it. So here goes what I did to fix it.

This is another case where you can basically find the answer online if you look at the right page - but there seems to be more pages with the unhelpful answers and I use the fish shell and ubuntu so it's a little different from the stuff I found that did work.

The Tutorial's Installation

This is how they tell you to install mocha and chai.

First, update npm (assuming you've already installed it somehow).

sudo npm install npm@latest -g

Then install mocha and chai using npm.

npm install mocha -g
npm install chai -g

This right here is actually where the trouble starts. If you try to install things globally, you need to run it as root, thus the use of sudo when updating npm. But their instructions on installing mocha and chai don't say to use sudo, which will result in a permission error, so did they forget to run it as root, or did they not mean to install it globally? I decided to re-run their instructions as root.

sudo npm install mocha -g
sudo npm install chai -g

This seemed to work, but when I ran mocha on the folder where I put the code given in the tutorial:

mocha color_unit_tests/

It gave me an error.

Error: Cannot find module 'chai'
Require stack:
- /home/hades/projects/ape-iron/p5tests/color_unit_tests/test.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1097:15)
    at Module._load (node:internal/modules/cjs/loader:942:27)
    at Module.require (node:internal/modules/cjs/loader:1163:19)
    at require (node:internal/modules/cjs/helpers:110:18)
    at Object.<anonymous> (/home/hades/projects/ape-iron/p5tests/color_unit_tests/test.js:5:16
)
    at Module._compile (node:internal/modules/cjs/loader:1276:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1330:10)
    at Module.load (node:internal/modules/cjs/loader:1139:32)
    at Module._load (node:internal/modules/cjs/loader:980:12)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:169:29)
    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)

So, maybe that wasn't the answer.

This Might Be the Wrong Way

I found a Stack Overflow question that described the exact problem I had, but one of the comments had this to say:

Mocha can be installed either locally or globally, but Chai can only be installed locally. Has to do with the way it is applied (i.e., to the specific app instance). – Steve Carey May 30, 2020 at 21:11

I don't know who "Steve Carey" is and whether what he is saying is true, but the chai installation instructions do tell you to install it locally, rather than globally, but when you do this for every project you end up with node_modules and package.json files all over the place. I suppose there's a reason for this, maybe to couple the version of chai you're using with the project, but I decided to try another way.

The Local Global

This answer on Stack Overflow describes how to install npm-packages into your home directory as your global directory. It assumes you're using bash, though, so I had to change it up a little bit.

Make a local package directory

First I made a local package directory.

mkdir ~/.npm-packages

Then I created a file called ~/.npmrc that had one line in it.

prefix = /home/hades/.npm-packages

With /home/hades/ being my home-directory.

Edit the Fish Configuration

At the bottom of the ~/.config/fish/config.fish file I added these lines.

set -x NPM_PACKAGES $HOME/.npm-packages

This is where npm will install stuff if you tell it to install files globally once we're done. The folder can be named anything, I imagine, but it will need to match what's in the .npmrc file.

When npm installs packages some of them will be executable commands (like mocha) and so I had to update the fish PATH.

fish_add_path $HOME/.npm-packages/bin

Although this will make mocha available, chai isn't an executable so you have to set the NODE_PATH variable so that node will no where to look for modules to import.

set --export NODE_PATH $NPM_PACKAGES/lib/node_modules

I was originally appending the current contents of NODE_PATH to the end, like you would with a regular path variable ($NPM_PACKAGES/lib/node_modules:$NODE_PATH) but for some reason this breaks something and the variable doesn't get set. Or at least it was always empty when I tried to run mocha. So the solution for me was to always clobber the entire path (the variable was empty before I started using it anyway).

And Now

Running the tests again:

mocha ../../ape-iron/p5tests/color_unit_tests/



these are my first tests for p5js
  ✔ should be a string
  ✔ should be equal to awesome


2 passing (5ms)

The path is different because I'm writing this post in a different repository, but, anyway, it looks like it works.

Sources

Drechsel E. Install NPM into home directory with distribution nodejs package (Ubuntu) [Internet]. Stack Overflow. 2019 [cited 2023 Jun 8]. Available from: https://stackoverflow.com/q/10081293
Unit Testing and Test Driven Development | p5.js [Internet]. [cited 2023 Jun 8]. Available from: https://p5js.org/learn/tdd.html

Ubuntu 22.04, Python 3.11 and the "emacs-jupyter Symbol's variable is void" Error

Cloistered Monkey — Thu, 25 May 2023 23:12:27 GMT

What This Is About

Ubuntu 22.04 no longer let's you install python packages globally using pip by default (you can override it but they warn you not to). This has caused a cascade of broken parts on my system, since I use python so much. This particular case started with me trying to start the jupyter kernel so that I could run some python code in org-mode and getting (what looked like) an error and fixing it ended up uncovering the fact that working with the new policy for pip broke my emacs setup a little too, so this is a dump of how I got it back up and running again. I recorded it as I was fixing things so there might be a better way, but this is the first pass I took.

The Jupyter Kernel Warning

This is what happened when I tried to start the jupyter kernel.

(Ape-Iron) hades@erebus ~> jupyter kernel
[KernelApp] Starting kernel 'python3'
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this val
idation.
[KernelApp] Connection file: /home/hades/.local/share/jupyter/runtime/kernel-a57a8231-bfea-468
0-9f8b-6bf1b1e3a7ac.json
[KernelApp] To connect a client: --existing kernel-a57a8231-bfea-4680-9f8b-6bf1b1e3a7ac.json
0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.

According to this Stack Overflow post the output, though scary-looking, is only a warning, and you should be able to ignore it. It's happening because python 3.11 uses a "frozen" version of python with code objects for some of the built-in python modules that get loaded when the interpreter starts up already pre-allocated in order to reduce their load time during python's start up (i.e. they set it up to start faster), and their doing this means that the debugger might not work correctly - but since I'm not using the debugger, it shouldn't matter.

Ah, but there's always a problem lurking behind the advice to ignore "harmless warnings". Even with the kernel running, I couldn't get python/jupyter to work in my org-babel source blocks, so there was more to do.

Getting emacs-jupyter Working

The Problem

The first clue as to what might be happing was this line in emacs' startup messages.

Symbol’s function definition is void: org-babel-execute:jupyter-python

It looked like emacs-jupyter wasn't loading properly. There was also this message in the output:

Error retrieving kernelspecs: (json-number-format 5)

Searching for that error-message brought up this bug-report on github:

https://github.com/emacs-jupyter/jupyter/issues/436

Wherein the author of the bug-report mentions that loading emacs-jupyter is failing because it's trying to parse the output of jupyter and the warnings I was seeing causes it to fail (the bug-report references a different jupyter command, but the problematic output is the same).

Testing Turning Off the Warning

The first thing I tried was to follow the directions in the output and supress the warnings by setting an environment variable.

set --universal --export PYDEVD_DISABLE_FILE_VALIDATION 1

Note: This is fish-shell syntax.

I restarted the jupyter kernel and the warnings had gone away, so this much worked.

Really Turning Off the Warning

Setting the environment variable at the command-line changes the environment for my user, but I'm running emacs as a daemon so I needed to edit the systemctl file for my emacs service. I opened up the ~/.config/systemd/user/emacs.service file in emacs and added the line to set the environment variable for the emacs daemon.

[Service]
Environment="PYDEVD_DISABLE_FILE_VALIDATION=1"

Then I restarted the service.

systemctl restart --user emacs

Which gave me a warning that my changes to the configuration have to be re-loaded before restarting the service.

Warning: The unit file, source configuration file or drop-ins of emacs.service changed on disk
. Run 'systemctl --user daemon-reload' to reload units.

Oops.

systemctl --user daemon-reload
systemctl restart --user emacs

This time the emacs startup messages didn't have the jupyter errors so it looked like things were fixed.

Swapping a Virtual Environment For pipx

Suppressing the warnings pretty much solved the problem, but while I was getting this fixed I was also trying to set up a USB Windows installer using WoeUSB and found that pipx couldn't install it because of a dependency error. Pipx is good at installing some standalone python commands but it won't install things that are just libraries and it seems to sometimes also have problems installing dependencies for the commands that it will install. This has come up for me before, and the old solution was just for me to install the dependencies separately using pip before trying to install whatever it was that I was installing with pipx. Now, though, since ubuntu is trying to keep you from installing python modules globally, installing the dependencies means they either have to be available through apt or you have to set up a virtual environment and install them there (when I say have to I mean that since that's the way I know how to do it, that's the way I have to do it, not that there aren't other ways to do it that I just don't know about).

Doing it this way is easy enough, since I use python virtual environments a lot anyway, but then I ran into another problem which was that once I got the virtual environment set up I found out I had to run woeUSB as root, which then bypasses the whole virtual environment setup. The solution to that was to pass the full path to the virtual environment's woeUSB launcher to sudo, but it took enough time experimenting with other ways to do it before I got to that step that I decided I should minimalize how much I use pipx as much as possible - and in particular I should avoid using it with my emacs setup, since emacs will sometimss just quietly fail if there's a python-based error and it's only when things don't work that I'll realize there's a problem. So I decided to go with a dedicated virtual environment instead of installing jupyter with pipx.

This, once again was not a big deal in hindsight, but it took enough experimenting with other options before coming to the conclusion that this was the way to go that I thought I should make a note to my future self about it. To get jupyter working with jupyter-emacs:

create a virtual environment (python3 -m venv emacs-environment) in the .virtualenvs folder
activate it, then use pip to install wheels and jupyter

In the /.emacs.d/init.el file, activate the virtual environment before you load emacs-jupyter or anything else that needs python:

(require 'pyvenv)
(pyvenv-activate "~/.virtualenvs/emacs-environment")

Then restart emacs. So far this seems to have fixed it.

Kernel Panic At the Lunar Lobster

Cloistered Monkey — Wed, 03 May 2023 23:20:07 GMT

What Is This About?

I got a notification yesterday (May 3, 2023) that there's a new version of Ubuntu out (Lunar Lobster - 23.04) so I ran do-release-upgrade while watching a video (An Honest Liar) about James Randi. At the end of the upgrade there was some kind of error message about not being able to configure the Linux headers, which in retrospect should have worried me, but I was distracted so I just dismissed it. Then today when I booted up the computer I got an error message saying Kernel Panic - Not syncing: VFS: Unable to mount root fs on unknown-block (0,0). I managed to get it working (eventually) but now that Ubuntu's been around long enough and search engines seem to be choked with outdated answers I thought I'd document what I did in case it happens on another update.

First Get It Working

The first thing I tried was to follow the answers on this Stackoverflow page. The only thing this did was get my computer up and running again - which is, I suppose a big thing, not to be minimized, but there's a lot of stuff on that page and the only relevant parts were:

Reboot
At the menu that comes up choose "Advanced Options for Ubuntu"
Pick a prior version of the Linux kernel (5.19.26 in my case) and let it finish starting up.

Not a Solution But A Finger Pointing

The first thing I tried from that page was from this answer:

sudo dpkg --configure -a

This gave me error processing package linux-image-6.2.0-generic (--configure). So I searched some more and tried the suggestion from this askubuntu answer:

sudo dpkg --purge linux-image-6.2.0-20-generic

This gives an error along the lines of "dependency problems, not purging". Oi. So then I tried:

sudo apt autoremove

This gave me another error:

This time in the error message it said to check out /var/lib/dkms/nvidia/510.108.03/build/make.log which turned out not to have any useful information (to me, anyway) but it did tell me that there was something going on with my nvidia drivers that was causing the configuration of the new Linux kernel to fail.

Blame It On the Drivers

"Something going on with my nvidia drivers" being a little too vague for me to troubleshoot, I decided to go the brute force way and uninstall the nvidia-drivers. This actually proved a little harder than I thought it would be (which seems to always be the case, maybe I'm too optimistic about this kind of stuff). Every time I tried to run

sudo apt remove nvidia-driver-510

apt would try and configure the Linux kernel and run into the same error that I had before and exit without uninstalling the driver. Some kind of chicken and the egg thing. So then I tried dpkg instead of apt first:

sudo dpkg --purge nvidia-driver-510
sudo apt autoremove

dpkg managed to uninstall the driver and running apt autoremove not only cleaned out the unused packages but also triggered the kernel configuration and this time… no errors.

From Nouveau To Nvidia

After a reboot it started up okay and this time uname -r showed that I was using the newer kernel (6.2.0-20-generic). Yay. But now when I tried to re-install the nvidia drivers neither ubuntu-drivers nor apt seemed to know that they existed. It turns out that updating the ubuntu installation removes the proprietary drivers from the apt sources. So I launched the "Software & Updates" GUI.

And checked the "Proprietary drivers for devices (restricted)" button.

I chose to update the apt listing when I closed the GUI and then installed the drivers at the command line:

sudo ubuntu-drivers install

And then, after another reboot, now it works (I checked with nvidia-smi).

And the End

I haven't run any tests other than using the system but this seems to have fixed the problem. Well, the problem with the kernel, updating also broke all my python virtual environments, but, oh, well, better than a kernel panic, I suppose.

Converting A Date To Day of the Year In Python

Cloistered Monkey — Wed, 08 Feb 2023 00:56:59 GMT

This is a quick note on how to take a date and convert it to the day of the year (and back again) using python.

Date To Day of Year

We're going to use python's built-in datetime object to create the date then convert it to a timetuple using its timetuple method. The timetuple has an attribute tm_yday which is the number of days in the year that the date represents.

from datetime import datetime

YEAR, MONTH, DAY = 2023, 2, 7

DAY_OF_YEAR = datetime(YEAR, MONTH, DAY).timetuple().tm_yday
print(DAY_OF_YEAR)

So, February 7, 2023 is the 38th day of the year.

Day Of Year To Date

Now to go in the other direction we start with the first day of the year (represented as a datetime object) and add the number of days into the year we want. You can't create a datetime object with day zero so we need to start it on day one and then subtract one day from the number of days that we want.

from datetime import timedelta

JANUARY = 1

date = datetime(YEAR, JANUARY, 1) + timedelta(DAY_OF_YEAR - 1)

print(date.strftime("%Y-%m-%d"))

2023-02-07

Easy-peasey.

Source

Answer on StackOverflow that shows how to get the day of the year from the date.
Answer to the same question that shows how to go from the day of the year back to the date.

PyTorch and the Unknown CUDA Error

Cloistered Monkey — Tue, 01 Nov 2022 00:58:27 GMT

The Problem

I recently decided to get back into using neural networks again and tried to update my docker container to get fastai up and running, but couldn't get CUDA working. After a while spent trying different configurations of CUDA, pytorch, pip, conda, and on and on I eventually found out that there's some kind of problem with using CUDA after suspending and then resuming your system (at least with linux/Ubuntu). This is a documentation of that particular problem and it's fixes (fastest but not necessarily the best answer: always shutdown or reboot the machine, don't suspend and resume).

The Symptom

This is what happens if I try to use CUDA after waking the machine from a suspend.

import torch

torch.cuda.is_available()

/home/athena/.conda/envs/neurotic-fastai/lib/python3.9/site-packages/torch/cuda/__init__.py:88: UserWarning: CUDA initialization: CUDA unknown error - this may be due to an incorrectly set up environment, e.g. changing env variable CUDA_VISIBLE_DEVICES after program start. Setting the available devices to be zero. (Triggered internally at /opt/conda/conda-bld/pytorch_1666642975993/work/c10/cuda/CUDAFunctions.cpp:109.)
  return torch._C._cuda_getDeviceCount() > 0

As you can see, the error message doesn't really give any useful information about what's wrong - there are a couple of suggestions but neither seems relevant or at least doesn't lead you to the fix.

The Disease and Its Cure

There's a post on the pytorch discussion boards about this error in which "ptrblck" says that he runs into this problem if his machine is put into the suspend state. While mentioning this he also says that restarting his machine fixes the problem, but restarting it every time seems to defeat the purpose of using suspend (and I'd have to walk to a different room to log in and decrypt the drive after restarting the machine - ugh, so much work).

Luckily, in a later post in the thread the same user mentions that you can also fix it by reloading the nvidia_uvm kernel module by entering these commands in the terminal:

sudo rmmod nvidia_uvm
sudo modprobe nvidia_uvm

Which seems to fix the problem for me right at the moment, without the need to restart the machine.

print(torch.cuda.is_available())

False

Ummm… oops. Well, it did sort of fix one problem - the CUDA unknown error, but now it's saying that CUDA isn't available on this machine. Every fix begets a new problem. Let's try it again after restarting the Jupyter kernel.

import torch
print(torch.cuda.is_available())

True

Okay, that's better, I guess. It feels a little inelegant to have to do this, but at least it seems to work.

The Cloistered Monkey

Fixing NVidia With Wayland on Ubuntu 24.04 (Noble Numbat)

Table of Contents

The Problem

The Problems

WayDroid and X11

The Next Problem - Sleep and Logging out

No Settings

Conclusion

Links

ProtonPass: Finding the Suggested Password

Table of Contents

What This Is About

The Start Of the Trouble

Getting It Out Of Proton Pass

The Firefox Extension

Add New Item

Password Generator

Password History

Conclusion

Noble Numbat and the Broken Virtual Environments

Table of Contents

What is this about?

What Happened

Short Version

The First Hint (pipx)

The Broken Virtual Environments

Then a Nikola Error

An Ape-Iron Problem (no distutils)

Links

Fish Links

Apt-Fast In Sparky Linux

Table of Contents

Background

The Intructions

Create An Apt Entry

Add the Keyring and Install

The apt-key Solution

The Start of the Solution

Getting the GPG File

Now Back to Setting it Up

Update and Install

And Now, Another Problem

What Have We Learned Today Children?

Links Collected

Fish, Mocha, Chai - A Local Global Installation In Ubuntu

What This Is About

The Tutorial's Installation

This Might Be the Wrong Way

The Local Global

Make a local package directory

Edit the Fish Configuration

And Now

Sources

Ubuntu 22.04, Python 3.11 and the "emacs-jupyter Symbol's variable is void" Error

What This Is About

The Jupyter Kernel Warning

Getting emacs-jupyter Working

The Problem

Testing Turning Off the Warning

Really Turning Off the Warning

Swapping a Virtual Environment For pipx

Other Links

Kernel Panic At the Lunar Lobster

Table of Contents

What Is This About?

First Get It Working

Not a Solution But A Finger Pointing

Blame It On the Drivers

From Nouveau To Nvidia

And the End

Converting A Date To Day of the Year In Python

Table of Contents

Date To Day of Year

Day Of Year To Date

Source

PyTorch and the Unknown CUDA Error

Table of Contents

The Problem

The Symptom