Installing the EQ10Q equalizer on Gentoo

EQ10Q appears to be the absolute best quality equalizer available for Linux. I have tried that that comes with pulse-effects, but found that it introduced occasional audio glitches; no good for audiophile use.

EQ10Q is an LV2 plugin, so I start from the assumption that you already have JACK installed (doing so is trivial so I will not include instructions here). The following set of steps are what I used to get EQ10Q working:

  1. Emerge dev-cpp/gtkmm, since eq10q depends upon it for it’s UI.
  2. Download and install eq10q (do not use the ebuild available in the darkelf repo).
  3. Attach the audio-overlay repo (layman -a audio-overlay).
  4. Emerge media-sound/carla (requires accepting the live ebuild in package.use, plus enable the gtk2 flag because the eq10q UI uses GTK2).
  5. Run carla and add /usr/local/lib/lv2 to where carla looks for LV2 plugins.

Sync music playlists to Android

tl;dr

MTP sucks hard, and none of the free linux music players seem to have a reliable method for syncing a library subset (such as a playlist) to another device. rsync plus SSH plus ADB (and a little light shell scripting) to the rescue!

the problem

I have music in my library that is not and will never be available on streaming services (well, not until it is cost-effective to stream Plex from the home that is ;o). Consequently I need to have that music copied onto my phone. The set of files is a subset of my entire library and is around 25GB. Wireless syncing is all well and good, but I don’t need the sync to be wireless and wired would be way faster.

the solution

write a shell script that takes the path of your music library and a playlist file as input, then rsync’s the files listed in the playlist over SSH to a localhost port. How in hell is that useful? Well, rather conveniently, ADB allows you to map a local port to one on your connected phone:

adb forward tcp:2222 tcp:2222

So as long as you have an SSH server running on your phone on that port (I use SSHelper) then the files will end up in SDCard/Music on the phone (this path is hard-coded in the script but obvious so can be changed). I get roughly 50MB/s transfer rate, which seems none too shabby; the whole 25GB playlist in around 11 minutes.

Scanning on Gentoo using a Lexmark MX310dn

SANE, the de facto linux scanning solution has only one open source driver (backend, in SANE parlance) that supports Lexmark scanner, and only a small number of USB models at that. Lexmark have released a closed-source SANE backend, available at their support site if you search for downloads related to the product you have. Unfortunately they do not offer a portage ebuild, only RPM or DEB packages so it takes a little more effort to get it installed and, at least in my case, needed a few further tweaks to get it operating.

Installing the Lexmark SANE backend

  1. Visit support.lexmark.com and search for your scanner.
  2. On the resulting page, in the Downloads tab, select the most recent Debian release.
  3. Download the offered .deb file (and any firmware update available).
  4. Put the .deb in a folder by itself, then follow ‘Option 2 : manual installation’ of the Arch Linux instructions to install it.

Segfault??

I now found that Xsane would launch and crash, and running the simplest way in to sane:

scanimage - L

would throw a segmentation fault. Great. In my case the fix was to simply disable USB scanner support (no great loss), as follows:

  1. Edit the Lexmark backend configuration file:

sudo nano /etc/sane.d/lexmark_nscan.conf

  1. Find the following variable and set it as below:

DISCOVER_USB_FLAG=0

  1. I also found it useful to make the following setting, else the scanner showed a second entry in the list for the Lexmark:

SHOW_PSEUDO_NETWORK_FLAG=0

Scanner not supported??

So huzzah, now Xsane will open and offer the Lexmark scanner in the list of those detected. When I tried to scan however, after a pause I was getting a window saying that my scanner model wasn’t supported. This seemed highly unlikely. Trying a scan from the command line:

scanimage -d lexmark_nscan > ~/test.jpg

showed me the problem; the scanner just wasn’t listening:

java.net.ConnectException: Connection refused

In my case, it turned out I had two problems; the printer’s firmware needed an update (I reckon it was around 2017 vintage beforehand), and I needed to enable the HTTP server on the printer (dig your way down into the printer’s settings page in the TCP/IP settings). I also enabled HTTPS support (further down the list in the same menu).

It is a shame that the HTTP server is required, given the increased attack surface (printers are hardly known for their strong security after all!) but on the other hand it did provide the most convenient method to update the firmware.

With these two things done I was off to the races. The method of operation is distinctly bizarre (you’ll see), but if you follow the steps it works fine. I also found that if you use the auto document feeder, when you click Scan it runs all the pages through at once but only records the first sheet in a multi-document project. Subsequent clicks of the Scan button however would bring in each successive page.

Done!

Mounting NFS shares when using dhcpcd as a network manager under Gentoo

tl;dr

If you use NFS shares and/or want your workstation to have a static LAN IP and want seamless wired<->wireless LAN roaming under Gentoo/OpenRC, then dhcpcd is your go-to network manager. But not without a bit of manual tweaking..

Use Case

You have the following requirements:

  • your workstation is a laptop that moves between wired and wireless connections within your LAN
  • you use NFS shares that you would like auto-reconnected when roaming between connections, and auto-connected on boot regardless of the connection available at that time (at the very least, do not hang on boot due to unavailable shares)
  • you require your workstation to have a static LAN IP so as to be able to reliably expose services it hosts outside the LAN (i.e. so as to forward ports from your router to it)

Solution

I don’t know if it’s just me, but network roaming under Gentoo using the default netifrc + ifplugd system is just not reliable. Maybe it’s that ifplugd isn’t oriented towards wireless connections, maybe it’s that my usual wired ethernet port is on a USB-C dock, maybe it’s the NFS shares I use, but I was continually finding myself without a path to my LAN when switching from one form of connection to another or have it lock up trying to remount shares. Having bashed my head against this for some weeks I re-read the Gentoo network management article for the millionth time, once again decided against migrating to Network Manager (and its associated systemd leanings), but for the first time noticed that it mentions in passing that the simplest network management solution is to use dhcpcd in Master mode.

So then, I uninstalled ifplugd and the netifrc services in favour of dhcpcd. The latter is admirably install-and-forget, but will not assign a static address to any interface by default. Whilst it can support static IP assignment, in order to use it one needs to specify the configuration for every local interface else confusing badness will transpire1The approach I chose instead is to use DHCP and apply IP reservation at the DHCP server to maintain a predictable address. Explanation of how to do that is beyond the scope of this article (I don’t know what DHCP server you’re using).

Okay, so now we have a system that will pick up an IP address as it roams from wired to wireless connections and back. Mount-at-boot NFS mounts will not handle this cleanly at all though, so the second part of the magic is to do away with static mounts in /etc/fstab entirely, in favour of using autofs to mount them on demand. Installation and configuration is documented admirably at the Gentoo site, so I will not duplicate that here.

That’s all there is to it! It seems trivial in retrospect, but I put up with all manner of half-working solutions on the way here, so I thought it worth documenting anyway.

 

 

  1. If you have multiple local ethernet ports it’ll assign the same address to all of them regardless of whether they’re even up, resulting in bizarre routing problems. The worst of these issues is that down interfaces receive a metric of 0, that is to say the most preferred, meaning that LAN routing fails because it tries to use interfaces that aren’t up!↩︎

Upgrading the Gentoo kernel when running ZFS on an NVME SSD

Blending together the Gentoo Kernel Upgrade Guide with the excellent Gentoo/ZFS/NVME SSD installation articles by Guy Robot I synthesized the following set of steps, which I am well pleased to say went without a hitch first time :o)
Please note this is an upgrade article only and assumes you already have a working system that boots ZFS. If you’re starting from scratch please follow Guy Robot’s execellent articles linked above.
If you’re already there and genuinely only want to upgrade your kernel then please read on..

  1. Obvously enough, start with updating the relevant packages (sys-kernel/linux-firmware, sys-kernel/gentoo-sources, sys-fs/zfs, sys-fs/zfs-kmod; note that since v0.8 ZFS no longer relies upon sys-kernel/spl so this can be unmerged). In my case I just went for a complete emerge update @world:

    emaint -a sync
    emerge –update –deep –changed-use –keep-going @world

  2. Ensure that the file /etc/portage/savedconfig/sys-kernel/linux-firmware points to (or is) a file containing the list of only the firmware blobs to compile into the kernel (the remainder should be commented out or deleted). If you are not doing so already, I recommend setting the savedconfig USE variable for the sys-kernel/linux-firmware package so that future emerges retain your config; your set of required firmware is not likely to change from one kernel build to the next.
  3. The /usr/src/linux symlink points to the kernel source package directory. This needs to be updated to point to the location we wish to use for this build. Emerging the gentoo-sources package with USE=symlink would do this for us automatically, but there are other packages out there that need to know where to find our current source, so it’s best that we manage this manually. The Gentoo way is to use the eselect command:

    eselect kernel list
    eselect kernel set <the index number of the new source dir>

  4. We’ll now do some work in the kernel dir. Change into it explicitly to ensure picking the new location of the symlink:

    cd /usr/src/linux

  5. Copy the currently-running kernel’s config to the new kernel’s source dir to use it as a basis:

    cp /boot/kernels/<currently-running kernel dir>/.config /usr/src/linux/.config

  6. We already have the NVME support enabled from the original install (see Guy Robot’s article for details), so next we need to update the config to match the new kernel version (i.e. bring in options that are new since the last time we built it, selecting the recommended setting for each):

    make olddefconfig

  7. This step is probably unnecessary, but I ran the graphical menuconfig and saved, just to ensure any stray Gentoo-sepcific options got picked up:

    make menuconfig

  8. Right, we’re ready to build the kernel. FWIW this took ~1 hour on my Intel i7-7600U. Set the -j parameter to the number of processor cores you have plus one:

    make -j5

  9. Now we have the latest kernel sources we must emerge ZFS again so that it’s compiled against them (the same applies to any other kernel modules you use):

    emerge sys-fs/zfs sys-fs/zfs-kmod

    N.B. The following command might be enough to achieve this in one shot, but it was unclear to me whether zfs also needed to be rebuilt if zfs-kmod is rebuilt which is why I did the above:

    emerge @module-rebuild

  10. Install the kernel and modules (they aren’t actually used until we reconfigure grub to know about them so not to worry):

    make modules_install
    make install

  11. The kernel and it’s ancillaries will be installed to /boot. For neatness and the ability to roll back in case of problems we will keep the existing kernel alongside the new, so create subdirs in the form /boot/kernels/<kernel version dir> and move/rename vmlinuz, config and System.map from /boot to there.
  12. Now we need to build an init ramdisk that’ll import our ZFS pools at boot time. First step here is to use Fearedbliss’ tool to create one that is Gentoo/ZFS-friendly (N.B. I’m assuming you’ve already added the necessary Portage overlay to your repos.conf, as explained in Guy Robot’s article) :

    emerge bliss-initramfs
    bliss-initramfs

    Select ‘1’ for ZFS and ‘n’ to enter your own kernel version. When it has finished, copy the resulting initrd-<kernel version> file from /boot to /boot/kernels/<kernel version dir>/initrd

  13. We now need to edit the configuration within the init ramdisk to have it mount our zpools explicitly. Folllow the instructions in Guy Robot’s article starting from “We now need to modify the initramfs.” and up until “Now you have a custom initramfs” :o)
  14. Next we need to let grub know about our new kernel so we can select it at boot. Edit /boot/grub/grub.cfg to set the timeout to 3 (temporarily until we’re happy with the new kernel) and add a menu item like the below, updating the kernel version number appropriately:

    menuentry “Gentoo – 4.19.52-gentoo-FC.01” { linux /@/kernels/4.19.52-gentoo-FC.01/vmlinuz root=rpool/ROOT/gentoo resume=/dev/nvme0n1p3 by=id elevator=noop quiet logo.nologo initrd /@/kernels/4.19.52-gentoo-FC.01/initrd }

  15. Some packages depend on your existing kernel source when building. To prevent it being unmerged when new versions are released, preserve the gentoo-sources package version by adding it to @world:

    emerge –noreplace sys-kernel/gentoo-sources:<version>

  16. Take a deep breath and test your work:

    reboot

OpenRC – managing dependency when you’ve multiple network interfaces

tl;dr

If you have multiple network interfaces and cannot guarantee that all will always be up, but have other services that depend upon network availability, set rc_depend_strict=”NO” in /etc/rc.conf.

I habitually use a wireless network at home and tether my cellphone whilst out and about. I was finding that the VPN service refused to start when I switch network, complaining that the non-started network was not available. By default, Gentoo openvpn’s rc init script has a depend on ‘net’, which I had assumed was interpreted as ‘any network interface’, but in fact by default in OpenRC this identifier is interpreted as all network interfaces. To change this behaviour edit /etc/rc.conf and set rc_depend_strict=”NO”. Simple as that – net will now be interpreted as any interface instead of all interfaces.

Using ffmpeg to downmix Dolby DTS or 5.1 audio to stereo

So you have a surround sound audio file (DTS, 5.1, quadrophonic, whatever) that you want to play back on your old-skool stereo hi-fi system. It’ll play, but you only get to hear front-left & front-right of the many channels. What’s needed is a way to mix the content of those other channels into FL & FR at an appropriate volume. ffmpeg, our favourite audio swiss-army knife, has the capability built in to downmix to a chosen number of output channel with the -ac option, as follows:

ffmpeg -i input_filename -ac 2 output_filename.wav

I could not however find any documentation on the panning and gain levels this option uses for the surround channels, and to my ear the rear channels are mixed too low (possibly because in my case the source was an old Quadrophonic record release with four equally-utilized channels). Luckily, ffmpeg also exposes the ability to finely tune downmixing, using the ‘pan’ filter. From the excellent and exhaustive documentation (here), plus gain values from someone who trod this path before using mplayer, I synthesised this command line:

ffmpeg -i input_filename -af "pan=stereo|FL<FL+0.5*FC+BL+0.6*SL|FR<FR+0.5*FC+BR+0.6*SR" -acodec pcm_s32le output_filename.wav

Notes:

  1. For quadrophonic source material the ‘+0.5*FC’ and ‘+0.6*SL’ (and right channel equivalents) are superfluous but are included for compatibility with more modern x.1 surround source material.
  2. The output file uses the wav suffix and I am specifying 32 bit samples (ffmpeg passes through sample rate as-is from the source to destination file). Modern multi-channel content is usually ‘hi-rez’, that is to say higher resolution than the usual 16bit/44kHz of CD quality audio and we want to ensure the mixdown does not reduce the quality of the source material; these conversion choices ensure that the quality of the source material is retained . Once you’ve used this approach to downmix to a .wav file you can then use your converter of choice to turn the .wav into a more manageable format such as FLAC or MP3.
  3. If your source material is a DVD-A disc or .iso, the audio files can be found in the AUDIO_TS subfolder. Filename suffixes vary, but you’ll be looking for the huge files in that folder.

References

https://eknet.org/main/linux/downmixdts.html

Configure Firefox to respect local host file domain name entries

The scenario

You’ve added ‘mynasbox  192.168.1.50’ to your hosts file, you want to be able to type mynasbox in the Firefox URL bar and get to its web-based config UI, but instead you’re seeing a ‘There seems to be a problem with this URL’ error message.

tl;dr

In Firefox about:config, set browser.fixup.dns_first_for_single_words to true to be able to use single-word domain name mappings in your hosts file at the same time as retaining the ability to type searches in the Firefox URL bar.

Update 2020-06-12

The fix described here still works but now has one downside; single-word searches are treated as domain names and do not route (unless of course they’re in your local hosts file).

The Full Story

This topic usually goes off the rails when discussed on ‘net forums because Firefox does not in and of itself directly read your hosts file – it relies upon the operating system to resolve domain names for it. In practice though, things are far from that simple; Firefox has had quite a few features added over the years to assist users in getting to web sites even if they don’t type in a proper domain name, which can mean that what you typed gets invisibly altered before any DNS query takes place, confounding the more technical among us.

There are two features of Firefox config that are at play here – keyword and fixup. Both are accessed by browsing to about:config and accepting the dire warning, then typing the feature name in the search box. The keyword feature seems rather unpopular these days (go search if you’re interested in what it’s for), and though you can achieve the desired result by setting this to false, if you do so you also lose the ability to type searches in the Firefox URL bar. Consequently it’s best to leave this at its default value of true (and indeed this must be the case for the fix below to work correctly).

fixup is the feature where Firefox will try to turn what you type into a parseable URL. It is this that causes the eventual DNS lookup not to match your hosts file entries. Feel free to play with the set of browser.fixup options available, however I find that setting browser.fixup.dns_first_for_single_words to true is the simplest general fix for the problem statement, on the basis that all my local servers have one-word names in the hosts file.

References

https://superuser.com/questions/382905/how-to-prevent-firefox-converting-localhost-urls-into-search-queries

https://support.mozilla.org/en-US/questions/1117449

https://support.mozilla.org/en-US/questions/1011327

Install Gentoo linux on ZFS

Gentoo, being a source-based linux distribution, has a very particular installation process. One has to make many decisions at every step along the way, since installation and configuration effectively happen at the same time, and certain things can be very hard to change once the installation has completed. The prime example of this hard-to-change-after-the-fact situation is the choice of file system on which the OS is being installed. Whilst there are a variety of filesystems available for use under linux, in my opinion ZFS remains the best one available, at least until BtrFS matures and becomes stable enough for use in a production environment.

The Gentoo article on the topic of installing on ZFS is very good, but out of date in that it does not cover the extra steps required when installing on an NVMe SSD. My life was saved by a blogger named Guy Robot, whose excellent article I followed to achieve success. Thanks Guy!

How to configure (Gentoo) Linux to have ALSA bitperfect (& Pulseaudio convenient) sound

Preamble

Bitperfect audio on linux (or any OS, really) means your music player’s transport must connnect directly to the sound hardware without there being a mixer in the way.Why? Two reasons:

  1. In order to mix, incoming audio streams all need to be resampled to a common audio format (usually 48kHz/16bit). This rides roughshod over those 192kHz/32bit high-resolution files you paid good money for from HDAudio or wherever.
  2. Adding additional software to the signal path cannot improve audio quality.

To connect music players direct to sound cards on linux the choice seems to be between ALSA (long history, well-understood) and Jack (pro-audio production capabilities). For the audiophile ALSA will do just fine – we don’t need the flexibility and low latency Jack offers.

Configure the high-quality audio player

I chose gmusicbrowser as my iTunes-a-like media library. This guide tells you how to configure it to talk directly to ALSA so as to ensure bitperfect playback. The guide doesn’t reference the fact that gmusicbrowser supports a few different audio transports, and it is within the chosen transport where you need to make the configuration listed in the guide. I have chosen gstreamer as the transport, on the basis that it has a vast number of audio format plugins in its library (especially that it supports ALAC and AAC). Obviously this means you need to install a few plugins to get a good range of audio format support, but that’s not the focus of this howto – I’ll leave it as an exercise for the reader.

Audio for the rest of the system

Now then, didn’t I say something about “no mixer” earlier on? That’s going to be a pain in the butt if I want to have my web browser, video player, etc. all sharing the same soundcard, so what to do? Well, on linux Pulseaudio is the de facto solution for shared soundcard usage, and is installed in pretty much every linux distribution out there. Pulseaudio’s architecture is interesting in that it doesn’t boot up when the OS boots up. Instead it runs on demand, when an app tries to access it. The only problem is that the linux GUI (KDE Plasma in my case) simply tries to access it as soon as that GUI starts up, consequently giving Pulseaudio exclusive control of the sound card. This means that, because we configured it to talk directly to the card via ALSA, gmusicbrowser will complain that it cannot access the sound device when we try to play music.

So then, how do we get these two audio management solutions to co-exist?

Switching between Pulseaudio and bitperfect audio

At a high level, to configure the system to be able to switch between our two ‘audio modes’, we do this:

  1. Set Pulseaudio to not spawn automatically.
  2. Start Pulseaudio explicitly when the linux GUI starts.
  3. Use a script to toggle whether Pulseaudio is running or not.

Here’s those steps in detail:

  1. To tell Pulseaudio not to spawn automatically, create a file named client.conf in your home directory at the following path:

~/.config/pulse/client.conf

This file needs to contain just the following line:

autospawn = no

  1. To manually start Pulseaudio, we simply need to run it and tell it to run as a daemon:

pulseaudio -D

It is trivial to create a bash script that executes the above, then call that script when your GUI starts up. I’m running gentoo with the KDE Plasma 5 GUI, so I simply go into the System Settings application and browse to Startup and Shutdown->Autostart then click Add Script.. to add a call to my script that executes Pulseaudio. YMMV.

  1. I’ve written a script that will detect whether or not Pulseaudio is running, then execute it or kill it as necessary to toggle its state. Here’s the content of that script:

#!/bin/sh
# obtain the pid of the pulseaudio process and count the number of return values
case “$(pidof pulseaudio | wc -w)” in

0)  # pidof returned nothing so pulseaudio is not running, hence start it (daemonized)
pulseaudio -D
;;
*)  # pidof returned at least one value so pulseaudio running, hence kill it
# pulseaudio –kill
# use killall to handle cases where there happens to be more than one instance running
killall pulseaudio
;;
esac

I’m not going to insult your intelligence by telling you how to save that in a file and make it executable. With the script in hand, in KDE Plasma it is trivial to add a button to the taskbar panel (using the Quicklaunch widget) that can run any chosen script. One can also alter the icon the button uses, so as to select a ‘speaker’ icon from the library of available icons. In this way, I have a button on the taskbar that can toggle whether Pulseaudio is running or not.

Day-to-day usage

With the above setup, on startup the system will be using Pulseaudio and all apps will share soundcard usage, the system-wide Pulseaudio Volume Control application will work and so will hardware volume keys on the keyboard. As soon as I want to switch to audiophile-quality playback I click the button which shuts down Pulseaudio. From that moment, the system-wide volume controls stops working, as do the hardware volume control keys, and no normal app (web browser etc.) will be able to play audio. Any application that is configured to talk directly to ALSA will however now be able to do so, and gain sole usage of the soundcard. In this way, gmusicbrowser can play back those high resolution audio files in their native bit rate. Volume control is managed only within each such piece of software.

Other info

The reason to start the system up with Pulseaudio enabled is that certain software (I’m looking at you Firefox) might not output sound at all if it doesn’t find Pulseaudio when it is started. I would also like to express my thanks to Rizlaw and yay101 from the head-fi.org forums, whose threads at that site were pivotal to my understanding whilst working out this configuration.