Install Gentoo linux on ZFS

Gentoo, being a source-based linux distribution, has a very particular installation process. One has to make many decisions at every step along the way, since installation and configuration effectively happen at the same time, and certain things can be very hard to change once the installation has completed. The prime example of this hard-to-change-after-the-fact situation is the choice of file system on which the OS is being installed. Whilst there are a variety of filesystems available for use under linux, in my opinion ZFS remains the best one available, at least until BtrFS matures and becomes stable enough for use in a production environment.

The Gentoo article on the topic of installing on ZFS is very good, but out of date in that it does not cover the extra steps required when installing on an NVMe SSD. My life was saved by a blogger named Guy Robot, whose excellent article I followed to achieve success. Thanks Guy!

How to configure (Gentoo) Linux to have ALSA bitperfect (& Pulseaudio convenient) sound

Preamble

Bitperfect audio on linux (or any OS, really) means your music player’s transport must connnect directly to the sound hardware without there being a mixer in the way.Why? Two reasons:

  1. In order to mix, incoming audio streams all need to be resampled to a common audio format (usually 48kHz/16bit). This rides roughshod over those 192kHz/32bit high-resolution files you paid good money for from HDAudio or wherever.
  2. Adding additional software to the signal path cannot improve audio quality.

To connect music players direct to sound cards on linux the choice seems to be between ALSA (long history, well-understood) and Jack (pro-audio production capabilities). For the audiophile ALSA will do just fine – we don’t need the flexibility and low latency Jack offers.

Configure the high-quality audio player

I chose gmusicbrowser as my iTunes-a-like media library. This guide tells you how to configure it to talk directly to ALSA so as to ensure bitperfect playback. The guide doesn’t reference the fact that gmusicbrowser supports a few different audio transports, and it is within the chosen transport where you need to make the configuration listed in the guide. I have chosen gstreamer as the transport, on the basis that it has a vast number of audio format plugins in its library (especially that it supports ALAC and AAC). Obviously this means you need to install a few plugins to get a good range of audio format support, but that’s not the focus of this howto – I’ll leave it as an exercise for the reader.

Audio for the rest of the system

Now then, didn’t I say something about “no mixer” earlier on? That’s going to be a pain in the butt if I want to have my web browser, video player, etc. all sharing the same soundcard, so what to do? Well, on linux Pulseaudio is the de facto solution for shared soundcard usage, and is installed in pretty much every linux distribution out there. Pulseaudio’s architecture is interesting in that it doesn’t boot up when the OS boots up. Instead it runs on demand, when an app tries to access it. The only problem is that the linux GUI (KDE Plasma in my case) simply tries to access it as soon as that GUI starts up, consequently giving Pulseaudio exclusive control of the sound card. This means that, because we configured it to talk directly to the card via ALSA, gmusicbrowser will complain that it cannot access the sound device when we try to play music.

So then, how do we get these two audio management solutions to co-exist?

Switching between Pulseaudio and bitperfect audio

At a high level, to configure the system to be able to switch between our two ‘audio modes’, we do this:

  1. Set Pulseaudio to not spawn automatically.
  2. Start Pulseaudio explicitly when the linux GUI starts.
  3. Use a script to toggle whether Pulseaudio is running or not.

Here’s those steps in detail:

  1. To tell Pulseaudio not to spawn automatically, create a file named client.conf in your home directory at the following path:

~/.config/pulse/client.conf

This file needs to contain just the following line:

autospawn = no

  1. To manually start Pulseaudio, we simply need to run it and tell it to run as a daemon:

pulseaudio -D

It is trivial to create a bash script that executes the above, then call that script when your GUI starts up. I’m running gentoo with the KDE Plasma 5 GUI, so I simply go into the System Settings application and browse to Startup and Shutdown->Autostart then click Add Script.. to add a call to my script that executes Pulseaudio. YMMV.

  1. I’ve written a script that will detect whether or not Pulseaudio is running, then execute it or kill it as necessary to toggle its state. Here’s the content of that script:

#!/bin/sh
# obtain the pid of the pulseaudio process and count the number of return values
case “$(pidof pulseaudio | wc -w)” in

0)  # pidof returned nothing so pulseaudio is not running, hence start it (daemonized)
pulseaudio -D
;;
*)  # pidof returned at least one value so pulseaudio running, hence kill it
# pulseaudio –kill
# use killall to handle cases where there happens to be more than one instance running
killall pulseaudio
;;
esac

I’m not going to insult your intelligence by telling you how to save that in a file and make it executable. With the script in hand, in KDE Plasma it is trivial to add a button to the taskbar panel (using the Quicklaunch widget) that can run any chosen script. One can also alter the icon the button uses, so as to select a ‘speaker’ icon from the library of available icons. In this way, I have a button on the taskbar that can toggle whether Pulseaudio is running or not.

Day-to-day usage

With the above setup, on startup the system will be using Pulseaudio and all apps will share soundcard usage, the system-wide Pulseaudio Volume Control application will work and so will hardware volume keys on the keyboard. As soon as I want to switch to audiophile-quality playback I click the button which shuts down Pulseaudio. From that moment, the system-wide volume controls stops working, as do the hardware volume control keys, and no normal app (web browser etc.) will be able to play audio. Any application that is configured to talk directly to ALSA will however now be able to do so, and gain sole usage of the soundcard. In this way, gmusicbrowser can play back those high resolution audio files in their native bit rate. Volume control is managed only within each such piece of software.

Other info

The reason to start the system up with Pulseaudio enabled is that certain software (I’m looking at you Firefox) might not output sound at all if it doesn’t find Pulseaudio when it is started. I would also like to express my thanks to Rizlaw and yay101 from the head-fi.org forums, whose threads at that site were pivotal to my understanding whilst working out this configuration.

macOS network shared drive connection options (NFS rocks, AFP & SMB suck)

I use an aging Western Digital NAS drive and have found it harder and harder to make macOS continue to connect to its shares using AFP. Even the fixes I link above are a pain to get working reliably, due to the file system’s weird behaviour these days with plist files. On a borrowed Sierra MacBook Pro it finally gave up the ghost and refused to talk any longer. Mount attempts were returning alternately errno 32 and 89, suspicions led toward .AppleDB folders on the shares before I decided enough was enough.

“Never mind” I said to myself, “Apple wants us to use Samba these days anyway, I’ll configure that instead”. AFP performance was never exactly stellar, but oh my god Samba is abysmal!!1 I’ll admit I wasn’t helping myself in that the files in question were multi-hundred gigabyte sparse disk images on the NAS, but still and all there had to be another way..

Fortunately there is; the venerable protocol from Sun named NFS.

For whatever reason, Apple don’t make it easy or obvious how to connect NFS shares, so here’s the steps to follow:

    1. On your server, set up one or more NFS shares. How to do this will depend on your server; in my case the NAS GUI Folder Shares simply has a tick-box to enable NFS for a given share.
    2. Open Terminal and in your favourite text editor, add the following line to the end of /etc/auto_master:

      /mnt/nfs auto_nfs

    3. Create a new file named /etc/auto_nfs, with the following content, all on one line. Note that the items in italics are placeholders for your specifics, and that all NFS sharenames on WD NAS drives are prefixed with /nfs/ (Update: or are they prefixed with /DataVolume/ ? A later configuration on an El Capitan client needed the latter). I spent a long time playing with the fstype parameter list to find something that worked; there are many resources online if you would like to understand each parameter in more detail:

      sharename -fstype=nfs,vers=3,rw,soft,intr,rsize=32768,wsize=32768,noatime,timeo=1200,retrans=10,proto=tcp,resvport,locallocks,noacl nfs://ip address of server/sharename

    4. Create a mount point for each share listed in auto_nfs. Do not create these in /Volumes – macOS will delete them when you restart your machine:

      sudo mkdir /mnt/nfs

    5. Finally, to inform the NFS client of the new configuration, execute the following:

      sudo automount -cv

    The extra bonus of this approach is that it uses the automount system, meaning the OS will mount any of these shares as soon as you attempt to browse to them. To make it easier to access these mount points more easily from Finder, use Go To (⌘⇪G) and type in /mnt, then drag the folder ‘nfs’ shown in Finder to your Favourites bar over on the left.

  1. H/T to the publishers of the following articles that helped me coalesce this solution:
  2. https://smekkley.wordpress.com/2014/01/18/time-machine-via-nfs-on-mavericks/
  3. http://blog.centurio.net/2016/03/16/automount-network-shares-on-mac-os-for-use-in-itunes/
  4. https://lb9mg.no/2016/03/23/using-os-x-time-machine-with-nfs/

 

Update:

  1. It turns out that a security change in 10.11.5 might be the cause of SMB slowness, and here’s how to disable that feature to get better throughput. ↩︎

Which Firefox tab is eating my computer’s performance?

Firefox is once again the leading web browser, now that they’ve caught up with Chrome’s multi-threaded approach for tab management. Where Chrome still has an edge though is its built in Task Manager window, showing you the resource usage of each individual tab. Yes, Firefox has the about:performance page, but in my experience it isn’t good at showing me which tab is causing my laptop fan to do a jet engine impersonation, so how to proceed when I want to cool things down?

Fortunately, Firefox’s threads are separated out in the macOS Activity Monitor. Note that I have ordered the view in %CPU descending:

macOS Activity Monitor

Whilst there’s no way to match a particular ‘FirefoxCP Web Content’ process to a particular tab, if you highlight the one that’s using the most CPU time, click the ‘Quit’ button at top-left (the one with a white X on a black circle) and select Quit on the dialog, you will kill that one process without bringing down Firefox itself.

How to know which tab you killed? Well, switch back to Firefox and cursor through your open tabs. When you reach the one you’ve killed, you’ll see a “Gah. Your tab just crashed.” message, with a handy ‘Reload’ button, so if it was one you absolutely didn’t want to lose all is not lost.

That’s it! Obviously the approach I’ve described here uses a Mac’s Activity Monitor, but I expect there are equivalent ways to achieve the same on Linux. I don’t think Windows offers adequately fine-grained process control, but I’ll be more than happy to be contradicted in the comments!

N.B. The astute will notice that there are far fewer FirefoxCP Web Content process than you have open tabs. I don’t know how Firefox metes out threads to tabs, but in my experience with this approach CPU-hungry tabs always seem to have a process all their own; I’ve not yet had one process kill take down multiple tabs.

How to unlock your iPod without resetting it

So you’ve pulled that dusty old iPod out of a drawer and find that you’ve forgotten the PIN code screen lock. You want to get into it without resetting/restoring it or having to resync all your music? Yes it’s possible, thanks to Apple’s hilarious iPod ‘security model’:

  1. Connect the iPod to your laptop
  2. Use a file manager that shows hidden files. Obviously only macOS is a challenge here – I don’t use Finder but perhaps it still hides the iPod filesystem even if you’ve set it to show hidden files. I guess either get a proper file manager or just use Terminal?
  3. Browse to the folder <whatever the iPod is called>/iPod_Control/Device. In Terminal I typed (note forward slashes before spaces in path):

    cd /Volumes/my\ iPod/iPod_Control/Device/

  4. Rename the file named _locked to _unlocked.
  5. Eject the iPod (Close Terminal first else iTunes will complain).
  6. Yep, that really is all there is to it!

Yay Bandcamp!!

Just a  quick note to say props to Bandcamp.com (the pre-eminent musicians-selling-their-wares-direct-to-the-fans site), who’ve just published a note to show that they’re significantly profitable and have been for four years. This is the context of a music industry of diminishing sales, shrinking margins and even Apple suggesting that they’re going to get out of the music download game.

Just goes to show that it’s not music sales that are broken, it’s the music industry that is broken.

Apple Music will (update: might) delete your music

It’s as simple as the title says; Apple Music™ will delete your music. As reported here, when you sign up for Apple Music, your entire catalogue is uploaded to their servers (excepting files it considers it already has a match for), then deletes all your local files.

Update: Fortunately Apple are considering this a bug and are taking extraordinary measures to fix it.

The idea is that they’ll then stream your content to you wherever you are, but this naive premise has several dire problems:

  • if you don’t have a network connection you don’t have any music.
  • Apple are a hardware company; their DNA just isn’t suited to online services; I for one would not trust my data to their cloud.
  • If you do re-download the files they’ve deleted, more problems arise:
    • you can only do so one file at a time which is a ball-ache and would take days
    • if they matched your files incorrectly, the file you download isn’t necessarily the same version of a song you had previously
    • they do not respect file format and bit depth – if you have files in a format they don’t support (e.g. WAV) you get back a 256kbps AAC file.

 

In summary, teh suck :0(

Privoxy for Android

UPDATE: Version support list

It has been tried several times in the past with limited success, but now confidence is high that there is a broadly compatible release of Privoxy for Android devices.

It is available at Sourceforge or here, the project’s mirror site.

This release statically links all dependencies in order to be as compatible as possible, at the cost of raising the binary’s size from ~600K to ~1.1MB; it was considered well worth the size growth in order to be applicable to the widest range of Android devices and versions.

So far Privoxy for Android has been explicitly tested on point releases of Android 4 through 6 inclusive, and on Samsung and Lenovo tablets, a Samsung Galaxy S4 phone and a Oneplus 2 phone. Feedback from the user community indicates no issues in operation on Android 7.

HTC One proximity sensor failure whilst in phone call

TL;DR – if you’re going to customise your phone, always ensure that the firmware version matches the Android version in the ROM. A mismatch can cause very slight and subtle bugs!

I noticed that my HTC One, customised with the Android Revolution HD ROM (Android 4.3) and the Team7even kernel, had a problem whereby the screen would switch off as soon as I started a phone call (even if I did not bring the phone close to my head) and would not switch back on even after the call completed; I would have to wake it up with the button.

Using a phone tester app from Play Store I worked out that the proximity sensor had stopped responding. I furthermore worked out that it was fine immediately after turning on the phone, but after the first time it entered deep sleep then the problem would express itself.

A lot of digging and testing later I have worked out that it was caused by a mismatch of the phone firmware (Android 4.1-based – v1.29.631.17) and the ROM Android version (4.3). Non-trivial to fix; I had to go back to stock everything (good thing I kept a Nandroid of my stock ROM from when I first got the phone) so as to be able to pick up the OTA upgrade to Android 4.3 (3.22.631.1). This cleanly upgraded the phone firmware, from which I’m now free to install any ROM (just so long as it is based on Android 4.3!).

Roxio Toast mis-reporting DVD DL capacity – how to fix

I wanted to burn a dual-layer DVD the other day, but confusingly Toast quoted the space available as 4.21GB, where of course it ought to be 7.92GB. This was the case regardless of whether a disk was inserted. Much playing around eventually found the cause, and a solution.

It turns out that one of the disks on a spindle of blank DVD DL (DVD+R, though I don’t think that matters) was faulty, and indeed only had 4.21GB available. Unfortunately it seems there’s a bug in Toast such that it then stores indefinitely this incorrect capacity as being what a DVD DL can store, preventing one from using the full capacity of a DVD DL thenceforth.

The solution to this is to delete Toast’s preference plist, that’s found at /Users/<username>/Library/Preferences/com.roxio.toast.plist. Once this is done and Toast it restarted, it’ll take you through the ‘first time launch’ spiel (but won’t lose your serial number registration) and, once you’re done with that, DVD DL capacity will be correctly reported once more.