feat(spectrum): capture macOS output with a Core Audio process tap

SYNC mode on macOS needed a virtual loopback driver (BlackHole, Loopback,
Soundflower) routed through a Multi-Output Device, and shelled out to
'ffmpeg -f avfoundation' to read it. macOS 14.2 lets a process tap the
system mix directly, so do that instead: CATapDescription ->
AudioHardwareCreateProcessTap -> private aggregate device -> IOProc,
straight through ctypes. No driver, no helper binary, no ffmpeg.

The tap hands out native-rate interleaved float32, so the analyser can
now retune its window and band edges when the capture rate differs from
44.1 kHz, and mono-downmixes the frames before the FFT.

A tap that was never granted System Audio Recording permission is
created happily but only ever delivers digital silence: treat that as a
failed capture instead of flat bars, explain it in the status bar and
fall back to the stream tap. A tap that renders nothing *yet* (playback
still starting, or mpv pointed at another device) is not a permanent
failure either - the stream tap covers for a minute, then the tap is
retried.

Drop the avfoundation device enumeration, the loopback-name heuristics
and the microphone safeguards' tests, and document the new permission
and its diagnostics instead.
This commit is contained in:
beefcarver committed 2026-10-30 22:53:29 -04:00
1 parent 7042707e84
commit 48e480d58c
8 files changed
+988 -282

No files matched your search

+38 -45
View File
@@ -22,10 +22,10 @@ implemented here from the public SomaFM endpoints and open protocols.
spectrum; or a full-height oscilloscope. Three modes (`v`).
- **Synced spectrum, not a guess** — by default the analyser records your
actual audio output, so the bars follow what you hear (`SYNC`): the
PulseAudio/PipeWire monitor via `parec` on Linux, an `ffmpeg`/avfoundation
loopback input (BlackHole, Loopback, Soundflower, …) on macOS. Without an
output capture it falls back to a muted second `ffmpeg` decode of the stream
(`FFT`, can drift), then to a synthesized display (`SIM`).
PulseAudio/PipeWire monitor via `parec` on Linux, and a native Core Audio
process tap on macOS (no driver, no BlackHole, nothing to install). Without
an output capture it falls back to a muted second `ffmpeg` decode of the
stream (`FFT`, can drift), then to a synthesized display (`SIM`).
- **Real artwork on iTerm2, Kitty and WezTerm** — Kitty uses the kitty
graphics protocol; iTerm2 (including on macOS) and WezTerm use their native
iTerm2 inline-image protocol. Images occupy exactly the ON AIR art cell box
@@ -65,11 +65,14 @@ Python **3.10+** and a few command-line audio tools on `$PATH`.
| `python3` | runtime (3.10 or newer; no exact version pin) |
| [`uv`](https://docs.astral.sh/uv/) | creates the venv and installs every Python dependency from `uv.lock` |
| `mpv` | playback and JSON IPC control |
| `ffmpeg` | muted stream decode for the fallback analyser, and the macOS output capture |
| `ffmpeg` | muted stream decode for the fallback analyser |
| `parec` / `pactl` *(Linux)* | synced monitor capture for the spectrum (`pulseaudio-utils`; works with PipeWire) |
| a loopback audio device *(macOS, optional)* | synced output capture for the spectrum: BlackHole, Loopback, Soundflower, … |
| D-Bus session bus *(Linux)* | MPRIS desktop media integration (`playerctl` is handy for testing) |
On macOS the spectrum capture is part of the app: it uses the Core Audio
process-tap API (macOS 14.2 or newer), so there is no loopback driver to
install.
On macOS the media integration needs nothing extra: mpv is what the system's
Now Playing session (Control Center, media keys) talks to.
@@ -110,33 +113,20 @@ the synced spectrum needs `parec`/`pactl`.
brew install mpv ffmpeg uv # ffplay ships with ffmpeg
```
That is enough to run: playback, artwork in iTerm2/Kitty/WezTerm, the muted
`ffmpeg` spectrum tap and the macOS Now Playing integration all work out of the
box. The first capture attempt makes macOS ask for **microphone** access for
whatever terminal you use — that permission is what lets `ffmpeg` read the
loopback device.
That is enough to run: playback, artwork in iTerm2/Kitty/WezTerm, the synced
Core Audio spectrum tap and the macOS Now Playing integration all work out of
the box.
For a *synced* spectrum (`SYNC` instead of `FFT`) macOS needs an output
loopback, because CoreAudio does not let an ordinary process record the
speakers directly:
The first time the analyser starts it asks macOS for **System Audio
Recording** permission — a one-off prompt for your terminal (iTerm2, Kitty,
WezTerm, …). Approve it under *System Settings → Privacy & Security → System
Audio Recording*; no loopback driver, Multi-Output Device or audio routing is
needed. If the permission is denied the tap only ever hands out silence, the
analyser notices and falls back to `FFT` instead of showing dead bars.
```bash
brew install blackhole-2ch # or Loopback / Soundflower
```
Then either
1. create a **Multi-Output Device** in *Audio MIDI Setup* containing both your
speakers/headphones and *BlackHole 2ch*, and select it as the system output,
or
2. set *BlackHole 2ch* as the output and monitor the audio elsewhere.
The app auto-detects loopback-style inputs (BlackHole, Loopback, Soundflower,
“Multi-Output”, “Aggregate”) and never grabs a plain microphone by accident.
Pick one explicitly with `SOMAFM_SPECTRUM_DEVICE` (name or avfoundation index,
see `ffmpeg -f avfoundation -list_devices true -i ""`). If the loopback turns
out to be silent (nothing routed into it), the analyser notices after a few
seconds and falls back to `FFT` instead of showing dead bars.
The tap records the **default output device**, which is also what mpv uses; if
you point mpv elsewhere with `SOMAFM_MPV_ARGS='--audio-device=...'` the
analyser falls back to `FFT`.
Terminal image rendering uses [Pillow](https://python-pillow.org/) and is
implemented directly (Kitty TGP + iTerm2 OSC 1337) — no terminal probing.
@@ -248,12 +238,16 @@ spectrum and the station list stay in sync. The status bar shows
engine, and uses `SIGSTOP`/`SIGCONT` when falling back to `ffplay`. On macOS
it also asks mpv to publish Now Playing and maps the media keys onto
`script-message`s that come back over the same IPC socket.
- `somafm_tui/spectrum.py` — 44.1 kHz mono s16le analyser. Prefers capturing
the real output: `parec` (PulseAudio/PipeWire monitor) on Linux, or
`ffmpeg -f avfoundation` on a loopback input on macOS (SYNC). Otherwise a
- `somafm_tui/spectrum.py` — analyser (44.1 kHz mono s16le from `parec`, or
the device's native float32 rate from the macOS tap). Prefers capturing the
real output: `parec` (PulseAudio/PipeWire monitor) on Linux, or a native
Core Audio process tap on macOS (SYNC, see `macos_tap.py`). Otherwise a
muted low-latency `ffmpeg` decode of the stream (FFT). Hann-windowed
2048-point FFT, log band grouping, auto-gain, fast attack/decay smoothing,
peak hold and waveform downsampling.
- `somafm_tui/macos_tap.py` — the macOS output capture, in pure `ctypes`
against Core Audio: `CATapDescription` → `AudioHardwareCreateProcessTap` →
private aggregate device → IOProc. No driver, no helper binary, no ffmpeg.
- `somafm_tui/mediasession.py` — picks the desktop media backend (MPRIS on
Linux, Now Playing on macOS) so the app has one platform-neutral interface.
- `somafm_tui/mpris.py` — `dbus-next` MPRIS service in its own thread with a
@@ -303,11 +297,11 @@ same dotfiles/config work everywhere; `XDG_CONFIG_HOME`, `XDG_CACHE_HOME` and
either, the UI still runs and tells you. Extra mpv args can be injected with
`SOMAFM_MPV_ARGS` (e.g. `SOMAFM_MPV_ARGS='--audio-device=...'`, which is the
macOS way to choose an output device).
- **Spectrum says `FFT` instead of `SYNC`?** `parec`/`pactl` are unavailable,
the monitor device failed, or you forced the stream tap. The bars are still
real audio, but the second stream can drift relative to playback. Use
`--spectrum-source monitor` to force sync, or `SOMAFM_SPECTRUM_DEVICE=<source>`
to pick a specific monitor/source. `SIM` means the bars are synthesized.
- **Spectrum says `FFT` instead of `SYNC`?** The monitor is unavailable or
failed, or you forced the stream tap. The bars are still real audio, but the
second stream can drift relative to playback. Use `--spectrum-source
monitor` to force sync, or `SOMAFM_SPECTRUM_DEVICE=<source>` on Linux to pick
a specific monitor. `SIM` means the bars are synthesized.
- **Diagnose artwork:** `uv run somafm-tui --diagnose-images` prints the
detected protocol, widget class, terminal env, dependency versions and cache
path. If `protocol` is `None`, force it with `SOMAFM_IMAGES=tgp` (Kitty) or
@@ -315,12 +309,11 @@ same dotfiles/config work everywhere; `XDG_CONFIG_HOME`, `XDG_CACHE_HOME` and
- **`SYNC` but the bars are flat?** mpv may be playing to a different sink (or
directly to ALSA). Point `SOMAFM_SPECTRUM_DEVICE` at that sink's monitor, or
use `--spectrum-source stream`.
- **macOS: spectrum stays `FFT`?** Install a loopback (BlackHole/Loopback) and
route your speakers through a Multi-Output Device, then check
`ffmpeg -f avfoundation -list_devices true -i ""` lists it. The first capture
triggers a macOS microphone-permission prompt; grant it to your terminal
(System Settings → Privacy & Security → Microphone), otherwise ffmpeg cannot
open the device and the analyser stays on `FFT`.
- **macOS: spectrum stays `FFT`?** Grant your terminal **System Audio
Recording** permission (*System Settings → Privacy & Security → System Audio
Recording*), then restart the app; the status bar explains when the tap is
silent because of this. Also check that mpv is playing to the default output
device, since that is what the tap records.
- **macOS: media keys do nothing?** Another app currently owns the Now Playing
session, or the app was started with `--no-mpris`, or the audio backend fell
back to `ffplay` (mpv is required for Now Playing). The status bar shows