On Book

Appearance

Audio System Architecture ​

Howler.js playback and Freesound integration for theatrical sound design.

Overview ​

On Book Pro's audio system provides:

  • Sound Library — Searchable collection of clips
  • Soundboard — Grid-based pads for instant SFX
  • Cue List — Sequenced playback for performances
  • Freesound Integration — Search and import public domain audio

Core Components ​

┌─────────────────────────────────────────────────────────┐
│                    Sound Feature                         │
├─────────────┬─────────────┬─────────────┬───────────────┤
│  Library    │  Soundboard │  Cue List   │  Freesound    │
│  (clips)    │  (pads)     │  (sequence) │  (search)     │
└──────┬──────┴──────┬──────┴──────┬──────┴───────┬───────┘
       │             │             │              │
       └─────────────┴─────────────┴──────────────┘
                          │
                   ┌──────▼──────┐
                   │  Howler.js  │
                   │  (playback) │
                   └──────┬──────┘
                          │
                   ┌──────▼──────┐
                   │  IndexedDB  │
                   │  (cache)    │
                   └─────────────┘

Sound Clip Lifecycle ​

1. Import from Freesound ​

typescript
// Search Freesound API
const searchFreesound = async (query: string) => {
    const response = await fetch(
        `https://freesound.org/apiv2/search/text/?query=${query}&token=${API_KEY}`
    );
    return response.json();
};

// Preview a sound (streaming)
const previewSound = (previewUrl: string) => {
    const howl = new Howl({
        src: [previewUrl],
        html5: true, // Stream without downloading
    });
    howl.play();
};

2. Download and Cache Locally ​

typescript
// Download full audio file
const downloadClip = async (freesoundId: number): Promise<Blob> => {
    const response = await fetch(
        `https://freesound.org/apiv2/sounds/${freesoundId}/download/`,
        { headers: { Authorization: `Bearer ${token}` } }
    );
    return response.blob();
};

// Store in IndexedDB
const cacheLocally = async (clipId: string, blob: Blob) => {
    const db = await openDB('sound-cache', 1);
    await db.put('blobs', blob, `sound_blob_${clipId}`);
};

3. Playback from Cache ​

typescript
// Load from IndexedDB
const loadFromCache = async (clipId: string): Promise<string> => {
    const db = await openDB('sound-cache', 1);
    const blob = await db.get('blobs', `sound_blob_${clipId}`);
    return URL.createObjectURL(blob);
};

// Create Howl instance
const createHowl = async (clip: SoundClip) => {
    const url = clip.localKey
        ? await loadFromCache(clip.id)
        : clip.previewUrl;

    return new Howl({
        src: [url],
        volume: clip.volume,
        loop: clip.loop,
        sprite: clip.trimEnd ? {
            trimmed: [clip.trimStart * 1000, (clip.trimEnd - clip.trimStart) * 1000]
        } : undefined,
    });
};

Waveform Visualization ​

The Sound feature includes a waveform visualizer for trim point selection:

typescript
// src/features/sound/waveform-visualizer.ts

export const drawWaveform = (
    canvas: HTMLCanvasElement,
    audioBuffer: AudioBuffer,
    options: WaveformOptions
) => {
    const ctx = canvas.getContext('2d');
    const data = audioBuffer.getChannelData(0);
    const step = Math.ceil(data.length / canvas.width);
    
    ctx.clearRect(0, 0, canvas.width, canvas.height);
    ctx.fillStyle = options.color || '#3b82f6';
    
    for (let i = 0; i < canvas.width; i++) {
        const min = Math.min(...data.slice(i * step, (i + 1) * step));
        const max = Math.max(...data.slice(i * step, (i + 1) * step));
        
        const y = ((1 + min) / 2) * canvas.height;
        const height = Math.max(1, ((max - min) / 2) * canvas.height);
        
        ctx.fillRect(i, y, 1, height);
    }
};

Soundboard Pad System ​

typescript
interface SoundPad {
    id: string;
    clipId: string;
    position: number;    // Grid position (0-15 for 4x4)
    color: string;
    label?: string;
}

// Trigger pad playback
const triggerPad = (pad: SoundPad) => {
    const clip = clips.find(c => c.id === pad.clipId);
    if (clip) {
        const howl = howlInstances.get(clip.id);
        howl?.play();
    }
};

IndexedDB Storage Pattern ​

typescript
// Sound-specific IndexedDB schema
import { openDB } from 'idb';

const initSoundDB = async () => {
    return openDB('sound-cache', 1, {
        upgrade(db) {
            // Store audio blobs
            if (!db.objectStoreNames.contains('blobs')) {
                db.createObjectStore('blobs');
            }
            // Store waveform data (pre-computed)
            if (!db.objectStoreNames.contains('waveforms')) {
                db.createObjectStore('waveforms');
            }
        },
    });
};

Freesound API Integration ​

Authentication ​

Freesound uses OAuth2. Store the API key in Firebase Functions:

typescript
// Firebase Function (server-side)
export const searchFreesound = onCall(async (request) => {
    const apiKey = process.env.FREESOUND_API_KEY;
    
    const response = await fetch(
        `https://freesound.org/apiv2/search/text/?query=${request.data.query}`,
        { headers: { Authorization: `Token ${apiKey}` } }
    );
    
    return response.json();
});

Client Usage ​

typescript
import { httpsCallable } from 'firebase/functions';

const searchFreesound = httpsCallable(functions, 'searchFreesound');

const results = await searchFreesound({ query: 'thunder' });

Best Practices ​

  1. Preload critical sounds — Load soundboard pads on tab mount
  2. Use IndexedDB for offline — Cache all imported clips locally
  3. Dispose Howl instances — Call unload() when unmounting
  4. Throttle waveform rendering — Don't redraw on every frame

Further Reading ​

Last updated: January 2026