monster/Godstrument
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

🌐 feed a tab into the world — external audio via getDisplayMedia

Browse Source 
Right-click sky → "🌐 feed a tab": getDisplayMedia({video:true, audio:{…}})
lets the user share another tab's audio (a YouTube tab, a stream). Wired via
createMediaStreamSource into preSat → the full FX chain + master, so it's
audible live, crush-able, and resampleable by GODSONIQ from one wiring. Video
track stopped immediately; a track 'ended' listener handles Stop-sharing.

Audio constraints (verified against MDN/W3C/Chrome docs): suppressLocalAudio
Playback so the tab isn't heard twice, and echoCancellation/noiseSuppression/
autoGainControl off so the WebRTC voice-DSP doesn't mangle music. Chromium-only
(feature-detected; graceful ticker hints for unsupported / no-audio-track);
macOS grabs a tab not all-system; DRM services capture silent, YouTube fine.

Grimoire (truth) + manual Appendix J updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
type-two 2026-07-10 09:38:33 +10:00
parent 5ffa3a4bfb
commit 78718e9459
2 changed files with 50 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
GODSTRUMENT_MANUAL_SOURCE.md
View File

@ -588,6 +588,8 @@ The matrix pours out clean control values whether or not anyone is listening. Ch
**🎛 GODSONIQ — resample the world onto the keys.** A cheeky nod to the Ensoniq ASR-10: right-click the sky → **🎛 GODSONIQ** and it records ~3 seconds of the *live* master mix — whatever the world is playing right now, this exact never-repeating take — into a buffer. From then on every pluck **replays that capture, pitched across the keyboard** (middle C plays it at speed, an octave up plays it twice as fast). It runs through the full FX chain, so route `crush` at it and you get the classic 12-bit sampler grit. Right-click → 🎛 again to grab a fresh slice; **🎸 string voice** (or toggling it off) drops you back to the FM tine. The browser-native bounce: freeze a moment of the planet, then play the planet like an instrument.
**🌐 Feed a tab into the world.** Right-click the sky → **🌐 feed a tab** and the browser's share-picker opens — choose another tab (a YouTube video, a live radio stream, a Bandcamp page), tick **“Share tab audio,”** and that sound pours straight into Godstrument's FX chain. It's *in the world* now: route `crush` to grit it, drench it in the world's reverb, or hit **🎛 GODSONIQ** to resample it and play it pitched across the keys. Click 🌐 again (or the browser's *Stop sharing*) to cut it. **The fine print:** this is a **Chrome / Edge** trick (Safari and Firefox can't capture tab audio at all); on a Mac it grabs one *tab's* sound, not the whole system (for that you'd route audio through a virtual device like BlackHole); and DRM'd services (Netflix, Spotify, Apple Music) usually come through silent — ordinary YouTube is fine.
**Zero — the moods, the gentle front door.** On your very first visit the instrument greets you with **🌅 first light** instead of the full flood: a bass line, soft hats, a small melody, modest space — a thing shaped like a song, with the world still playing underneath it. Right-click the sky → **🌗 mood** to move between *first light*, *ambient drift* (no drums, all atmosphere, the planet as weather), and **the full storm** — the factory instrument with every one of the world's hands on it at once. Ease in, then open the door.
**One — the built-in browser synth.** Click **♪** and a full voice rack wakes inside the page: **lead** (bitcoin's price and your market's average price both quantize to it — minor pentatonic and G dorian), **bass** (quake depth chooses its notes in a low minor), **pad** (brightened by Tokyo's warmth, daylight, harmony, the birth rate, longer lives), **drone** (thickened by aircraft aloft, barometric pressure, Saturn's slow wheel, the weight of the national debt), and **perc** (BTC volatility and market activity drive the hats). Over them sit the effects the world modulates: **reverb** (earthquakes open the void, room light sizes it), **delay** (your hand's Y feeds it, Mercury retrograde smears it), **saturation** (Delhi's air becomes grit, hard aspects add dissonance, wildfires burn it, hunger grinds beneath), and **glitch** (every Wikipedia edit on Earth, every light flash, every new wildfire fires a burst). Nothing to install; the planet plays a synth in your tab.
@ -1019,3 +1021,8 @@ Every feed the world sends is one of eight temperaments; the shape dictates how
- **🌊 Juno-106 chorus** — an always-on subtle BBD-style stereo chorus on the synth's master bus (two short delays warbled by slow LFOs, panned wide) that warms and widens the dry oscillators.
- **🔊 12-bit bitcrush** — a world-drivable lo-fi stage on the synth's master bus (an AudioWorklet doing true bit-depth quantise + sample-rate reduction). Route anything to `crush` and the planet degrades its own DAC — clean by default (16-bit).
- **🎛 GODSONIQ (resampler)** — a cheeky nod to the Ensoniq ASR-10: right-click the sky → *GODSONIQ* records ~3s of the live master mix (a recorder AudioWorklet taps the full post-FX bus) into a buffer, then every pluck replays that capture pitched across the keys (playbackRate = 2^((midi-60)/12), middle C = unity). Plays through the full FX chain, so `crush` gives it the classic 12-bit sampler grit. 🎸 string voice exits back to the FM tine. The browser-native bounce — capture the world's never-repeating take, then play it.
- **⬗ Parameter locks (p-locks)** — Pocket-Operator / Elektron style: a texture voice (filter, glitch, reverb, drive) gets a per-step `lock` lane that pins its value to a fixed level on that step (blocky, disjointed), overriding the world; drag a step below the lane to unlock it and hand that step back to the data. Applied in `computeDests` via `applyGroove`; travels with the patch.
- **🌊 Juno-106 chorus** — an always-on subtle BBD-style stereo chorus on the synth's master bus (two short delays warbled by slow LFOs, panned wide) that warms and widens the dry oscillators.
- **🔊 12-bit bitcrush** — a world-drivable lo-fi stage on the synth's master bus (an AudioWorklet doing true bit-depth quantise + sample-rate reduction). Route anything to `crush` and the planet degrades its own DAC — clean by default (16-bit).
- **🎛 GODSONIQ (resampler)** — a cheeky nod to the Ensoniq ASR-10: right-click the sky → *GODSONIQ* records ~3s of the live master mix (a recorder AudioWorklet taps the full post-FX bus) into a buffer, then every pluck replays that capture pitched across the keys (playbackRate = 2^((midi-60)/12), middle C = unity). Plays through the full FX chain, so `crush` gives it the classic 12-bit sampler grit. 🎸 string voice exits back to the FM tine. The browser-native bounce — capture the world's never-repeating take, then play it.
- **🌐 Tab feed (external audio in)** — right-click the sky → *feed a tab* calls `getDisplayMedia({video:true, audio:{...}})`; the user picks another browser tab and ticks 'Share tab audio', and its sound (a YouTube tab, a live stream) is wired via `createMediaStreamSource` into `preSat` → the full FX chain and master, so it's audible live, `crush`-able, and resampleable by GODSONIQ. Constraints disable echoCancellation/noiseSuppression/autoGainControl (voice DSP that mangles music) and set suppressLocalAudioPlayback (no double audio). Chromium-only (Chrome/Edge; Safari & Firefox don't deliver display audio); on macOS it captures a TAB not all-system (BlackHole/Loopback is the system-wide workaround); DRM'd services (Netflix/Spotify/Apple Music) usually capture silent, plain YouTube is fine. Video track is stopped immediately; a track 'ended' listener handles the browser's Stop-sharing.

44
viz/index.html
View File

@ -1020,6 +1020,8 @@
<p><b>🎛 GODSONIQ — resample the world onto the keys.</b> A cheeky nod to the Ensoniq ASR-10: right-click the sky → <b>🎛 GODSONIQ</b> and it records ~3 seconds of the <em>live</em> master mix — whatever the world is playing right now, this exact never-repeating take — into a buffer. From then on every pluck <b>replays that capture, pitched across the keyboard</b> (middle C plays it at speed, an octave up plays it twice as fast). It runs through the full FX chain, so route <kbd>crush</kbd> at it and you get the classic 12-bit sampler grit. Right-click → 🎛 again to grab a fresh slice; <b>🎸 string voice</b> (or toggling it off) drops you back to the FM tine. The browser-native bounce: freeze a moment of the planet, then play the planet like an instrument.</p>
<p><b>🌐 Feed a tab into the world.</b> Right-click the sky → <b>🌐 feed a tab</b> and the browser's share-picker opens — choose another tab (a YouTube video, a live radio stream, a Bandcamp page), tick <b>“Share tab audio,”</b> and that sound pours straight into Godstrument's FX chain. It's <em>in the world</em> now: route <kbd>crush</kbd> to grit it, drench it in the world's reverb, or hit <b>🎛 GODSONIQ</b> to resample it and play it pitched across the keys. Click 🌐 again (or the browser's <i>Stop sharing</i>) to cut it. <b>The fine print:</b> this is a <b>Chrome / Edge</b> trick (Safari and Firefox can't capture tab audio at all); on a Mac it grabs one <em>tab's</em> sound, not the whole system (for that you'd route audio through a virtual device like BlackHole); and DRM'd services (Netflix, Spotify, Apple Music) usually come through silent — ordinary YouTube is fine.</p>
<p><b>Zero — the moods, the gentle front door.</b> On your very first visit the instrument greets you with <b>🌅 first light</b> instead of the full flood: a bass line, soft hats, a small melody, modest space — a thing shaped like a song, with the world still playing underneath it. Right-click the sky → <b>🌗 mood</b> to move between <i>first light</i>, <i>ambient drift</i> (no drums, all atmosphere, the planet as weather), and <b>the full storm</b> — the factory instrument with every one of the world's hands on it at once. Ease in, then open the door.</p>
<p><b>One — the built-in browser synth.</b> Click <b>♪</b> and a full voice rack wakes inside the page: <b>lead</b> (bitcoin's price and your market's average price both quantize to it — minor pentatonic and G dorian), <b>bass</b> (quake depth chooses its notes in a low minor), <b>pad</b> (brightened by Tokyo's warmth, daylight, harmony, the birth rate, longer lives), <b>drone</b> (thickened by aircraft aloft, barometric pressure, Saturn's slow wheel, the weight of the national debt), and <b>perc</b> (BTC volatility and market activity drive the hats). Over them sit the effects the world modulates: <b>reverb</b> (earthquakes open the void, room light sizes it), <b>delay</b> (your hand's Y feeds it, Mercury retrograde smears it), <b>saturation</b> (Delhi's air becomes grit, hard aspects add dissonance, wildfires burn it, hunger grinds beneath), and <b>glitch</b> (every Wikipedia edit on Earth, every light flash, every new wildfire fires a burst). Nothing to install; the planet plays a synth in your tab.</p>
@ -5542,6 +5544,14 @@
() => { const ok = Synth.godsoniq(3);
eventTicker.unshift({ text: ok ? "🎛 GODSONIQ — sampling the world for 3s… then every pluck replays it, pitched across the keys (🎸 string voice to exit)"
: "🎛 GODSONIQ needs the audio worklet — press ♪ first, then try again", tAdded: now(), src: "sys" }); });
ctxItem("🌐 " + (Synth.feeding() ? "tab feed: live — click to stop" : "feed a tab (YouTube, a stream…) into the world"),
async () => { const r = await Synth.feedTab();
const msg = r.stopped ? "🌐 tab feed stopped"
: r.ok ? "🌐 tab feed is live — it's in the world now: route crush for lo-fi, or 🎛 GODSONIQ to resample it pitched"
: r.reason === "unsupported" ? "🌐 tab-audio capture needs Chrome or Edge (Safari & Firefox can't). On a Mac it grabs one TAB's sound, not all-system"
: r.reason === "no-audio" ? "🌐 pick the tab AND tick “Share tab audio” in the picker — no audio came through"
: "🌐 tab feed cancelled";
eventTicker.unshift({ text: msg, tAdded: now(), src: "sys" }); });
ctxItem("⌨ keys — remap the keyboard", () => openKeys());
if (commune.room) {
const gOwner = commune.alloc["@godtime"];
@ -6977,6 +6987,37 @@
recNode.port.postMessage({ rec: true, len: Math.floor((secs || 3) * ctx.sampleRate) });
return true;
}
// 🌐 external tab feed — pull another tab's audio (a YouTube tab, a live stream) into the
// world via getDisplayMedia. Wired into preSat, so it flows the whole FX chain to the master:
// audible live AND resampleable by GODSONIQ (which taps N.out). Chrome/Edge only; on a Mac the
// picker grabs a TAB's audio (tick "Share tab audio"), not all-system (that needs a virtual device).
let extStream = null, extSrc = null, extGain = null;
function stopFeed() {
if (extStream) extStream.getTracks().forEach(t => t.stop());
if (extSrc) { try { extSrc.disconnect(); } catch (e) {} }
if (extGain) { try { extGain.disconnect(); } catch (e) {} }
extStream = extSrc = extGain = null;
}
async function feedTab() {
const md = navigator.mediaDevices;
if (!md || !md.getDisplayMedia) return { ok: false, reason: "unsupported" };
if (extStream) { stopFeed(); return { ok: true, stopped: true }; } // second click = stop
if (!ctx) { ctx = new AC(); N = build(); loadWorklets(); }
let stream;
try { // video:true is mandatory to get tab audio in Chromium; kill the voice-DSP that mangles music, and mute the tab's own speaker output so we don't hear it twice
stream = await md.getDisplayMedia({ video: true, audio: { echoCancellation: false, noiseSuppression: false, autoGainControl: false, suppressLocalAudioPlayback: true } });
} catch (e) { return { ok: false, reason: "cancelled" }; }
const aud = stream.getAudioTracks();
if (!aud.length) { stream.getTracks().forEach(t => t.stop()); return { ok: false, reason: "no-audio" }; } // user forgot "Share tab audio"
stream.getVideoTracks().forEach(t => t.stop()); // we only want the sound
extStream = stream;
extSrc = ctx.createMediaStreamSource(stream);
extGain = ctx.createGain(); extGain.gain.value = 0.9;
extSrc.connect(extGain); extGain.connect(N.preSat); // into the FX chain → master
if (!on) start();
aud[0].addEventListener("ended", stopFeed); // browser's "Stop sharing" button
return { ok: true };
}
function start() {
if (!AC) return;
if (!ctx) { ctx = new AC(); N = build(); loadWorklets(); }
@ -7053,7 +7094,8 @@
return { update(dests) { if (dests) Object.assign(D, dests); }, pluck,
toggle() { on ? stop() : start(); paint(); return on; }, playing() { return on; },
stringVoice(v) { if (v === undefined) return ksMode; if (!ctx) start(); ksMode = !!v; sampleMode = false; return ksMode; },
godsoniq(secs) { return godsoniq(secs); }, sampling() { return sampleMode && !!sampleBuf; } };
godsoniq(secs) { return godsoniq(secs); }, sampling() { return sampleMode && !!sampleBuf; },
feedTab() { return feedTab(); }, feeding() { return !!extStream; } };
})();
// ---------------------------------------------------------------------------