Skip to content

LSL Update#204

Open
jdpigeon wants to merge 11 commits intomainfrom
device-lsl
Open

LSL Update#204
jdpigeon wants to merge 11 commits intomainfrom
device-lsl

Conversation

@jdpigeon
Copy link
Copy Markdown
Contributor

@jdpigeon jdpigeon commented Apr 18, 2026

Summary

Adds Lab Streaming Layer (LSL) support to BrainWaves, enabling:

  • Interop with LabRecorder and any LSL-compatible acquisition/analysis tool
  • Connecting to arbitrary third-party LSL devices alongside Muse and Neurosity (new DEVICES.LSL option)
  • First-class Neurosity Crown support via @neurosity/sdk (previously Muse-only)
  • Synchronized stimulus markers pushed from lab.js experiments to the LSL network

Architecture

  • LSL runs in the main process (src/main/lsl/) using node-labstreaminglayer — native FFI can't run in sandboxed renderers
  • BLE acquisition stays in the renderer via Web Bluetooth (muse-js, @neurosity/sdk) — no noble/native BLE deps required
  • Shared types in src/shared/lslTypes.ts, aliased as @shared in both Vite blocks
  • Outlet manager pushes every connected device to LSL; inlet manager discovers/subscribes to external streams and forwards epochs to the renderer over IPC
  • Epoch batching in the renderer (~8–16 msg/sec per device) keeps IPC overhead negligible
  • We do not use Neurosity's device-side LSL — managing our own outlets keeps stream metadata consistent and avoids duplicates in LabRecorder

Test plan

  • - Connect a real Muse; confirm stream appears in LabRecorder with correct channel names and sample rate
  • - Connect a Neurosity Crown; confirm stream appears in LabRecorder
  • - Run lab.js experiment; confirm markers are recorded synchronized with EEG
  • - Discover and subscribe to an external LSL stream; confirm live visualization
  • - Disconnect Muse mid-session; confirm toast + clean teardown
  • - npm run package on macOS arm64, macOS x64, Windows x64; confirm liblsl loads from the packaged build
  • - Ubuntu smoke test for Web Bluetooth (--enable-experimental-web-platform-features is already set)

jdpigeon and others added 7 commits April 12, 2026 13:45
Deletes cortex.js and emotiv.ts entirely. Removes all Emotiv branches
from device epics, experiment epics, pyodide epics, components, and
constants. DEVICES.EMOTIV, EMOTIV_CHANNELS, parseEmotivSignalQuality,
and Cortex credential env vars are all gone. Muse is now the only
supported device, laying the groundwork for LSL-based connectivity.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Electron 22+ no longer shows a native Bluetooth picker automatically.
Instead it fires select-bluetooth-device on webContents, requiring the
main process to call the callback with a deviceId. Without this handler
requestDevice() hung silently, leaving the search in a perpetual
SEARCHING state.

Changes:
- main/index.ts: register select-bluetooth-device handler that auto-selects
  the first Muse headset as BLE discovery progresses; add bluetooth:cancelSearch
  IPC handler so the renderer can reject a pending requestDevice() on timeout
- preload/index.ts: expose cancelBluetoothSearch() to renderer
- muse.ts: cache BluetoothDevice from getMuse() so connectToMuse() reuses
  it instead of firing a redundant requestDevice() call; add cancelMuseScan()
- deviceEpics.ts: call cancelMuseScan() in searchTimerEpic so the pending
  requestDevice() promise is cleaned up when the 3s search window expires
- docs/device-connectivity.md: full connectivity flow diagram and bug analysis

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
In React 18, setState is always batched — calling setState in an async
componentDidMount continuation schedules a re-render but does not
immediately commit the DOM change. The subsequent querySelector('webview')
therefore returned null, the dom-ready listener was never attached, and
subscribeToObservable was never called.

Fix: defer webview setup to componentDidUpdate, triggered when viewerUrl
transitions from empty to set. At that point React has already committed
the DOM update, so the webview element exists. Because componentDidUpdate
runs synchronously before the browser event loop can process the webview
load, the dom-ready listener is in place before it fires.

This fixes signal not flowing on the Explore EEG screen when navigating
to it while already connected.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds Lab Streaming Layer support so BrainWaves can publish EEG and
stimulus markers as LSL streams and ingest data from external LSL
devices. Enables LabRecorder integration and multi-device experiments.

Phase 1: Main-process LSLOutletManager + IPC bridge forwards batched
Muse EEG samples as an LSL outlet. Adds @shared alias, asarUnpack for
native bindings, and fixes MUSE_CHANNELS hardcoding in experimentEpics.

Phase 2: Neurosity Crown SDK support — getNeurosity/connectToNeurosity
mirror the Muse driver; deviceEpics route by deviceType.

Phase 3: LSLInletManager + UI to discover and connect to external LSL
streams. lslForwardEpic skips LSL inlet sources to avoid feedback loops.

Phase 4: RunComponent emits stimulus markers via sendLSLMarker alongside
the existing injectMuseMarker call, preserving the CSV-embedded marker
path used by the Pyodide analysis pipeline.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Subscribes to lsl:status IPC at the App level and surfaces errors
via react-toastify. Completes Phase 5 production hardening (decimation,
BLE disconnect detection, error surfacing).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@jdpigeon jdpigeon marked this pull request as ready for review April 18, 2026 21:35
@jdpigeon
Copy link
Copy Markdown
Contributor Author

Still need to test with an actual Muse and Crown, but things are looking pretty solid

Copy link
Copy Markdown
Collaborator

@teonbrooks teonbrooks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will give this a full pass at the beginning of next week. added some quick comments and a few questions.

I am liking how this is shaping up!


export interface LSLMarker {
label: string; // e.g. 'stimulus_onset', '1', '2'
rendererTimestamp: number; // performance.now() at event time
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the timestamp is separate from the one that LSL creates, right? is this still to just have as a reference for the timing in the renderer itself?

- Keep `injectMuseMarker` — it embeds markers in the raw EEG CSV, which the existing Pyodide analysis pipeline depends on.

4. **Implement clock sync** (optional, needed if sub-5ms precision required):
- Periodically send a round-trip IPC ping: renderer records `t0 = performance.now()`, main records `lsl_local_clock()`, renderer records `t1`. Offset ≈ `lsl_local_clock() - (t0 + t1) / 2`.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm... why is it (t0 + t1)/2?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this dealing the fact that let's say a stimulus presentation event happens on the main thread, we signal it through the ipc in the renderer to send the LSL marker?

- **Backpressure for high-density inlets**: for 64+ channel streams at 1kHz+, decimate in main before forwarding to renderer. Full-rate stays on LSL network for LabRecorder.
- **Graceful error handling**: BLE disconnects, LSL network loss, inlet timeouts, `node-labstreaminglayer` FFI errors.
- **Platform testing**: macOS arm64, macOS x64, Windows x64. Confirm liblsl binary path resolves correctly post-packaging.
- **Electron packaging verification**: `npm run package`, install the `.dmg`/`.exe`, run with LabRecorder.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would LabRecorder be a separate app or is it incorporated into BrainWaves?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we can add the channel names here and export it. same pattern with neurosity. that way constants stay general

jdpigeon and others added 4 commits April 19, 2026 10:04
node-labstreaminglayer 0.3.0 only ships an x86_64 liblsl.dylib in its
prebuild dir, which fails to load on arm64 Macs. patchDeps.mjs now
detects darwin-arm64 and symlinks the Homebrew-installed framework
binary over the bundled stub. No-op on x64 macs, Linux, and Windows.

Requires: brew install labstreaminglayer/tap/lsl

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two related fixes that together unbreak Pyodide in production:

1. Protocol handler in main was looking at resources/webworker/src/ but
   electron-builder copies pyodide assets to resources/pyodide/. Update
   pyodideRoot to match the actual extraResources destination.

2. Worker was relying on import.meta.url to find pyodide.asm.wasm and
   python_stdlib.zip relative to pyodide.mjs. That works in dev (Vite
   middleware serves siblings from node_modules) but fails in prod where
   the bundled .mjs has no siblings. Set indexURL so pyodide fetches
   runtime files through the pyodide:// protocol handler — works in both.

Verified by installing the packaged dmg and running test plot.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Replace stale port-17173 http-server section with current
  pyodide:// protocol handler reality
- Document the prod resourcesPath/pyodide/ extraResources destination
- Add indexURL requirement for prod (siblings of pyodide.mjs aren't
  bundled, so import.meta.url resolution fails) — gotcha hit during
  packaging verification

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants