I'm building a desktop app in Tauri (Rust + SvelteKit) and ran into a brutal performance wall when a beta tester tried to import 120,000 files. The app froze for 47 seconds.
The problem was straightforward in hindsight: I was sending all scanned files across Tauri's IPC bridge. ~70MB of JSON each way, UI thread blocked, user assumes crash.
Streaming in chunks helped responsiveness but killed throughput (8s scan → 45s with IPC overhead). The real fix was asking: does the frontend actually need this data?
It didn't. The frontend needed a summary—file counts, type breakdown, detected packages. The actual file list could stay in Rust, referenced by UUID, and never cross the bridge at all.
Result: 8 seconds with live progress, 70MB less memory, and a "send to background" feature so users can keep working during long scans.
The write-up covers the naive approach, the failed streaming attempt, the cache pattern that worked, and a subtle race condition I introduced when implementing background scanning.
1 comment
[ 1.9 ms ] story [ 10.0 ms ] threadThe problem was straightforward in hindsight: I was sending all scanned files across Tauri's IPC bridge. ~70MB of JSON each way, UI thread blocked, user assumes crash.
Streaming in chunks helped responsiveness but killed throughput (8s scan → 45s with IPC overhead). The real fix was asking: does the frontend actually need this data?
It didn't. The frontend needed a summary—file counts, type breakdown, detected packages. The actual file list could stay in Rust, referenced by UUID, and never cross the bridge at all.
Result: 8 seconds with live progress, 70MB less memory, and a "send to background" feature so users can keep working during long scans.
The write-up covers the naive approach, the failed streaming attempt, the cache pattern that worked, and a subtle race condition I introduced when implementing background scanning.