CJK glyphs missing on desktop (Windows/Linux/macOS)

[salmon-21]

Description

Non-Latin characters (Japanese, Chinese, Korean) display as ? on desktop platforms. The embedded font PlexSansIcon only covers Latin-1 and icon glyphs.

The Web build bundles LXGW NeoXiHei-Code for CJK coverage (4c262a1), but desktop clientPlatformLocateFontBinary() implementations are stubs.

Note on current Web font

LXGW NeoXiHei uses Simplified Chinese (G-standard) glyph forms — from its README:

将日本写法的字体改造成中国大陆规范写法的尝试

This means shared CJK characters (e.g. 直, 骨, 令) display as Chinese variants, which look incorrect to Japanese users.

Recommended approach

The industry standard for multi-CJK-language applications is locale-based font variant selection:

1. Bundle region-specific font variants (e.g. Noto Sans CJK JP/SC/TC/KR, ~8MB each)
2. Detect OS locale at startup
3. Load the matching variant as a fallback font via ImFontConfig::MergeMode

Locale	Font variant	Default glyph forms
ja	Noto Sans CJK JP	Japanese (JIS)
zh-CN	Noto Sans CJK SC	Simplified Chinese (GB)
zh-TW, zh-HK	Noto Sans CJK TC	Traditional Chinese
ko	Noto Sans CJK KR	Korean
Other	Noto Sans CJK SC	Simplified Chinese (fallback)

Size considerations

Bundling all 4 variants adds ~32MB. Alternatives:

• Bundle only SC + JP (~16MB) to cover the most likely user bases
• Use system fonts where available and bundle as fallback
• Ship a "thin" build and download the font on first launch

Infrastructure already in place

• clientPlatformLocateFontBinary() API in Platform.hpp
• Merge-mode font loading in SDLMain.cpp (loads once per session)
• Web build already demonstrates the pattern with LXGW NeoXiHei-Code

🤖 Generated with Claude Code

[mos9527]

Sorry for closing this prematurely by accident. So for context:

• We do embedded a font in binaries, namely PlexSansIcons (which contrary to the sans suffix, it does contain icons used throughout the UI)
• We are not bundling NeoXiHei, for the same reason where not a lot of people bundles files for cross-platform binary builds.
  • MSVC absolutely despises us if we'd also embed it with https://github.com/mos9527/SonyHeadphonesClient/blob/rewrite/tooling/BinaryEmbed.cpp. It crashes with very scary error logs.
  • GCC/Clang probably plays along, haven't tested those.
  • For Windows, the good old trick of using overlay works. But what about the other platforms?
  • Maybe we can tell the linker to explicitly pack a section of font data for us, but what about Web or the fact that we'd parse PE/ELF/MachO binaries ourselves?

So bundling font binaries that large is by no means portable. As for downloading fonts.. well, the web does it because it's web. Downloading font files natively is by and large another nightmare that solves very little problems.

And like what I've said in the comments, locating fonts automatically is, hell. We used to have an option for users to specify font files manually, though that's barely a solution at all :(

Thanks for bringing this up again though - I'll certainly look into more solutions in the coming days.

[salmon-21]

Reference: How ImHex solves the same problem

ImHex (same ImGui + SDL + CMake architecture) uses libromfs to embed font files into the binary:

• Packs files into the .rodata section of a static library (not C array generation — avoids the MSVC crash issue with BinaryEmbed.cpp)
• Cross-platform: Windows/Linux/macOS/Emscripten all work
• Access via romfs::get("fonts/...") at runtime
• CMake integration: just add_subdirectory(libromfs) and target_link_libraries

ImHex's font loading flow

From font_loader.cpp:

• L74: Bundled fallback via romfs — romfs::get("fonts/JetBrainsMono.ttf")
• L64: User custom font — AddFontFromFileTTF(fontPath, ...)
• L88: Merge additional fonts (CJK, icons, etc.)

Also relevant: ImGui v1.92 dynamic glyph loading

Since this project uses ImGui v1.92.4, glyph ranges like GetGlyphRangesJapanese() are no longer needed — glyphs load on-demand (ImGui FONTS.md, v1.92.0 release notes). Just loading the font file is sufficient for full CJK coverage.

Possible approach for this project

1. Adopt libromfs (or similar .rodata embedding) to replace BinaryEmbed.cpp
2. Bundle locale-specific Noto Sans CJK variants in romfs/fonts/
3. Detect OS locale at startup → load matching variant via MergeMode
4. Same mechanism works for Emscripten (replaces current NeoXiHei bundling)

🤖 Generated with Claude Code

[mos9527]

Your Claude is right that libromfs doesn't just emit C arrays - they used std::array instead ;)

And for what it's worth - none of the font files they embed is even over 1MB. I'd say confidently that the embded-into-array-and-hope-the-compiler-doesn't-die approach is not very feasible for our 8MB (which is one of the smallest CJK-compatible fonts) blob.

They do use fontconfig, which is something I've laid my eyes on as well. I'd personally move forward with that should native builds be concerned. More inputs are still welcome though!

[salmon-21]

Cross-platform system font discovery for CJK

Since bundling large font files into the binary is not practical, here's how each platform can locate system CJK fonts:

Linux: fontconfig

FcPattern *pat = FcPatternCreate();
FcPatternAddString(pat, FC_FAMILY, "sans-serif");
FcPatternAddString(pat, FC_LANG, "ja");  // or "zh-cn", "ko"
FcConfigSubstitute(NULL, pat, FcMatchPattern);
FcDefaultSubstitute(pat);
FcResult result;
FcPattern *match = FcFontMatch(NULL, pat, &result);

Windows: hardcoded paths or DirectWrite

Check known fonts by locale — these are always present on the corresponding CJK-locale Windows:

Locale	Font	Path	Availability
JP	Yu Gothic UI	C:\Windows\Fonts\YuGothM.ttc	Win 10+
JP	Meiryo	C:\Windows\Fonts\meiryo.ttc	Vista+
JP	MS Gothic	C:\Windows\Fonts\msgothic.ttc	All
ZH-CN	Microsoft YaHei	C:\Windows\Fonts\msyh.ttc	Vista+
ZH-CN	SimSun	C:\Windows\Fonts\simsun.ttc	All
ZH-TW	Microsoft JhengHei	C:\Windows\Fonts\msjh.ttc	Vista+
KO	Malgun Gothic	C:\Windows\Fonts\malgun.ttf	Vista+

Or use DirectWrite IDWriteFontCollection for proper enumeration.

macOS: CoreText

CTFontRef font = CTFontCreateWithName(CFSTR("Hiragino Sans"), 0, NULL);

System CJK fonts: Hiragino Sans (JP), PingFang SC/TC (ZH), Apple SD Gothic Neo (KR). Always preinstalled.

Locale detection

SDL3 provides SDL_GetPreferredLocales() — returns the user's locale list, useful for selecting the right font family/variant across all platforms.

Web: the remaining gap

With native platforms using system fonts, the Web build would still be stuck with the bundled NeoXiHei (Chinese G-standard glyph forms). Local Font Access API exists but is Chromium-only.

A practical solution: fetch from Google Fonts CDN at runtime, selecting the locale-appropriate Noto Sans CJK variant based on navigator.language:

navigator.language	Font variant
ja	Noto Sans CJK JP
zh-CN	Noto Sans CJK SC
zh-TW, zh-HK	Noto Sans CJK TC
ko	Noto Sans CJK KR
other	Noto Sans CJK SC (current user base)

This adds no bundle size — the font loads over the network while the WASM module initializes. Each user gets the correct glyph forms for their locale.

The fetched font can be cached client-side via the Cache API or IndexedDB, so subsequent visits load instantly without network access. Google Fonts responses also include HTTP cache headers, so the browser's built-in cache will handle the common case automatically.

Reference implementations

• Ghostty: fontconfig on Linux, CoreText on macOS
• wezterm: fontconfig/DirectWrite/CoreText
• servo/font-kit: cross-platform font loading library (Rust, good API reference)

🤖 Generated with Claude Code