行业动态
developers.cloudflare.com ·
@Cloudflare Developers
Cloudflare AI Search 关键词检索新增中日韩查询支持
Cloudflare AI Search 的 trigram 分词器现已支持日语、中文和韩语查询，包括「経費精算の締め日はいつですか？」这类无空格的自然语言问句；此前无空格查询会被当作单一精确短语，通常返回不到关键词结果。官方指出 porter 分词器对这些语言效果不佳，建议中日韩内容将 keyword_tokenizer 设为 trigram，但更换分词器会触发全量重建索引。
AI推荐理由 Cloudflare AI Search 官方 changelog，明确说明 trigram 分词器现可匹配中日韩查询（含无空格的自然语言问句），并指出此前无空格查询被当作单一精确短语导致通常无结果、porter 分词器不适合这些语言、切换分词器会触发全量重建索引。属于可验证的产品能力更新，对多语言检索与 RAG 应用有直接参考价值；局限是仅覆盖关键词检索分词层面，未给出召回率等量化效果。
72
正文
[AI Search](https://developers.cloudflare.com/ai-search/) keyword search with the `trigram` tokenizer now matches Japanese, Chinese, and Korean queries, including full natural-language questions such as `経費精算の締め日はいつですか？`. Previously, a query without spaces was treated as a single exact phrase and usually returned no keyword results.
The `porter` tokenizer does not work well for these languages. If your content is in Japanese, Chinese, or Korean, set `keyword_tokenizer` to `trigram`. Changing the tokenizer triggers a full reindex.
Refer to [Keyword search](https://developers.cloudflare.com/ai-search/configuration/indexing/keyword-search/#keyword-tokenizer) for tokenizer details.