diff --git a/spec/numbers.md b/spec/numbers.md index facaeb1d4..1f6a3694e 100644 --- a/spec/numbers.md +++ b/spec/numbers.md @@ -468,3 +468,18 @@ | 三日間 | 3일간 | 유지 | 사례: `tests/rules/cases/laugh-legs-month-one.tsv` + +## NUM-34 수 뒤에 홀로 쓴 コ는 조수사 `개`다 + +수(아라비아 숫자, 한자 수, 何, 数) 뒤에 가타카나 コ 한 글자만 오면 個를 가타카나로 쓴 조수사라서 정규화 단계가 個로 읽고 `개`다(メール784コっす 메일 784개예요, +三コください 세 개 주세요, 何コ 몇 개). コ 뒤에 가타카나가 이어지면 그 낱말이다(3コース 3코스, 5コマ, 2コンボ, 100コイン). + +| 원문 | 출력 | 구분 | +|---|---|---| +| メール784コっす | 메일 784개예요 | 고침 | +| 201コだよね | 201개지 | 고침 | +| 三コください | 세 개 주세요 | 고침 | +| 3コース | 3코스 | 유지 | +| 100コイン | 100코인 | 유지 | + +사례: `tests/rules/cases/ko-counter.tsv` diff --git a/src/core/pipeline/normalize.cpp b/src/core/pipeline/normalize.cpp index aff7d53f6..32c1b8d2e 100644 --- a/src/core/pipeline/normalize.cpp +++ b/src/core/pipeline/normalize.cpp @@ -673,6 +673,17 @@ std::string NormalizeForms(std::string_view text) { } else { c = MapForm(c); } + // A katakana コ alone after a number is the counter 個 written in katakana (784コ 784個, 三コ, 何コ); + // before more katakana it starts a word (3コース, 5コマ). + if (c == U'コ' && !out.empty()) { + const char32_t prev = LastRune(out).cp; + const char32_t next = At(text, i).cp; + const bool number = (prev >= U'0' && prev <= U'9') || (prev >= U'0' && prev <= U'9') || + In(prev, U"〇一二三四五六七八九十百千何数"); + const bool word = (next >= U'ァ' && next <= U'ヺ') || next == U'ー' || (next >= 0xFF66 && next <= 0xFF9F) || + next == 0x3099 || next == 0x309A; + if (number && !word) c = U'個'; + } const text::Rune mark = At(text, i); if (const char32_t composed = ComposeKanaPair(c, mark.cp)) { c = composed; diff --git a/src/core/pipeline/normalize.h b/src/core/pipeline/normalize.h index 1bf03487e..f3170804a 100644 --- a/src/core/pipeline/normalize.h +++ b/src/core/pipeline/normalize.h @@ -129,7 +129,8 @@ std::string ComposeKanaPairs(std::string_view text); // small vowel or a katakana letter composed (ウ゛ァ ヴァ); the CP932 mapping differences (〜 ~, // − -, ‖ ∥, ¢ £ ¬ in full width) and kanji forms outside JIS X 0208 (頰 頬, 𠮟 叱) mapped; // ‼ ⁉ ⁈ as two marks (!! !? ?!) without a variation selector after them; U+0000..U+0003 as -// spaces. Invalid UTF-8 bytes are kept. +// spaces; a katakana コ alone after a number as the counter 個 (784コ 784個, not 3コース). Invalid UTF-8 +// bytes are kept. std::string NormalizeForms(std::string_view text); // A run of two or more っ or ッ after a letter and before a letter other than a small kana, ー diff --git a/tests/rules/cases/ko-counter.tsv b/tests/rules/cases/ko-counter.tsv new file mode 100644 index 000000000..f0abc309c --- /dev/null +++ b/tests/rules/cases/ko-counter.tsv @@ -0,0 +1,17 @@ +# 규칙: NUM-34 +rule input expected kind layer +normalize:ko-counter メール784コっす has:784개예요 fix normalize +normalize:ko-counter 201コだよね 201개지 fix normalize +normalize:ko-counter 446コの 446개의 fix normalize +normalize:ko-counter 594コ 594개 fix normalize +normalize:ko-counter 12コ 12개 fix normalize +normalize:ko-counter 三コください 세 개 주세요 fix normalize +normalize:ko-counter 十コある 열 개 있다 fix normalize +normalize:ko-counter 何コある? 몇 개 있어? fix normalize +normalize:ko-counter 3コース 3코스 keep normalize +normalize:ko-counter 5コマ lacks:개 keep normalize +normalize:ko-counter 2コンボ 2콤보 keep normalize +normalize:ko-counter 100コイン 100코인 keep normalize +normalize:ko-counter 3コスト 3코스트 keep normalize +normalize:ko-counter 3コロ 3코로 keep normalize +normalize:ko-counter コ、コイツ lacks:개 keep normalize