Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions spec/numbers.md
Original file line number Diff line number Diff line change
Expand Up @@ -468,3 +468,18 @@
| 三日間 | 3일간 | 유지 |

사례: `tests/rules/cases/laugh-legs-month-one.tsv`

## NUM-34 수 뒤에 홀로 쓴 コ는 조수사 `개`다

수(아라비아 숫자, 한자 수, 何, 数) 뒤에 가타카나 コ 한 글자만 오면 個를 가타카나로 쓴 조수사라서 정규화 단계가 個로 읽고 `개`다(メール784コっす 메일 784개예요,
三コください 세 개 주세요, 何コ 몇 개). コ 뒤에 가타카나가 이어지면 그 낱말이다(3コース 3코스, 5コマ, 2コンボ, 100コイン).

| 원문 | 출력 | 구분 |
|---|---|---|
| メール784コっす | 메일 784개예요 | 고침 |
| 201コだよね | 201개지 | 고침 |
| 三コください | 세 개 주세요 | 고침 |
| 3コース | 3코스 | 유지 |
| 100コイン | 100코인 | 유지 |

사례: `tests/rules/cases/ko-counter.tsv`
11 changes: 11 additions & 0 deletions src/core/pipeline/normalize.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -673,6 +673,17 @@ std::string NormalizeForms(std::string_view text) {
} else {
c = MapForm(c);
}
// A katakana コ alone after a number is the counter 個 written in katakana (784コ 784個, 三コ, 何コ);
// before more katakana it starts a word (3コース, 5コマ).
if (c == U'コ' && !out.empty()) {
const char32_t prev = LastRune(out).cp;
const char32_t next = At(text, i).cp;
const bool number = (prev >= U'0' && prev <= U'9') || (prev >= U'0' && prev <= U'9') ||
In(prev, U"〇一二三四五六七八九十百千何数");
const bool word = (next >= U'ァ' && next <= U'ヺ') || next == U'ー' || (next >= 0xFF66 && next <= 0xFF9F) ||
next == 0x3099 || next == 0x309A;
if (number && !word) c = U'個';
}
const text::Rune mark = At(text, i);
if (const char32_t composed = ComposeKanaPair(c, mark.cp)) {
c = composed;
Expand Down
3 changes: 2 additions & 1 deletion src/core/pipeline/normalize.h
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,8 @@ std::string ComposeKanaPairs(std::string_view text);
// small vowel or a katakana letter composed (ウ゛ァ ヴァ); the CP932 mapping differences (〜 ~,
// − -, ‖ ∥, ¢ £ ¬ in full width) and kanji forms outside JIS X 0208 (頰 頬, 𠮟 叱) mapped;
// ‼ ⁉ ⁈ as two marks (!! !? ?!) without a variation selector after them; U+0000..U+0003 as
// spaces. Invalid UTF-8 bytes are kept.
// spaces; a katakana コ alone after a number as the counter 個 (784コ 784個, not 3コース). Invalid UTF-8
// bytes are kept.
std::string NormalizeForms(std::string_view text);

// A run of two or more っ or ッ after a letter and before a letter other than a small kana, ー
Expand Down
17 changes: 17 additions & 0 deletions tests/rules/cases/ko-counter.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# 규칙: NUM-34
rule input expected kind layer
normalize:ko-counter メール784コっす has:784개예요 fix normalize
normalize:ko-counter 201コだよね 201개지 fix normalize
normalize:ko-counter 446コの 446개의 fix normalize
normalize:ko-counter 594コ 594개 fix normalize
normalize:ko-counter 12コ 12개 fix normalize
normalize:ko-counter 三コください 세 개 주세요 fix normalize
normalize:ko-counter 十コある 열 개 있다 fix normalize
normalize:ko-counter 何コある? 몇 개 있어? fix normalize
normalize:ko-counter 3コース 3코스 keep normalize
normalize:ko-counter 5コマ lacks:개 keep normalize
normalize:ko-counter 2コンボ 2콤보 keep normalize
normalize:ko-counter 100コイン 100코인 keep normalize
normalize:ko-counter 3コスト 3코스트 keep normalize
normalize:ko-counter 3コロ 3코로 keep normalize
normalize:ko-counter コ、コイツ lacks:개 keep normalize
Loading