How do you make a Japanese word frequency dictionary from Wikipedia or novel corpora?
YuzuLingo · Nov 1, 2026
You take a ranked word list from a corpus, write those ranks into Yomitan term_meta_bank files with mode freq, then zip the JSON with an index.json. Kanjium’s wikipedia_freq.txt and novels_freq.txt are common starting lists; other open frequency projects work the same way once you control the license.
Start from a ranked list
A frequency list is usually expression, optional reading, and a rank or score. Wikipedia-based lists favor encyclopedic vocabulary. Novel corpora favor dialogue and fiction wording. Ambiguous readings sometimes carry notes in the source; keep those flags when you map rows so the reader can prefer the matching reading.
Map rows into term-meta banks
Yomitan frequency data belongs in term_meta_bank_*.json, not in glossary term banks. Each meta row stores the expression and a frequency object or number the schema allows for mode freq. Split large corpora across numbered bank files the same way glossary builders do. Validate against the published term-meta schema before you zip.
Write index.json and package the zip
index.json needs a clear title, revision, and format 3. Put every JSON file at the archive root and compress with Deflate. Name the dictionary so users can tell Wikipedia ranks from novel ranks when both are installed. Do not mix pitch mode entries into the same banks unless you intend a combined meta package and understand how your reader picks the first hit.
How readers use the rank
After a glossary match, the reader looks up the expression in enabled frequency dictionaries and shows the rank. Lower ranks usually mean more common words, depending on how the source numbered its list. Frequency never replaces a definition. It only answers how often that form appeared in the chosen corpus.
License and maintenance
Only redistribute lists whose license allows it, and credit the corpus in the dictionary title or description. Rebuild when you refresh the source dump. Keep frequency as its own zip so you can swap Wikipedia for novels without touching JMdict or pitch packages.
