Package Details: fcitx-mozc-ut-unified-full 2.26.4500.102.20210922-1

Git Clone URL: https://aur.archlinux.org/fcitx-mozc-ut-unified-full.git (read-only, click to copy)
Package Base: fcitx-mozc-ut-unified-full
Description: Fcitx Module of A Japanese Input Method for Chromium OS, Windows, Mac and Linux (the Open Source Edition of Google Japanese Input) with New UT dictionaries (default dictionaties.)
Upstream URL: https://osdn.net/users/utuhiro/pf/utuhiro/files/
Licenses: custom
Conflicts: fcitx-mozc, fcitx-mozc-neologd-ut, fcitx-mozc-neologd-ut+ut2, fcitx-mozc-ut, fcitx-mozc-ut-unified, fcitx-mozc-ut2, mozc, mozc-neologd-ut, mozc-ut, mozc-ut-unified, mozc-ut2
Submitter: YEK
Maintainer: YEK
Last Packager: YEK
Votes: 2
Popularity: 0.000163
First Submitted: 2021-01-04 17:26 (UTC)
Last Updated: 2021-09-22 16:34 (UTC)

Latest Comments

1 2 Next › Last »

YEK commented on 2021-09-12 15:09 (UTC)

@tuxsavvy

Thanks again for contributing to this one!

I'm very sorry to inform you that there has been a complete rewrite of the script in upstream's 20210907 package. Including make-dictionaries.bash, which was contributed last time, requires close inspection to be merged with this change, and I couldn't take the time to do so, so I've reflected the upstream as is for now. This change has also been applied to GitHub.

The upstream can have very large changes, and when the upstream is updated, the AUR packages are no longer buildable. If we are going to make changes to the script, we need to figure out how to deal with this problem. For example, it may be necessary to have a forked mozc repository as well.

Pull requests could not be processed due to upstream changes. If possible, it would be helpful if they could be reworked based on the changes made.

If you are honestly using the upstream one as a base, this kind of thing is likely to happen more often in the future and needs to be considered.

tuxsavvy commented on 2021-09-06 09:39 (UTC)

@YEK Thanks for response!

With regards to the revised PKGBUILD, that is fine. I figured it would be unsuitable for your workflow and decided to proceed ahead with the other suggestion; to propose the make-dictionary.sh file, and will just refine my own customPKGBUILD for my own internal use. All good. (^^ ;)

I do want to apologise for misunderstanding on the part about OSDN. My intention was to avoid being a burden when collaborating. Because the PKGBUILD and all referenced to OSDN, I figured it would be faster to simply convert the forked project into Git, learn to use mercurial in the meantime so that it would be easier for you as well as any other contributors, and still provide contributions through both Git and mercurial.

Thank you very much for creating a copy on the Github! I have finally managed to create a working converted replica of your mercurial repository into Git, which I do personally prefer, in that there is a record of changes. https://osdn.net/projects/support/ticket/42762 is the support ticket that I encountered but I figured a workaround since, and used that for the time being.

With the conversion, I too have encountered artefacts like mozcdic-ut-* in which because Github has a hard limit of 100MB per blob/file, I have removed traces of those into my Github repo. Sorry, but thanks for providing your Github repo. I will still try and do PR (Pull Request) to it, but otherwise my repo won't be an identical copy of yours.

All that said and done, I do have a suggestion about replacing the tarballs for source=() array into something like Git submodules. Perhaps something similar to how fcitx-mozc has it. In addition, another suggestion is with regards to the Japanese postal codes, and to use an alternative variant as prescribed in the same fcitx-mozc 's PKGBUILD

YEK commented on 2021-09-05 10:39 (UTC)

@tuxsavvy

Thanks for contributing!

I didn't upload the revised PKGBUILD, so I'm sure what I said didn't make sense to you. Sorry about that.

I'm using OSDN only to simplify file distribution, and OSDN is not suitable for collaboration, so it's better not to accept contributions on OSDN.

So this time, I set up a new repository on GitHub (with Git!). https://github.com/reasonset/mozcdic-ut-unified-full

This repository excludes the build artifact mozcdic-ut-* by gitignore, so that it can be used for script development. I hope you can make use of it.

YEK commented on 2021-09-05 10:06 (UTC)

Uploaded now. Sorry it took so long.

YEK commented on 2021-09-05 09:32 (UTC)

@gcarrillo

Sorry, I thought I had updated it. Please wait a moment while I check right now why it hasn't been updated.

@tuxsavvy

Please wait a moment.

gcarrillo commented on 2021-09-02 17:28 (UTC)

Mozc version needs to be updated to 2.26.4472.102 and fcitx to 20210822. Thanks for all the hard work maintaining this package!

tuxsavvy commented on 2021-08-23 17:24 (UTC) (edited on 2021-08-23 17:30 (UTC) by tuxsavvy)

Hi YEK, thanks for the response!

I agree, there weren't that much of improvement I could make with PKGBUILD. Most of it were really minor, and I thought I would just chime in on what I am modifying in other parts, in hoping it would make things simpler.

As for the make-dictionaries.sh thanks for including! I was actually looking for this link, https://osdn.net/users/reasonset/pf/aur_mozc_ut_unified/wiki/FrontPage in hoping to have some sort of VCS to work with. Though ideally would have preferred git as I have a private repo on gitlab, with the last revision plus this one, one other branch, and a branch with my changes that I could showcase. I will try to work with Mercurial instead.

I noticed you have pushed my modified shell script into your repo, but I have also made changes in several Ruby files. Please refer to the following diff below. I could try and make changes if you wish directly into your repository, but will need to take some time to get used to Mercurial and OSDN.

  • alt-cannadic/modify-cannadic.rb
--- modify-cannadic.rb.orig
+++ modify-cannadic.rb
@@ -84,11 +84,6 @@
 # main
 # ==============================================================================

-`wget -nc https://ja.osdn.net/projects/alt-cannadic/downloads/50881/alt-cannadic-110208.tar.bz2`
-`rm -rf alt-cannadic-110208`
-`tar xf alt-cannadic-110208.tar.bz2`
-`mv alt-cannadic-110208/{gcanna.ctd,g_fname.ctd} .`
-
 $filename = "gcanna.ctd"
 $dicname = "mozcdic-altcanna.txt"
 modify_cannadic
  • edict/modify-edict2.rb
--- modify-edict2.rb.orig
+++ modify-edict2.rb
@@ -108,9 +108,6 @@
 # main
 # ==============================================================================

-`rm -f edict2`
-`wget -nc http://ftp.monash.edu/pub/nihongo/edict2.gz`
-`gzip -dk edict2.gz`
 $filename = "edict2"
 $dicname = "mozcdic-edict2.txt"

  • jawiki-all-titles/count-jawiki-titles.rb
--- count-jawiki-titles.rb.orig
+++ count-jawiki-titles.rb
@@ -61,8 +61,6 @@
 # main
 # ==============================================================================

-`rm -f jawiki-latest-all-titles-in-ns0`
-`wget -nc https://dumps.wikimedia.org/jawiki/latest/jawiki-latest-all-titles-in-ns0.gz`
 $filename = "jawiki-latest-all-titles-in-ns0.gz"
 $dicname = "jawiki-latest-all-titles-in-ns0.counts"
  • jawiki-articles/get-entries-from-jawiki-articles.rb
--- get-entries-from-jawiki-articles.rb.orig
+++ get-entries-from-jawiki-articles.rb
@@ -223,8 +223,6 @@
 jawiki = "jawiki-latest-pages-articles.xml.bz2"
 mozcdic = "jawiki-ut.txt"

-`wget -nc https://dumps.wikimedia.org/jawiki/latest/#{jawiki}`
-
 reader = Bzip2::FFI::Reader.open(jawiki)
 $dicfile = File.new(mozcdic, "w")
 core_num = `grep cpu.cores /proc/cpuinfo`.chomp.split(": ")[-1].to_i - 1
  • neologd/convert-neologd-to-mozc.rb
--- convert-neologd-to-mozc.rb.orig
+++ convert-neologd-to-mozc.rb
@@ -133,14 +133,11 @@
 # ==============================================================================
 # main
 # ==============================================================================
-
 require 'open-uri'
 url = "https://github.com/neologd/mecab-ipadic-neologd/tree/master/seed"
 neologdver = URI.open(url).read.split("mecab-user-dict-seed.")[1]
 neologdver = neologdver.split(".csv.xz")[0]

-`wget -nc https://github.com/neologd/mecab-ipadic-neologd/raw/master/seed/mecab-user-dict-seed.#{neologdver}.csv.xz`
-`7z x -aos mecab-user-dict-seed.#{neologdver}.csv.xz`
 $filename = "mecab-user-dict-seed.#{neologdver}.csv"
 $dicname = "mozcdic-neologd.txt"


  • nicoime/modify-nicoime.rb
--- modify-nicoime.rb.orig
+++ modify-nicoime.rb
@@ -85,9 +85,6 @@
 # main
 # ==============================================================================

-`wget -nc http://public.s3.tkido.com.s3-website-ap-northeast-1.amazonaws.com/nicoime.zip`
-`rm -f nicoime_*.txt`
-`7z x nicoime.zip`
 $filename = "nicoime_msime.txt"
 $dicname = "mozcdic-nicoime.txt"


  • skk/modify-skkdic.rb
--- modify-skkdic.rb.orig
+++ modify-skkdic.rb
@@ -73,9 +73,6 @@
 # main
 # ==============================================================================

-`wget -nc http://openlab.jp/skk/dic/SKK-JISYO.L.gz`
-`rm -f SKK-JISYO.L`
-`gzip -dk SKK-JISYO.L.gz`
 $filename = "SKK-JISYO.L"
 $dicname = "mozcdic-skkdic.txt"


  • sudachidict/convert-sudachiduct-to-mozc.rb
--- convert-sudachiduct-to-mozc.rb.orig
+++ convert-sudachiduct-to-mozc.rb
@@ -148,9 +148,6 @@
 # main
 # ==============================================================================

-`wget -nc https://github.com/WorksApplications/SudachiDict/raw/develop/src/main/text/core_lex.csv`
-`wget -nc https://github.com/WorksApplications/SudachiDict/raw/develop/src/main/text/notcore_lex.csv`
-
 $filename = "core_lex.csv"
 $dicname = "mozcdic-sudachidict-core.txt"
 convert_sudachidict_to_mozc
  • zipcode/fix-ken_all.rb
--- fix-ken_all.rb.orig
+++ fix-ken_all.rb
@@ -69,9 +69,6 @@
 # main
 # ==============================================================================

-`rm -f KEN_ALL.CSV`
-`wget -nc https://www.post.japanpost.jp/zipcode/dl/kogaki/zip/ken_all.zip`
-`unzip ken_all.zip`
 $filename = "KEN_ALL.CSV"
 $dicname = "KEN_ALL.CSV.fixed"
 fix_ken_all

YEK commented on 2021-08-23 12:29 (UTC)

Hi tuxsavvy, thanks for the great contribution.

First, about PKGBUILD. I think it's an appropriate improvement, but due to the way I work with meld to import upstream PKGBUILD, it doesn't make updating easier, so I decided not to adopt it. However, there was a problem with the addressing method of the source, so I fixed that this time.

Next, about make-dictionaries. Thanks for the great content. I've included it in this package.

make-dictionaries is included in the upstream project (Utuhiro's mozcdic-ut : https://osdn.net/users/utuhiro/pf/utuhiro/wiki/FrontPage). It would be better if you could adopt it there.

The project page where the builds are stored is at https://osdn.net/users/reasonset/pf/aur_mozc_ut_unified/wiki/FrontPage. It's a Mercurial repository, so if you can help, I'd be very happy.

tuxsavvy commented on 2021-08-17 11:05 (UTC)

#!/usr/bin/env bash

UTDICDATE="20210725"

altcannadic="true"
edict="true"
jawikiarticles="true"
jinmeiut="true"
neologd="true"
nicoime="true"
skk="true"
sudachidict="true"


# ==============================================================================
# Make each dictionary
# ==============================================================================

function clean() {
dirs -c
echo ":: Cleaning up working directory ..."
find "$PWD" -name "mozcdic-ut.txt" -delete
find .. -maxdepth 1 -name "mozcdic-ut-*.txt" -delete
find .. -type f \( -name "*.zip" -o -name "*.gz" \) -delete
# find .. -type f -name "*.bz2" -delete
find ../mozc -type d -name "mozc-master" -exec rm -r '{}' \;
}

function get-mozc-dict-defs() {
echo ":: Fetching the latest mozc dictionary definitions ..."
pushd ../mozc &>/dev/null || exit
# https://stackoverflow.com/a/18194523
svn checkout https://github.com/google/mozc/trunk/src/data/dictionary_oss --quiet ;\
mv dictionary_oss/id.def "$PWD" ;\
cat dictionary_oss/dictionary*.txt > "$PWD/mozcdic.txt" ;\
rm -rf dictionary_oss
popd &>/dev/null || exit
}

function get-and-process-alt-cannadic() {
echo ":: Checking for revised canna dictionary file ..."
# acc = Academic Computer Club, Umeå University ウメオ, スウェーデン
# bfsu = 北京外国語大学 北京, 中国
# constant = Constant Hosting ピスカタウェイ, ニュージャージー州, 米国
# dotsrc = Dotsrc.org オールボー大学, オールボー, デンマーク
# gigenet = GigeNET シカゴ, イリノイ州, アメリカ合衆国
# ipconnect = IP-Connect LLC ヴィーンヌィツャ, ウクライナ
# jaist = 北陸先端科学技術大学院大学 石川、日本
# liquid = Liquid Telecom ナイロビ, ケニヤ
# nchc = National Center for High-performance Computing 新竹, 台湾
# osdn = OSDN 東京、日本
# plug = Purdue Linux Users Group ウエストラファイエット, インディアナ州, 米国
# rwthaachen = アーヘン工科大学 アーヘン, ドイツ
# tuna = 清華大学 TUNA 協会 北京, 中国
# xtom_hk = xTom Hong Kong Limited 香港, 中国
# xtom_us = xTom.com Inc. ロサンゼルス、カリフォルニア州、米国 
# ymu = 山形大学 山形, 日本
local _dicname="alt-cannadic"
local _mirror="jaist"
local _relid="50881"
local _reldate="110208"
local _redirurl="https://ja.osdn.net/frs/redir.php"
local _dicext=".tar.bz2"
local _url="$_redirurl?m=$_mirror&f=$_dicname%2F$_relid%2F$_dicname-$_reldate$_dicext"
local _sha256sum="d352f4f90fac02219043d9fe5829925215d4bb6173782430c78ccddd38094a98"
if [[ $altcannadic = "true" ]]; 
  then
    pushd "../$_dicname" &>/dev/null || exit
# https://stackoverflow.com/a/11856444
      if ! [ -f "$_dicname-$_reldate.tar.bz2" ];
        then 
            echo ":: File not found. Downloading ..."
            curl --output "$_dicname-$_reldate.tar.bz2" --location "$_url"
      fi
# https://unix.stackexchange.com/a/545866
# https://superuser.com/a/1468626
      if [[ $(sha256sum --check --status <(echo "$_sha256sum  $_dicname-$_reldate$_dicext") ; echo $?) = 1 ]];
        then
                    echo ":: Checksum failed, retrying ..."
                rm "$_dicname-$_reldate$_dicext"
                                        popd &>/dev/null || exit
          get-and-process-alt-cannadic
              else
                      echo ":: Processing revised canna dictionary file ..."
              find "$PWD" -type d -name "$_dicname-$_reldate" -exec rm -rf '{}' \;
              tar -xf "$_dicname-$_reldate$_dicext"
              find "$_dicname-$_reldate" -type f \( -name "gcanna.ctd" -o -name "g_fname.ctd" \) -exec mv --target-directory="$PWD" '{}' \;
                  ruby modify-cannadic.rb
          cat mozcdic-altcanna-jinmei.txt >> ../src/mozcdic-ut.txt
          cat mozcdic-altcanna.txt >> ../src/mozcdic-ut.txt
      fi
    popd &>/dev/null || exit
fi
} 

function get-and-process-edict() {
if [[ $edict = "true" ]]; 
  then
  echo ":: Checking for Jim Breen's EDICT2 file ..."
  pushd "../edict" &>/dev/null || exit
  find "$PWD" -type f -name "edict2" -delete
    if ! [ -f "edict2.gz" ];
        then
            echo ":: File not found. Downloading ..."
            curl --output "edict2.gz" --location "http://ftp.edrdg.org/pub/Nihongo/edict2.gz"
    fi
    echo ":: Processing Jim Breen's EDICT2 ..."
        gzip --decompress --keep edict2.gz
  ruby modify-edict2.rb
  cat mozcdic-edict2.txt >> ../src/mozcdic-ut.txt
        popd &>/dev/null || exit
fi
}

function get-and-process-jawiki-titles() {
local _titles_file="jawiki-latest-all-titles-in-ns0"
local _md5sum_file="jawiki-latest-md5sums.txt"
local _url="https://dumps.wikimedia.org/jawiki/latest"
local _titles_md5=$(sed '/all-titles-in-ns0\.gz/!d ; s/\ .*$//' <(curl --silent --location "$_url/$_md5sum_file"))
echo ":: Checking for Japanese Wikipedia all titles file ..."
pushd "../jawiki-all-titles" &>/dev/null || exit
find "$PWD" -type f -name "$_titles_file" -delete
if ! [ -f "$_titles_file.gz" ];
    then
        echo ":: File not found. Downloading ..."
        curl --output "$_titles_file.gz" --location "$_url/$_titles_file.gz"
fi
if [[ $(md5sum --check --status <(echo "$_titles_md5  $_titles_file.gz") ; echo $?) = 1 ]];
  then
          echo ":: Checksum failed, retrying ..."
          rm "$_titles_file.gz"
          popd &>/dev/null || exit
      get-and-process-jawiki-titles
        else
                echo ":: Processing Japanese Wikipedia all titles file ..."
        ruby count-jawiki-titles.rb
fi
popd &>/dev/null || exit
}

function get-and-process-jawiki-articles() {
local _articles_file="jawiki-latest-pages-articles.xml.bz2"
local _md5sum_file="jawiki-latest-md5sums.txt"
local _url="https://dumps.wikimedia.org/jawiki/latest"
local _articles_md5=$(sed '/pages-articles\.xml\.bz2/!d ; s/\ .*$//' <(curl --silent --location "$_url/$_md5sum_file"))
if [[ $jawikiarticles = "true" ]]; then
  echo ":: Checking for Japanese Wikipedia articles file ..."
  pushd "../jawiki-articles/" &>/dev/null || exit
    if ! [ -f "$_articles_file" ];
        then
            echo ":: File not found. Downloading ..."
            curl --output "$_articles_file" --location "$_url/$_articles_file"
        else
            echo ":: File found. Verifying file integrity ..."
    fi
    if [[ $(md5sum --check --status <(echo "$_articles_md5  $_articles_file") ; echo $?) = 1 ]];
      then
              echo ":: Checksum failed, retrying ..."
              #rm "$_articles_file"
                    popd &>/dev/null || exit
        get-and-process-jawiki-articles
                else
                        echo ":: Processing Japanese Wikipedia articles file for cost adjustments ..."
        ruby convert-jawiki-ut-to-mozc.rb
        ruby ../src/filter-entries.rb mozcdic-jawiki.txt
        cat mozcdic-jawiki.txt >> ../src/mozcdic-ut.txt
        fi
  popd &>/dev/null || exit
fi
}

function process-jinmei() {
if [[ $jinmeiut = "true" ]]; 
        then
                echo ":: Processing jinmei for Japanese names ..."
                pushd ../jinmei-ut &>/dev/null || exit
                ruby modify-jinmei-ut.rb
                cat mozcdic-jinmei-ut.txt >> ../src/mozcdic-ut.txt
                popd &>/dev/null || exit
fi
}

function get-and-process-neologd() {
local _url="https://github.com/neologd/mecab-ipadic-neologd/raw/master/seed"
local _file="mecab-user-dict-seed"
local _version="20200910"
local _file_suffix=".csv.xz"
if [[ $neologd = "true" ]]; then
  echo ":: Checking for Neologism dictionary file ..."
  pushd "../neologd" &>/dev/null || exit
        if ! [ -f "$_file.$_version$_file_suffix" ];
        then
            echo ":: File not found. Downloading ..."
            curl --output "$_file.$_version$_file_suffix" --location "$_url/$_file.$_version$_file_suffix"
        fi
        echo ":: Processing Neologism dictionary file ..."
  xz --decompress --keep "$_file.$_version$_file_suffix"
  ruby convert-neologd-to-mozc.rb
  ruby ../src/filter-entries.rb mozcdic-neologd.txt
  cat mozcdic-neologd.txt >> ../src/mozcdic-ut.txt
        popd &>/dev/null || exit
fi
}

function get-and-process-nicoime() {
local _filename="nicoime.zip"
local _url="http://public.s3.tkido.com.s3-website-ap-northeast-1.amazonaws.com"
if [[ $nicoime = "true" ]]; 
        then
  pushd "../nicoime" &>/dev/null || exit
        echo ":: Checking for niconico IME file ..."
        if ! [ -f "$_filename" ];
        then
            echo ":: File not found. Downloading ..."
                curl --output "$_filename" --location "$_url/$_filename"
        fi
        echo ":: Processing niconico IME file ..."
        find "$PWD" -type f -name "nicoime_*.txt" -delete
        unzip "$_filename"
        ruby modify-nicoime.rb
        cat mozcdic-nicoime.txt >> ../src/mozcdic-ut.txt
        find "$PWD" -type f -name "nicoime_*.txt" -delete
        popd &>/dev/null || exit
fi
}

function get-and-process-skk() {
local _filename="SKK-JISYO.L.gz"
local _url="http://openlab.jp/skk/dic"
if [[ $skk = "true" ]]; 
        then
        pushd "../skk" &>/dev/null || exit
        echo ":: Checking for Simple Kana to Kanji (SKK) dictionary file ..."
        if ! [ -f "$_filename" ];
        then
            echo ":: File not found. Downloading ..."
            curl --output "$_filename" --location "$_url/$_filename"
        fi
        echo ":: Processing Simple Kana to Kanji (SKK) dictionary file ..."
        find "$PWD" -type f -name "SKK-JISYO.L" -delete
        gzip --decompress --keep "$_filename"
        ruby modify-skkdic.rb
        cat mozcdic-skkdic.txt >> ../src/mozcdic-ut.txt
        popd &>/dev/null || exit
fi
}

function get-and-process-sudachidict() {
local _filename="core_lex.csv"
local
_url="https://github.com/WorksApplications/SudachiDict/raw/develop/src/main/text/"

if [[ $sudachidict = "true" ]]; 
        then
        pushd "../sudachidict" &>/dev/null || exit
        echo ":: Checking for Sudachi Dictionary files ..."
        if ! [ -f "$_filename" ];
        then
            echo "File: $_filename not found. Downloading ..."
            curl --output "$_filename" --location "$_url/$_filename"
        fi
        if ! [ -f "not$_filename" ];
        then
            echo "File: not$_filename not found. Downloading ..."
            curl --output "not$_filename" --location "$_url/not$_filename"
        fi
        echo ":: Processing Sudachi Dictionary files ..."
        ruby convert-sudachiduct-to-mozc.rb
        ruby ../src/filter-entries.rb mozcdic-sudachidict-*.txt
        cat mozcdic-sudachidict-*.txt >> ../src/mozcdic-ut.txt
        popd &>/dev/null || exit
fi
}

function fetch_and_process-Japan-postcode() {
local _filename="ken_all.zip"
local _url="https://www.post.japanpost.jp/zipcode/dl/kogaki/zip"
echo ":: Checking Japan postal codes file  ..."
pushd "../zipcode" &>/dev/null || exit
find "$PWD" -type f -name "KEN_ALL.CSV" -delete
        if ! [ -f "$_filename" ];
        then
            echo "File not found. Downloading ..."
                curl --output "$_filename" --location "$_url/$_filename"
  fi
echo ":: Processing Japan postal codes ..."
unzip "$_filename"
ruby fix-ken_all.rb
ruby generate-chimei.rb
cat mozcdic-chimei.txt >> ../src/mozcdic-ut.txt
find "$PWD" -type f -name "KEN_ALL.CSV" -delete
popd &>/dev/null || exit
}

function extract_new_entry_and_apply_jawiki_costs() {
cd ../src/


# ==============================================================================
# Extract new entries and apply jawiki costs
# ==============================================================================

ruby extract-new-entries.rb mozcdic-ut.txt
ruby apply-jawiki-costs.rb mozcdic-ut.txt.extracted

rm -f ../mozcdic*-ut-*.txt
mv mozcdic-ut.txt.extracted ../mozcdic-ut-$UTDICDATE.txt
}

function make_mozcdic-ut_pkg() {
# ==============================================================================
# Make a mozcdic-ut package
# ==============================================================================

cd ../../
rm -rf mozcdic-ut-$UTDICDATE
mkdir mozcdic-ut-$UTDICDATE
rsync -av mozcdic-ut-dev/* mozcdic-ut-$UTDICDATE --exclude=id.def \
--exclude=jawiki-latest* --exclude=jawiki-ut.txt --exclude=KEN_ALL.* --exclude=*.csv \
--exclude=*.xml --exclude=*.gz --exclude=*.bz2 --exclude=*.xz --exclude=*.zip
rm -f mozcdic-ut-$UTDICDATE/*/mozcdic*.txt*
}

function main() {
clean
get-mozc-dict-defs
get-and-process-alt-cannadic
get-and-process-edict
get-and-process-jawiki-titles
get-and-process-jawiki-articles
process-jinmei
get-and-process-neologd
get-and-process-nicoime
get-and-process-skk
get-and-process-sudachidict
fetch_and_process-Japan-postcode
extract_new_entry_and_apply_jawiki_costs
make_mozcdic-ut_pkg
}

main