Asiya is a machine translation assessment toolkit for evaluating the results of machine translation ( http://asiya.lsi.upc.edu/ ). It is mainly written in Perl.
How to use Asiya to perform BLEU metrics?
I watched the youtube video: https://www.youtube.com/watch?v=rA5De9Z4uWI
And created a configuration file (Asiya.config):
input=raw
srclang=en
srccase=en
trglang=ja
trgcase=ja
src=corpus.tok/test.tok.en
ref=corpus.tok/hyp.tok.ja
sys=corpus.tok/test.tok.ja
some_metrics= BLEU NIST METEOR-ex Ol
My machine translation output file is in corpus.tok/hyp.tok.ja, the source file is in corpus.tok/test.tok.en, and the link file (correct translation) is in corpus.tok/test.tok.ja. They are tokenized text files, each line is a sentence.
And when I ran:
/home/expert/asiya/bin/Asiya.pl -eval single -g all -metric_set some_metrics Asiya.config
I got this error:
Smartmatch is experimental at /home/expert/asiya/bin/../lib/IQ/Common.pm line 784.
Smartmatch is experimental at /home/expert/asiya/bin/../lib/IQ/Common.pm line 791.
Smartmatch is experimental at /home/expert/asiya/bin/../lib/IQ/Scoring/ESA.pm line 339.
Use of uninitialized value in concatenation (.) or string at /home/expert/asiya/bin/../lib/IQ/Config.pm line 251.
[ASIYA] directory </tools> does not exist!
, . ? , Asiya?
Asiya BLEU?
Asiya, BLEU BLEU ?
( http://nlp.lsi.upc.edu/redmine/boards/11/topics/138)