Search SRILM-USER Archives

[SRILM] lattice rescoring with FLM

Hi, everybody,

I'm doing lattice rescoring with FLM (SRILM 1.5.0).
However, I  somehow screw it up - may be somebody
could give me some hints on possible inconsistencies?

I have lattices generated with large bigram model and
then rescore it with small domain-specific trigram
model (word and FLM). For FLM rescoring I convert all
word nodes (HTK format) to FLM representation (e.g.
W=HELLO -> W=W-HELLO:S-HELLO:I-NULL:T-Db--x-) and then
rescore with
lattice-tool -in-lattice-list [list] -unk -read-htk
-no-nulls -no-htk-nulls -htk-words-on-nodes -factored
-lm [FLM_specification_file] -write-htk -htk-logbase
2.71828 -htk-lmscale 12.0 -htk-wdpenalty -10.0
-out-lattice-dir [dir]
I get lots of warnings "might produce unnormalized LM"
(that might be expected due to small size of
domain-specific training data?)
So, even if I use FLM that imitates simple word
trigram LM, the final accuracy on rescored lattices is
3 times lower than with conventional LM. When I check
the perplexities, they coincide. So, there should be
something with the way I do FLM lattice rescoring.
Did I miss something?

best regards,
Ilya

      ___________________________________________________________
Yahoo! Mail is the world's favourite email. Don't settle for less, sign up for
your free account today http://uk.rd.yahoo.com/evt=44106/*http://uk.docs.yahoo.com/mail/winter07.html

Click here to go to the SRILM home page.