Divider
  Speech Technology and Research Laboratory
  People
  Current Research Activities
  Past Research Activities
  Publications
  Career Opportunities
  Seminars
  Technologies for License
  In the News
  Contact Us
  STAR Search
  Information and Computing Sciences Division
SpacerAbout UsDividerR and D DivisionsDividerCareersDividerNewsroomDividerContact UsDividerSRI HomeSpacer

Spacer
         
  SRI Logo

Search SRILM-USER Archives

Match: Format: Sort by:
Search:

Re: Kullback Leibler

From: Andreas Stolcke <stolcke at ADDRESS HIDDEN>
Date: Fri, 08 Jun 2007 16:12:53 PDT

In message <1594.145.116.14.61.1181339193.squirrel at ADDRESS HIDDEN>you w
rote:
> Dear Andreas Stolcke,
>
> just a very small question for curiosity. Is there already some tool
> included in the SRILM toolkit to calculate the Kullback Leibler divergence
> between two LMs?

No. it would be cool to have such a tool.

For certain models (e.g., ngram models)
you could come up with an exact computation, similar to what the pruning
algorithm uses.

For the general case you could sample from one of the LMs and
compute an empirical cross-entropy.

Feel free to implement something ...

Andreas

Click here to go to the SRILM home page.

 

About Us  Vertical divider  R&D Divisions  Divider  Careers  Divider  Newsroom  Divider  Contact Us
©2006 SRI International, 333 Ravenswood Avenue, Menlo Park, CA 94025-3493
SRI International is an independent, nonprofit corporation. Privacy policy
Last modified Dec 02, 2008