Thursday, September 1, 2011

What is Latent Semantic Analysis or Latent Semantic Indexing?


LSA : Latent Semantic Analysis
LSI : Latent Semantic Indexing
Keywords : latent semantic indexing, LSI, singular value decomposition, SVD, eigenvectors, documents, queries, cosine similarity, term count mode.
Latent Semantic Analysis (LSA), also known as Latent Semantic Indexing (LSI) literally means analyzing documents to find the underlying meaning or concepts of those documents. If each word only meant one concept, and each concept was only described by one word, then LSA would be easy since there is a simple mapping from words to concepts.
LSI or LSA is an obvious method of retrieving relevant pages is by matching the terms of a search query with the same text found in all web pages.
How Latent Semantic Analysis Works : 
Latent Semantic Analysis arose from the problem of how to find relevant documents from search words. The fundamental difficulty arises when we compare words to find relevant documents, because what we really want to do is compare the meanings or concepts behind the words. LSA attempts to solve this problem by mapping both words and documents into a "concept" space and doing the comparison in this space.

Since authors have a wide choice of words available when they write, the concepts can be obscured due to different word choices from different authors. This essentially random choice of words introduces noise into the word-concept relationship. Latent Semantic Analysis filters out some of this noise and also attempts to find the smallest set of concepts that spans all the documents.

In order to make this difficult problem solvable, LSA introduces some dramatic simplifications.
  • Documents are represented as "bags of words", where the order of the words in a document is not important, only how many times each word appears in a document.
  • Concepts are represented as patterns of words that usually appear together in documents. For example "leash", "treat", and "obey" might usually appear in documents about dog training.
  • Words are assumed to have only one meaning. This is clearly not the case (banks could be river banks or financial banks) but it makes the problem tractable.
                  http://blog.ckumar.in/
                  

2 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. Amazing blog on TECHNOLOGY. I have found the same site where you can now find Exclusive Custom Websites on Chiropractic Web Sites.
    It can be helpful for those who are looking for Custom websites and have a tight budget. Moreover, The best theme

    Exclusive, Custom Websites

    Mobile Compatible Animation and Patient Education Videos & Library and much more.

    ReplyDelete

I Tech - Technology Updates's Facebook Wall