Identifying the context of word in sentence

4
votes

I created classifier to classy the class of nouns,adjectives, Named entities in given sentence. I have used large Wikipedia dataset for classification.

Like :

Where Abraham Lincoln was born?

So classifier will give this short of result - word - class

  • Where - question
  • Abraham Lincoln - Person, Movie, Book (because classifier find Abraham Lincoln in all there categories)
  • born - time

When Titanic was released?

  • when - question
  • Titanic - Song, movie, Vehicle, Game (Titanic classified in all these categories)

Is there any way to identify exact context for word?

Please see :

  1. Word sense disambiguation would not help here. Because there might not be near by word in sentence which can help

  2. Lesk algorithm with wordnet or sysnet also does not help. Because it for suppose word Bank lesk algo will behave like this

    ======== TESTING simple_lesk ===========

    TESTING simple_lesk() ...

    Context: I went to the bank to deposit my money

    Sense: Synset('depository_financial_institution.n.01')

    Definition: a financial institution that accepts deposits and channels the money into lending activities

    TESTING simple_lesk() with POS ...

    Context: The river bank was full of dead fishes

    Sense: Synset('bank.n.01')

    Definition: sloping land (especially the slope beside a body of water)

Here for word bank it suggested as financial institute and slopping land. While in my case I am already getting such prediction like Titanic then it can be movie or game.

I want to know is there any other approach apart from Lesk algo, baseline algo, traditional word sense disambiguation which can help me to identify which class is correct for particular keyword?

Titanic -

1
nlpdata-miningnltksemantics
If, in the first example "born" is to be classified as "time", why does "release" in the second example not receive any classification? - Chthonic Project
@ChthonicProject: thanks for your comment. The reason behind this is feature based classification system. Time class has been trained using entities contexually expressing time like when , how long, duration, born and so on. It's not enough intelligent to identify released as time based event. Semi superwise and unsuperwised learning system might have identified it - user123

1 Answers

3
votes

Thanks for using the pywsd examples. With regards to wsd, there are many other variants and i'm coding them by myself during my free time. So if you want to see it improve do join me in coding the open source tool =)

Meanwhile, you will find the following technologies more relevant to your task, such as:

The above technologies usually includes several sub-tasks such as:

Essentially you're asking for a tool that is an NP-complete AI system for language/text processing, so I don't really think such a tool exists as of yet. Maybe it's IBM Watson.

if you're looking for the field to look into, the field is out there but if you're looking at tools, most probably wikification tools are closest to what you might need. (http://nlp.cs.rpi.edu/paper/WikificationProposal.pdf)