NLP - Extracting "Correct" Noun Phrases

1
votes

I'm stuck with a problem related to NLP and I was hoping I could get some advice to help solve it. I am currently writing a program in which given a sentence that contains a monetary number, the program will be able to return the noun phrase that corresponds to that number. For example, given this sentence:

"That bike costs $100."

I am matching the noun phrase "bike" to "$100". I am using Stanford NLP's parser to find the noun phrases in a sentence, and as you know it is possible to have noun phrases within noun phrases. For example, in the sentence:

"The purchase price for these goods was $50."

The two noun phrases "purchase price" and "goods" are both contained under the noun phrase "The purchase price for these goods", as identified by the Stanford parser. Thus, it would make sense when matching to only take into account what I refer to as base noun phrases, or noun phrases that don't have any other noun phrases contained within it. But this doesn't always work, because in this sentence for example:

"He incurred $23 thousand of costs related to office space and $10 thousand of costs related to office supplies."

The base noun phrases identified by the parser would be "costs" x2, "office supplies", and "office space". But none of these noun phrases properly describe the numbers "$23 thousand" and "10 thousand". We really need are the more general noun phrases "costs of office supplies" and "costs of office space" to adequately describe the numbers.

So I'm stuck in trying to figure out a method to extract noun phrases that aren't too broad or too specific, as shown by the last two examples. But so far, it just doesn't seem possible to develop a method that will work for all the different sentences out there. If someone could offer any advice on how to approach this problem, it would be greatly appreciated.

Thanks in advance for your time.

2
regexnlp

2 Answers

1
votes

This sounds like an open problem, where what is too broad or too specific is probably specific to the construction. So even if we solve this sentence, it might not transfer well to others.

In your last example, "He incurred $23 thousand of costs related to office space..." it is the argument to the "of" of the PP "$23 thousand" you want.

If you put the sentence into the parser (there is an online demo here), you get

(ROOT
  (S
    (NP (PRP He))
    (VP (VBD incurred)
    (NP
    (NP
      (NP
        (QP ($ $) (CD 23) (CD thousand)))
      (PP (IN of)
        (NP
          (NP (NNS costs))
          (VP (VBN related)
            (PP (TO to)
              (NP (NN office) (NN space)))))))

(...)

So if you can recognise this kind of construction and locate the dollar amount, then you might be able to just take the argument of its PP:

(NP
  (NP (NNS costs))
   (VP (VBN related)
        (PP (TO to)
          (NP (NN office) (NN space)))))

Similary, for the "purchase price" example, you want the argument of "for", which is "these goods".

1
votes

Look on this library I've implemented recently: https://github.com/krzysiekfonal/grammaregex

I think this is exactly what you look for - so you can parse your sentence finding one matching word when match is done according to regex-like filter processed on sentence-tree. One inconvinient thing for you might be it doesn't use coreNLP but spacy library(https://spacy.io/) - in future I'd like to make it independent of NLP lib.

The main thing which is taken into consideration in this lib - what I think you need to do your stuff - is that not only postags are taken into consideration, but also dependency relation relations between words. Here you have an example:

>>>import spacy
>>>from grammaregex import print_tree, match_tree, find_tokens
>>>nlp = spacy.load("en") 
>>>doc = nlp(u"Mrs. Robinson graduated from the Wharton School of the University of Pennsylvania in 1980.") 
>>>sent = next(doc.sents)

>>>print_tree(sent, "tag_")
{ { { Mrs.->compound(NNP) } Robinson->nsubj(NNP) } graduated->ROOT(VBD) { from->prep(IN) { { the->det(DT) } { Wharton->compound(NNP) } School->pobj(NNP) { of->prep(IN) { { the->det(DT) } University->pobj(NNP) { of->prep(IN) { Pennsylvania->pobj(NNP) } } } } } } { in->prep(IN) { 1980->pobj(CD) } } { .->punct(.) } }
>>>match_tree(sent, "VBD/prep/IN/pobj/NNP")
True
>>>match_tree(sent, "VBD/prep/IN/pobj/VBD")
False
>>>match_tree(sent, "VBD/**/DT")
True
>>>find_tokens(sent, "VBD/prep/IN/pobj/*")
[School, 1980]
>>>find_tokens(sent, "VBD/prep/IN/*/[NNP,DT]")
[School]
>>>find_tokens(sent, "VBD/**/DT")
[the, the]

So all you need is use print_tree to see your sentence tree(you can also use even better-graphical visualization on https://displacy.spacy.io/) and then adjust your regex-string which will be used in find_tokens method.

Here you can find quick intro to this simple lib: https://medium.com/@krzysiek89dev/grammaregex-library-regex-like-for-text-mining-49e5706c9c6d#.vrrm88bt4