This got me searching, and I think I've found a solution. Not sure if this is actually documented or even intended, but it may very well work:
get_ipython().config['IPKernelApp']['pylab'] == 'inline'
get_ipython() appears to be a method only defined when running IPython; it returns what I assume is the current IPython session. Then, you can access the config attribute, which is a dictionary, that includes the 'IPKernelApp' element. The latter is a dictionary in itself that can contain a key pylab, which can be the string 'inline'.
I haven't tried extensively, but I'm guessing that the above line of code will evaluate to False if you're not running pylab inline.
More importantly, it will raise a KeyError when you're not running the notebook or the pylab option, so you'll need to catch that and take that raised exception as "no" for running a notebook with pylab inline.
Finally, get_ipython() may throw a NameError, and similar to the above, that of course also means you're not running IPython.
I've only tested this minimally, but importing this in my IPython notebook, and then on the default Python cmdline does show it to work.
Let us know if this works for you.