I run the following line at my terminal and I got an error, but it works fine in jupyter notebook.
df._repr_html_()
Traceback (most recent call last): File "", line 1, in File "/usr/lib/spark/python/pyspark/sql/dataframe.py", line 1020, in getattr "'%s' object has no attribute '%s'" % (self.class.name, name)) AttributeError: 'DataFrame' object has no attribute '_repr_html_'