Can I run non-linear regression without any given functional form using multi variable dataframe in Python?

Question

I try to analyze the multivariable with LinearRegression (e.g. randomforest, sklearn etc.) dataframe with Python.

However, it shows a huge RMSE so I try to apply non-linear regression to my data.

I want to run a non-linear regression to predict value of commodities, with lots of data (e.g. Quality, produced year, factory number, overall condition & lots of categories ..).

I try to find on Google and Stackoverflow, but I can find only using curve_fit, polyfit way which already predict the functional form between X and Y. Besides, my input data key value is larger than 10 so I want to using ML or some other packages to apply non-linear regression.

Please help and advise me!

Answer 1

If you're looking to fit a curve to data with minimal assumptions about the function that gets applied to one or more predictors, you are entering the realm of Nonparametric Regression.

For a linear regression, you are considering some varitation of: y = f(x) + u, often in a form like y = B_0 + B_1*x_1 + u, where the assumptions are linearity in both the parameters and the data.

Nonparametric methods attempt to estimate the form of the function, rather than just the B's in a model.

Some simple nonparametric methods are often called "smoothers"; these attmept to fit a smooth line to data (a computed trace in a scatterplot is an example of this). Other methods attempt to run regressions that do some form of weighting for distances of data points, or other penalties that help fit the line without being bound by assumptions like linearity across the whole dataset; a couple of the most prominent ones are loess & lowess (two distinct things, not just a spelling preference). As with any methods, there are reasons to choose one over the other, each has drawbacks and benefits, and require a little reading before using them indiscriminately.

For a more detailed explanation, this is a really good presentation.

To work on this in Python, here is the documentation on statsmodels' library of nonparametric methods.