Linear regression with constrained intercept

Question

I do want to make a constrained linear regression with the intercept value to be like: lowerbound<=intercept<=upperbound.

I do know I can constrain the coefficients with some python libraries but couldn't find one where I can constrain the intercept.

What I want is to get the best solution that fits to my data points with the minimal possible error under the constraint where the intercept is in the range I defined.

How can I do it in python?

@seed the question was changed to ask about a range for the intercept, and no longer asks about a fixed value. — James Phillips
Scipy's curve_fit will accept bounds. For example to set a upper bound only on a parameter, that parameter's bound would be [-numpy.inf, upper bound]. Note that if bounds are used for curve_fit, the initial parameter estimates must all be within the specified bounds. — James Phillips

James Phillips James Phillips · Accepted Answer · 2019-11-02T20:22:01

Here is an example of using curve_fit with parameter bounds. In this example parameter "a" is unbounded, parameter "b" is bounded and the fitted value is within those bounds, and parameter "c" is bounded and the fitted value is at a bound.

import numpy
import matplotlib
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit

xData = numpy.array([5.0, 6.1, 7.2, 8.3, 9.4])
yData = numpy.array([ 10.0,  18.4,  20.8,  23.2,  35.0])


def standardFunc(data, a, b, c):
    return a * data + b * data**2 + c


# some initial parameter values - must be within bounds
initialParameters = numpy.array([1.0, 1.0, 1.0])

# bounds on parameters - initial parameters must be within these
lowerBounds = (-numpy.Inf, -100.0, -5.0)
upperBounds = (numpy.Inf, 100.0, 5.0)
parameterBounds = [lowerBounds, upperBounds]

fittedParameters, pcov = curve_fit(standardFunc, xData, yData, initialParameters, bounds = parameterBounds)

# values for display of fitted function
a, b, c = fittedParameters

# for plotting the fitting results
xPlotData = numpy.linspace(min(xData), max(xData), 50)
y_plot = standardFunc(xPlotData, a, b, c)

plt.plot(xData, yData, 'D') # plot the raw data as a scatterplot
plt.plot(xPlotData, y_plot) # plot the equation using the fitted parameters
plt.show()

print('fitted parameters:', fittedParameters)

UPDATE: per the comments, here is a multivariate fitting example:

import numpy
from scipy.optimize import curve_fit

X1 = (-1.0, -2.2, -3.3, -4.4, -5.5, -6.7)
X2 = (21.0, 22.2, 23.3, 24.4, 25.5, 26.7)
X3 = (51.0, 52.2, 53.3, 54.4, 55.5, 56.7)
all_X_data = numpy.array([X1, X2, X3])

Y = (11.1, 12.1, 13.1, 14.1, 15.1, 16.1)


# function to be fitted
def modelFunction(data, a, b, c, offset):
    f = (data[0] * a) + (data[1] * b) + (data[2] * c) + offset
    return f


# some initial parameter values
# these might be estimated from scatterplots
initialParameters = (1.0, 1.0, 1.0, 1.0)

# perform the fit
fittedParameters, pcov = curve_fit(modelFunction, all_X_data, Y, initialParameters)

print('a, b, c, offset:', fittedParameters)

Linear regression with constrained intercept

1 Answers