Python & matplotlib plotting points beyond the domain, with poor curve fitting

Question

Let me preface this by saying I'm totally new to curve fitting in python, so I may be doing something completely and obviously wrong.

I have an experimental 'raw' data set consisting of temperature (x) vs signal (y).

I am trying to fit a Boltzmann equation to this data using scipy.curve_fit. My script isn't throwing any errors, but when I plot it in matplotlib, it utilizes x values between 0 and 600 when my experimental data domain only encompasses values between ~308 --> 400. Not only that, the curve it fits to the data >appears< completely off (shifted and skewed), but its derivative looks identical to that of the raw data... making me think some transformation was applied somewhere.

#Get the data
file = 'Data/my_excel_file.xlsx'
df = pd.read_excel(file, sheet_name='simplified')

#Define the Boltzmann function for fitting
def BoltzmannFitEquation(T, Fmin, Fmax, Tm, breadth):
    return Fmin + ((Fmax - Fmin) / (1 + np.exp((Tm - (T/breadth)))))

#Grabbing the y-values (signal) from the dataframe
signal = df['signal'].tolist()

#Convert my temps from the dataframe from C to K.
for temp in temps_c:
    temps_k.append(float(temp) + 273)

#Now lets fit a Boltzmann equation to the smoothed data
p0 = [0.9, 1.2, 347, 1] #initial predictions
c, cov = curve_fit(BoltzmannFitEquation, temps_k, signal, p0)

yp = BoltzmannFitEquation(temps_k, c[0], c[1], c[2], c[3]) #Plot of the prediction with the optimized coefficients

plt.plot(yp)

I have excluded a bunch of my code to simplify things -- but let me know if you would like to see something specifically it could help troubleshoot why I'm seeing this.

Blue lines are 'raw' data and derivative, orange lines are fitted curve and derivative.

Notice how the inflection points do not match on the top chart, but they do on the bottom. Why is the curve fit so bad? Why would it even include values outside of the domain?

Sure, here it is: s000.tinyupload.com/index.php?file_id=27511381335592876598 As per the code... its in the 'simplified' sheet — Vranvs
"I have excluded a bunch of my code to simplify things..." It is much easier for someone to help you if you provide a minimal, complete and verifiable example. Then someone can simply copy and run the code to reproduce the problem. — Warren Weckesser
You have plt.plot(yp). I think that should be plt.plot(temps_k, yp) instead, so that temps_k is used for the x values. If you use plt.plot(yp), the x values are [0, 1, 2, ..., len(temps_k) - 1]. — Warren Weckesser
Thank you Warren! Sorry for excluding the code -- you're right I should have included some more info. I usually don't for proprietary reasons >:) — Vranvs

James Phillips James Phillips · Accepted Answer · 2019-01-29T22:58:23

When I try the following code I appear to have an OK fit.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit

file = 'Data/my_excel_file.xlsx'
#file = '/home/zunzun/temp/temp.xlsx'
df = pd.read_excel(file, sheet_name='simplified')

#Define the Boltzmann function for fitting
def BoltzmannFitEquation(T, Fmin, Fmax, Tm, breadth):
    return Fmin + ((Fmax - Fmin) / (1 + np.exp((Tm - (T/breadth)))))

#Grabbing the data from the dataframe
signal = np.array(df['signal'].tolist())
temps_c = np.array(df['temperature'].tolist())

#Convert my temps from the dataframe from C to K.
temps_k = temps_c + 273.0

#Now lets fit a Boltzmann equation to the smoothed data
p0 = [0.9, 1.2, 347, 1] #initial predictions
c, cov = curve_fit(BoltzmannFitEquation, temps_k, signal, p0)

yp = BoltzmannFitEquation(temps_k, c[0], c[1], c[2], c[3]) #Plot of the prediction with the optimized coefficients

print("Fitted paraneters:", c)

plt.plot(temps_k, signal) # data
plt.plot(temps_k, yp) # fit
plt.show()

Python & matplotlib plotting points beyond the domain, with poor curve fitting

1 Answers