Understanding multiple Linear regression

1
votes

I am doing multiple regression problem. I have the below data set as below.

rank--discipline--yrs.since.phd--yrs.service--sex--salary
[  1           1             19           18    1  139750],......

I am taking salary as dependent variable, and other variable as independent variable. After doing data pre processing, I ran the gradient descent, regression model. I estimated bias(intercept), coefficient for all independent features. I want to do scattered plot for the actual values and regression line for the hypothesis I predicted. Since we have more than one features here,

I have the below questions.

  1. While plotting actual values (scatted plot), how do I decide the x-axis values. Meaning, I have list of values. for example, first row [1,1,19,18,1]=>139750 How do I transform or map [1,1,19,18,1] to x-axis.? I need to somehow make [1,1,19,18,1] to one value, so I can mark a point of (x,y) in the plot.

  2. While plotting regression line, what would be the feature values, so I can calculate the hypothesis value.? Meaning now, I have the intercept, and weight of all features, but I dont have the feature values. How do I decide upon the feature values now.?

I want to calculate the points and use matplot to do the jobs. I am aware that there are lot of tools available outside including matplotlib to do the job. But I want to get the basic understanding.

Thanks.

2
machine-learningregressionlinear-regression
If you have multiple targets in your dataset, the best approach is to plot each target separately, i.e. in each plot have only 1 target. As for your question 1, is the data stored as numpy arrays or pandas DataFrame? Coming to question 2, can you explain where you want to plot it. are there only two variables? Do you already have the coefficient and intercept, also most Importantly, can you post the result of df.describe() on your input database. Also if possible try to reframe your second question as I am still a bit confused by it. – anand_v.singh
Hi, I have exactly only one target (salary) in my data set and 5 features (rank, discipline,etc..). I am using pandas for df. my query on point1 is, lets say the first data instance is [1,1,19,18,1] and target value is 139750. how do I plot these values in x,y axis. Since I have more than one value for x axis, how I do i convert it.? Query on 2nd question is, lets say [1,2,3,4,5,6] is the intercept and co-efficients i have arrived for this dataset. intercept=1, and rest are the weight for the features. formula would be h(x)=1+2X1+3X2+4X3+5X4+6X5. what are the value I can take for X1, X2...X5? – Selva Ganesh S
Not a programming question, better suited for Cross Validated. – desertnaut

2 Answers

0
votes

I am still not sure I completely understand your question, so if something is not what you expected comment below and we will work it out.

Now,

Query 1: In all your datasets you are going to have multiple inputs and there is no way to view the target variable salary in your case with respect to all, in a single graph, what is usually done is either you apply the concept of dimensionality reduction on your data using t-sne (link) or you use principal component analysis (PCA) to reduce the dimensionality of your data, and make your output a function of two or three variables and then plot it on the screen, the other technique that I prefer is rather plotting target vs each variable separately as subplot, The reason for this is we don't even have a way to comprehend how we will see the data that is in more than three dimensions.

Query 2: If you are not determined to use matplotlib, I will suggest seaborn.regplot(), but let's also do it in matplotlib. Suppose the variable you want to use first is 'discipline' vs 'salary'.

from sklearn.linear_model import LinearRegression
lm = LinearRegression()
X = df[['discipline']]
Y = df['salary']
lm.fit(X,Y)

After running this lm.coef_ will give you the coefficient, and lm.intercept_ will give you the intercept, in a linear equation that forms this variable, then you can plot the data between two variables and a line using matplotlib easily.

0
votes

what you can do is ->

from pandas import plotting as pdplt

pdplt.scatter_matrix(dataframe, pass the remaining required parameters)

by this you will get a matrix of plots(in your case it's 6X6) which will exactly show how each column in your dataframe relates to the other columns and you can clearly visualise which feature dominates the result and also how the features are correlated to each other. If you ask me this is the first thing I used to do with such types of problems and then remove all correlated features and select the features which best approximate the output.

But as you have to plot a 2d plot and in the above approach you might get more than a single feature which dominate the output then what you can do is a miracle named PCA. If you ask me PCA is one of the most beautiful thing in machine learning. What it will do that is somehow merges all your feautres in some magical ratio which will generate principle components for your data. Principal components are those components which govern/major contribution to your model. You apply pca by simply importing from sklearn and then select the first principle component(as you need a 2d plot) or might select 2 priciple components and plot a 3d graph. But always remember this that these pricipal components are not the real features of your model but they are some magical combination and how PCA did so is very very interesting(by using concepts like eigen values and vectors) and you can build by your own also.

Apart from all these you can apply Singular Value decomposition(SVD) to your model which is the essence of whole linear algebra which is a type of matrix decomposition existing for all matrix. What this do is decompose your matrix into three matrix out of which the diagonal matrix which consists of singular values(a scaling factor) in descending order and what you have to do is that select the top singular values (in your case only the first one having highest magnitude) and construct back a feature matrix from 5 columns to 1 columns and then plot that. You can do svd by using the numpy.linalg

Once you applied any one of these methods then what you can do is learn your hypothesis with only the single most important selected feature and finally plot the graph. But take a tip, just for plotting a 2d graph you should avoid other important features beacuse maybe you have 3 principal components all having almost the same contribution and may the top three singular values are very close to each other. So take my words and take all important features into account and if you need the visualisation of these important features then use scatter matrix

Summary ->

All I want to mention is that you can do the same process with all these things and also can invent your own statistical or mathematical model for compressing your feature space. But for me I prefer to go with PCA and in such type of problems I even first plot the scatter matrix to get an visual intuition to the data. And also PCA and SVD helps to remove redundancy and hence overfitting.

For rest details refer to docs.

Happy machine learning...