A graph where the values are joined by lines is good if it illustrates a continuous connection. If we have data at our disposal that is in no way connected, it is much better to mark it with points. This is possible on a special type of graph — a scatter plot.
data.plot(x='column_x', y='column_y', kind='scatter')
An obvious shortcoming of a scatter plot is that it can have a huge number of points that merge into a single mass. Areas of higher density are not distinguished in a cloud of values. There are two ways to make a graph clearer:
The graph is divided into cells, and the points are recalculated in each cell. The cells are then filled with color: The more points, the denser the color. Such a graph is called a ** hexbin (i.e., a graph divided into hexagonal regions)**. The number of cells along the horizontal axis is set by the gridsize parameter, which is analogous to bins for hist().
data.plot(x='column_x', y='column_y', kind='hexbin', gridsize=our_gridsize, sharex=False, grid=True)
Like a histogram, this graph displays frequency. However, a histogram shows only one value, but there are two here. The increased frequency of certain combinations reflects consistency. Often the goal of data analysis is to show the relationship between the two values.
The interdependence of two values is called correlation. A graph allows us to confirm that the two values are clearly interconnected, or that they correlate. In the event of direct dependence of the values (the greater one is, the greater the other is), then it is said that correlation is positive. In the event there is an inverse relationship of the values (the greater one is, the less the other is), then it is said that the correlation is negative.
Numerically, the relationship is estimated using the Pearson correlation coefficient. It helps in determining how much one value changes when another changes as well; it takes values from -1 to 1. If the second value grows along with the first, then the Pearson correlation coefficient is positive. If one value changes when the other remains, the coefficient is equal to 0. If growth of one value is associated with a decrease in another, the coefficient is negative. The closer the Pearson correlation coefficient is to the extreme values (-1 or 1), the stronger the interdependence. A value close to zero means that the connection is weak or completely absent. The coefficient can be zero not because there is no connection between the values, but rather because it has a more complex, non-linear character. Therefore, the correlation coefficient can’t handle such a relationship.
The Pearson coefficient is calculated using the corr () method. The method is applied to the column with the first value, and the column with the second is passed as the parameter. Whichever comes first is not important:
print(data['column_1'].corr(data['column_2']))
print(data['column_2'].corr(data['column_1']))
When a task requires finding pairwise relationships of values, this can be done using pairwise scatter plots. In pandas, such a problem is not solved by data.plot (), but rather by a special method: pd.plotting.scatter_matrix (data).
pd.plotting.scatter_matrix(data)
In addition to pairwise scatter plots, a pairwise correlation coefficient can be obtained for all values. This can be done using a correlation matrix:
data.corr()