Linear Regression Example

The easy way

Simply input the arrays of independent and dependent data and call the linreg function to see the interxept and slope:

$x gets [1,2,3,4,5]                  /* independent data: array */
$y gets [1,2,1.3,3.75,2.25]          /* dependent data: array */
linreg($x, $y)

You could also get just the slope with slope($x, $y) or just the intercept with intercept($x, $y).

The hard way

You can use Palamedes matrix operations to do the calculation step-by-step to get extra information such as the OLS estimate for the variance in the error term...

/* Commands */
$x gets [1,2,3,4,5]                  /* independent data: array */
$y gets [1,2,1.3,3.75,2.25]          /* dependent data: array */
n gets count $y                      /* n: sample size */
p gets 2                             /* p: number of predictors */
ν gets n-p                           /* nu: degrees of freedom */
y gets [$y]^T                        /* n x 1 matrix */
ones gets n # 1
$X gets [ones, $x]^T                 /* design matrix: n x p matrix */
y; $X                                /* show what we have so far */

/* y = $X cross β */
/* Therefore $X^-1 cross y = $X^-1 cross $X cross β = β */
/* N.B. No need to construct a pseudo-inverse anymore: this is done automatically */
β gets $X^-1 cross y; β             /* response coefficients: p x 1 matrix */

/* break out the slope and intercept then show them */
intercept gets β after 0 after 0; intercept
slope gets β after 1 after 0; slope

ŷ gets $X cross β; ŷ                 /* fitted values */
residuals gets ŷ - y; residuals      /* residuals from the regression */

/* ss: OLS estimate for the variance in the error term */
ss gets ((residuals^T cross residuals) after 0 after 0)/ν; ss

/* standard error of the regression (SER) AKA standard error of the equation (SEE) */
sqrt(ss)
 /* Results */
$x ← [1, 2, 3, 4, 5]
$y ← [1, 2, 1.3, 3.75, 2.25]
n ← count($y)
p ← 2
ν ← n - p
y ← [$y] ^ T
ones ← n # 1
$X ← [ones, $x] ^ T

y → 
[[    1 ], ...
 [    2 ], ...
 [  1.3 ], ...
 [ 3.75 ], ...
 [ 2.25 ]]
$X → 
[[ 1, 1 ], ...
 [ 1, 2 ], ...
 [ 1, 3 ], ...
 [ 1, 4 ], ...
 [ 1, 5 ]]

β ← $X ^ -1 × y
β → 
[[  0.7849999999999997 ], ...
 [ 0.42500000000000027 ]]

intercept ← β ∘ 0 ∘ 0
intercept → 0.7849999999999997

slope ← β ∘ 1 ∘ 0
slope → 0.42500000000000027

ŷ ← $X × β
ŷ → 
[[               1.21 ], ...
 [ 1.6350000000000002 ], ...
 [ 2.0600000000000005 ], ...
 [ 2.4850000000000008 ], ...
 [  2.910000000000001 ]]
residuals ← ŷ - y
residuals → 
[[  0.20999999999999996 ], ...
 [ -0.36499999999999977 ], ...
 [   0.7600000000000005 ], ...
 [  -1.2649999999999992 ], ...
 [    0.660000000000001 ]]

ss ← ((residuals ^ T × residuals) ∘ 0 ∘ 0) / ν
ss → 0.9302499999999999

sqrt(ss) → 0.9644946863513556


AUTHORS BUGS DISCUSSION LICENSE NAME NEWS README SOURCE TODO TRY VERSION

Last update: Fri Sep 23 2016