Note: I found this piece on Erik’s computer after his passing. I wanted to share it with you.
Change the nomenclature. A “neural net” is a term for a mathematical system that establishes a transfer function to link an input set or vector of values to an output set or vector of values.
Can make a higher order polynomial pass through the domain of a set of points using a least-square approach. We don’t state that the curve has been “taught” or has “learned” how to pass closely by the points. It is not demonstrating “intelligence” by doing so, it’s just the math of polynomials, coordinates in space, and statistics of minimizing error by least squares. Also, if we interpolate to different points but within the (x,y) domain of the original points, we get close to the nearest neighbor points. This is not demonstrating “understanding” or “learning). If we put in coordinates outside the domains and get an answer, this is not “generalizing” the “knowledge”, this is also just the math. The operation of a neural net can be expressed in vectors and matrices.