ann_FF_Hess — computes Hessian by finite differences.
H The Hessian hypermatrix, have the same layout as W.*.W (not W.*.W', as usually found in literature); H(n1,i1,l1,n2.i2,l2) is the second derivative of error with respect to weights W(n1,i1,l1) and W(n2,i2,l2). x Matrix of input patterns, one pattern per column. t Matrix of target patterns, one pattern per column. N Row vector describing the number of neurons per layer. N(1) is the size of input pattern vector, N(size(N,'c')) is the size of output pattern vector (and also target). W The weight hypermatrix. dW, dW2 The quantities used to perturb each W parameter; dW is used for non-diagonal parameters, for diagonal parameters the supplemental perturbations quantities dW2 * dW are also used (note that very small values may easily lead to numerical instabilities) af The activation function to be used. This parameter is optional, default value "ann_log_activ", i.e. the logistic activation function. ef The error function to be used. This parameter is optional, default value "ann_sum_of_sqr", i.e. the sum-of-squares error function.
This function calculates the Hessian trough a finite differences procedure. This process is very slow and is provided for testing purposes only. Internals: or off-diagonal elements: two weights are perturbed at +/- dW; for on-diagonal elements the perturbing quantities are: +/- (1 +/- dW2) * dW for one weight; the corresponding Hessian term is calculated from the four resulting errors (at four W points).