Source

MLP Regressor#

A feedforward neural network with a continuous output layer suitable for regression problems. Like the Multi Layer Perceptron classifier, the MLP Regressor is able to tackle complex non-linear regression problems by forming higher-order representations of the input features using intermediate computational units called hidden layers.

Interfaces: Estimator, Learner, Online, Verbose, Persistable

Data Type Compatibility: Continuous

Parameters#

# Param Default Type Description
1 hidden array An array composing the user-specified hidden layers of the network in order.
2 batch size 100 int The number of training samples to process at a time.
3 optimizer Adam object The gradient descent optimizer used to update the network parameters.
4 alpha 1e-4 float The amount of L2 regularization to apply to the parameters of the network.
5 epochs 1000 int The maximum number of training epochs. i.e. the number of times to iterate over the entire training set before terminating.
6 min change 1e-4 float The minimum change in the training loss necessary to continue training.
7 window 3 int The number of epochs without improvement in the validation score to wait before considering an early stop.
8 holdout 0.1 float The proportion of training samples to use for validation and progress monitoring.
9 cost fn LeastSquares object The function that computes the loss associated with an erroneous activation during training.
10 metric RSquared object The metric used to score the generalization performance of the model during training.

Additional Methods#

Return the training loss at each epoch:

public steps() : array

Return the validation scores at each epoch:

public scores() : array

Returns the underlying neural network instance or null if untrained:

public network() : Network|null

Example#

use Rubix\ML\Regressors\MLPRegressor;
use Rubix\ML\NeuralNet\Layers\Dense;
use Rubix\ML\NeuralNet\Layers\Activation;
use Rubix\ML\NeuralNet\ActivationFunctions\ReLU;
use Rubix\ML\NeuralNet\Optimizers\RMSProp;
use Rubix\ML\CrossValidation\Metrics\RSquared;

$estimator = new MLPRegressor([
    new Dense(100),
    new Activation(new ReLU()),
    new Dense(100),
    new Activation(new ReLU(),
    new Dense(50),
    new Activation(new ReLU()),
    new Dense(50),
    new Activation(new ReLU()),
], 256, new RMSProp(0.001), 1e-3, 100, 1e-5, 3, 0.1, new LeastSquares(), new RSquared());

References#

  • G. E. Hinton. (1989). Connectionist learning procedures.
  • L. Prechelt. (1997). Early Stopping - but when?