Variables

Activation Functions and Derivatives

Implementations of supported activation functions and their derivatives.

Each activation function is implemented as a separate static function, along with its derivative.
The functions are designed to be efficient and suitable for use in the forward and backward passes of neural network training.
The derivative functions are implemented to return non-zero values to avoid issues with zero gradients during training, particularly for the boolean activation function.
The activation functions currently implemented include:

Boolean Step Function
Sigmoid Function

Additional activation functions can be added following the same pattern, ensuring that both the function and its derivative are defined and included in the ntact_activation dispatch table.

Parameters

x

Returns: float

  /
//BOOLEAN
static float boolean( float x ){
    return x >= 0.0f ? 1.0f : 0.0f;
}
// [!!] This derivative is not mathematically correct.
// It intentionally returns a constant value to avoid zero-gradient behavior.
static float boolean_d( float x ){
    return x= 1.0f;
}
//SIGMOID
static float sigmoid( float x ){
    return 1.0f / ( 1.0f + expf( -x ) );
}
static float sigmoid_d( float x ){
    x = sigmoid( x );
    return x * ( 1.0f - x );
}
// ...

float(* ntact_activation [NTACT_TOTAL_FUNCTIONS][2])(float)

Activation function and derivative dispatch tables.

Detailed Description

Activation Functions List.
http://tituxdev.github.io/NeuroTIC/src/CPU/ntactivation.c.

Author: Oscar Sotomayor (Titux)

Definition in file ntactivation.c.

Variable Documentation

◆ ntact_activation

float(* ntact_activation[NTACT_TOTAL_FUNCTIONS][2])(float) ( float )

Initial value:

={
    [NTACT_BOOLEAN]= { boolean , boolean_d },
    [NTACT_SIGMOID]= { sigmoid , sigmoid_d }
 
}

Activation function and derivative dispatch tables.

Defines the mapping between activation identifiers and their associated execution functions.
Indexing:

First index : activation identifier
Second index : [0] activation [1] derivative

The table is initialized with the supported activation functions, and can be extended to include additional functions as needed.

Referenced by activate().