Core N2D2

Introduction

In this section we will present the C++ core function that are binded to Python with the framework pybind. The binding of the C++ core is straightforward, thus this section can also be seen as a documentation of the C++ core implementation of N2D2.

If you want to use the raw python binding, you need to compile N2D2. This will create a ‘.so’ file in the lib folder. If you want to use the raw binding, you will need to have this file at the root of your project or in your PYTHONPATH.

You can then access the raw binding by importing N2D2 in your python script with the line import N2D2. It is however not recommended to use the raw binding, you should instead use the n2d2 python library.

DeepNet

Introduction

In order to create a neural network in N2D2 using an INI file, you can use the DeepNetGenerator:

net = N2D2.Network(seed-1)
deepNet = N2D2.DeepNetGenerator.generate(net, "../models/mnist24_16c4s2_24c5s2_150_10.ini")

Before executing the model, the network must first be initialized:

deepNet.initialize()

In order to test the first batch sample from the dataset, we retrieve the StimuliProvider and read the first batch from the test set:

sp = deepNet.getStimuliProvider()
sp.readBatch(N2D2.Database.Test, 0)

We can now run the network on this data:

deepNet.test(N2D2.Database.Test, [])

Finally, in order to retrieve the estimated outputs, one has to retrieve the first and unique target of the model and get the estimated labels and values:

target = deepNet.getTargets()[0]
labels = numpy.array(target.getEstimatedLabels()).flatten()
values = numpy.array(target.getEstimatedLabelsValue()).flatten()
results = list(zip(labels, values))

print(results)

[(1, 0.15989691), (1, 0.1617092), (9, 0.14962792), (9, 0.16899541), (1, 0.16261548), (1, 0.17289816), (1, 0.13728766), (1, 0.15315214), (1, 0.14424478), (9, 0.17937174), (9, 0.1518211), (1, 0.12860791), (9, 0.17310674), (9, 0.14563303), (1, 0.17823018), (9, 0.14206158), (1, 0.18292117), (9, 0.14831856), (1, 0.22245243), (9, 0.1745578), (1, 0.20414244), (1, 0.26987872), (1, 0.16570412), (9, 0.17435187)]

API Reference

Cells

Cell

AnchorCell

BatchNormCell

Cell

ConvCell

DeconvCell

DropoutCell

ElemWiseCell

FMPCell

FcCell

LRNCell

LSTMCell

NormalizeCell

ObjectDetCell

PaddingCell

PoolCell

ProposalCell

ROIPoolingCell

RPCell

ResizeCell

ScalingCell

SoftmaxCell

TargetBiasCell

ThresholdCell

TransformationCell

UnpoolCell

Frame

AnchorCell_Frame

AnchorCell_Frame_CUDA

BatchNormCell_Frame_float

BatchNormCell_Frame_double

BatchNormCell_Frame_CUDA_float

BatchNormCell_Frame_CUDA_double

Cell_Frame_float

Cell_Frame_double

Cell_Frame_CUDA_float

Cell_Frame_CUDA_double

Cell_Frame_Top

ConvCell_Frame_float

ConvCell_Frame_double

ConvCell_Frame_CUDA_float

ConvCell_Frame_CUDA_double

DeconvCell_Frame_float

DeconvCell_Frame_double

DeconvCell_Frame_CUDA_float

DeconvCell_Frame_CUDA_double

DropoutCell_Frame_float

DropoutCell_Frame_double

DropoutCell_Frame_CUDA_float

DropoutCell_Frame_CUDA_double

ElemWiseCell_Frame

ElemWiseCell_Frame_CUDA

FMPCell_Frame

FMPCell_Frame_CUDA

FcCell_Frame_float

FcCell_Frame_double

FcCell_Frame_CUDA_float

FcCell_Frame_CUDA_double

LRNCell_Frame_float

LRNCell_Frame_double

LRNCell_Frame_CUDA_float

LRNCell_Frame_CUDA_double

LSTMCell_Frame_CUDA_float

LSTMCell_Frame_CUDA_double

NormalizeCell_Frame_float

NormalizeCell_Frame_double

NormalizeCell_Frame_CUDA_float

NormalizeCell_Frame_CUDA_double

ObjectDetCell_Frame

ObjectDetCell_Frame_CUDA

PaddingCell_Frame

PaddingCell_Frame_CUDA

PoolCell_Frame_float

PoolCell_Frame_double

PoolCell_Frame_CUDA_float

PoolCell_Frame_CUDA_double

PoolCell_Frame_EXT_CUDA_float

PoolCell_Frame_EXT_CUDA_double

ProposalCell_Frame

ProposalCell_Frame_CUDA

ROIPoolingCell_Frame

ROIPoolingCell_Frame_CUDA

RPCell_Frame

RPCell_Frame_CUDA

ResizeCell_Frame

ResizeCell_Frame_CUDA

ScalingCell_Frame_float

ScalingCell_Frame_double

ScalingCell_Frame_CUDA_float

ScalingCell_Frame_CUDA_double

SoftmaxCell_Frame_float

SoftmaxCell_Frame_double

SoftmaxCell_Frame_CUDA_float

SoftmaxCell_Frame_CUDA_double

TargetBiasCell_Frame_float

TargetBiasCell_Frame_double

TargetBiasCell_Frame_CUDA_float

TargetBiasCell_Frame_CUDA_double

ThresholdCell_Frame

ThresholdCell_Frame_CUDA

TransformationCell_Frame

TransformationCell_Frame_CUDA

UnpoolCell_Frame

UnpoolCell_Frame_CUDA

Filler

Activation

Introduction

Activation functions in N2D2 are passed as arguments to initialize N2D2.Cell.

tanh = N2D2.TanhActivation_Frame_float()

Activation

Activation

LinearActivation

RectifierActivation

TanhActivation

SwishActivation

SaturationActivation

LogisticActivation

SoftplusActivation

Activation_Frame

LinearActivation_Frame

RectifierActivation_Frame

TanhActivation_Frame

SwishActivation_Frame

Solver

Target

Introduction

A N2D2.Target is associated to a N2D2.Cell, it define the output of the network. The computation of the loss and other tools to compute score such as the confusion matrix are also computed with this class.

To train a neural network you need to use N2D2.Target.provideTargets() then to N2D2.cell.propagate() then N2D2.Target.process() and finally N2D2.Cell.backpropagate(). (See the MNIST example.)

Databases

Introduction:

N2D2 allow you to import default dataset or to load your own dataset. This can be done suing Database objects.

Download datasets:

To import Data you can use a python Script situated in ./tools/install/install_dataset.py.

This script will download the data in /local/$USER/n2d2_data/. You can change this path with the environment variable N2D2_data.

Once the dataset downloaded, you can load it with the appropriate class. Here is an example of the loading of the MNIST dataset :

database = N2D2.MNIST_IDX_Database()
database.load(path)

In this example, the data are located in the folder path.

Database:

Database

MNIST_IDX_Database

Actitracker_Database

AER_Database

Caltech101_DIR_Database

Caltech256_DIR_Database

CaltechPedestrian_Database

CelebA_Database

CIFAR_Database

CKP_Database

DIR_Database

GTSRB_DIR_Database

GTSDB_DIR_Database

ILSVRC2012_Database

IDX_Database

IMDBWIKI_Database

KITTI_Database

KITTI_Object_Database

KITTI_Road_Database

LITISRouen_Database

N_MNIST_Database

DOTA_Database

Fashion_MNIST_IDX_Database

FDDB_Database

Daimler_Database

StimuliProvider

Transformation

Introduction

In order to apply transformation to a dataset, we use the transformation object.

Creation of different Transformation object.

dist = N2D2.DistortionTransformation()
dist.setParameter("ElasticGaussianSize", "21")
dist.setParameter("ElasticSigma", "6.0")
dist.setParameter("ElasticScaling", "36.0")
dist.setParameter("Scaling", "10.0")
dist.setParameter("Rotation", "10.0")

padcrop = N2D2.PadCropTransformation(24, 24)

ct = N2D2.CompositeTransformation(padcrop)
ct.push_back(dist)

To apply Transformation to a dataset, we use an object N2D2.StimuliProvider which acts as a data loader.

Transformations

Transformation

DistortionTransformation

PadCropTransformation

CompositeTransformation

AffineTransformation

ChannelExtractionTransformation

ColorSpaceTransformation

CompressionNoiseTransformation

DCTTransformation

DFTTransformation

EqualizeTransformation

ExpandLabelTransformation

WallisFilterTransformation

ThresholdTransformation

SliceExtractionTransformation

ReshapeTransformation

RescaleTransformation

RangeClippingTransformation

RangeAffineTransformation

RandomAffineTransformation

NormalizeTransformation

MorphologyTransformation

MorphologicalReconstructionTransformation

MagnitudePhaseTransformation

LabelSliceExtractionTransformation

LabelExtractionTransformation

GradientFilterTransformation

ApodizationTransformation

FilterTransformation

FlipTransformation

Containers

Introduction

N2D2 has his own Tensor implementation.

N2D2.Tensor_float([1, 2, 3])

Tensor can be also be created using numpy.array object.

N2D2.CudaTensor_float(numpy.array([[1.0, 2.0], [3.0, 4.0]]))

Tensor

CudaTensor