Databases

Introduction

The python library integrates pre-defined modules for several well-known database used in the deep learning community, such as MNIST, GTSRB, CIFAR10 and so on. That way, no extra step is necessary to be able to directly build a network and learn it on these database. The library allow you to add pre-process data with built in Transformation.

Database

The python library provide you with multiple object to manipulate common database.

Loading hand made database can be done using n2d2.database.DIR.

Like in the following example :

# Creating the database object
db = n2d2.database.DIR()

provider = n2d2.provider.DataProvider(db, data_dims)

# The zeroes represent the depth to seek the data.
db.load(data_path, 0, label_path, 0)

# With this line we put all the data in the learn partition:
db.partition_stimuli(learn=1, validation=0, test=0)
provider.set_partition("Learn")

inputs_tensor = provider.read_random_batch()

DIR

MNIST

ILSVRC2012

CIFAR100

Cityscapes

Transformations

Composite

PadCrop

Distortion

Rescale

ColorSpace

RangeAffine

SliceExtraction

RandomResizeCrop

ChannelExtraction

Sending data to the Neural Network

Once a database is loaded, n2d2 use n2d2.provider.DataProvider to provide data to the neural network.

The n2d2.provider.DataProvider will automatically apply the n2d2.transform.Transformation to the dataset. To add a transformation to the provider, you should use the method n2d2.transform.Transformation.add_transformation().

Example

In this example, we will show you how to create a n2d2.database.Database, n2d2.provider.Provider and apply n2d2.transformation.Transformation to the data.

We will use the n2d2.database.MNIST database driver, rescale the images to a 32x32 pixels size and then print the data used for the learning.

# Loading data
database = n2d2.database.MNIST(data_path=path, validation=0.1)

# Initializing DataProvider
provider = n2d2.provider.DataProvider(database, [32, 32, 1], batch_size=batch_size)

# Applying Transformation
provider.add_transformation(n2d2.transform.Rescale(width=32, height=32))

# Setting the partition of data we will use
provider.set_partition("Learn")

# Iterating other the inputs
for inputs in provider:
        print(inputs)