Databases¶
Introduction¶
The python library integrates pre-defined modules for several well-known database used in the deep learning community, such as MNIST, GTSRB, CIFAR10 and so on. That way, no extra step is necessary to be able to directly build a network and learn it on these database. The library allow you to add pre-process data with built in Transformation.
Database¶
The python library provide you with multiple object to manipulate common database.
Loading hand made database can be done using n2d2.database.DIR
.
Like in the following example :
# Creating the database object
db = n2d2.database.DIR()
provider = n2d2.provider.DataProvider(db, data_dims)
# The zeroes represent the depth to seek the data.
db.load(data_path, 0, label_path, 0)
# With this line we put all the data in the learn partition:
db.partition_stimuli(learn=1, validation=0, test=0)
provider.set_partition("Learn")
inputs_tensor = provider.read_random_batch()
DIR¶
MNIST¶
ILSVRC2012¶
CIFAR100¶
Cityscapes¶
Transformations¶
Composite¶
PadCrop¶
Distortion¶
Rescale¶
ColorSpace¶
RangeAffine¶
SliceExtraction¶
RandomResizeCrop¶
ChannelExtraction¶
Sending data to the Neural Network¶
Once a database is loaded, n2d2 use n2d2.provider.DataProvider
to provide data to the neural network.
The n2d2.provider.DataProvider
will automatically apply the n2d2.transform.Transformation
to the dataset.
To add a transformation to the provider, you should use the method n2d2.transform.Transformation.add_transformation()
.
Example¶
In this example, we will show you how to create a n2d2.database.Database
, n2d2.provider.Provider
and apply n2d2.transformation.Transformation
to the data.
We will use the n2d2.database.MNIST
database driver, rescale the images to a 32x32 pixels size and then print the data used for the learning.
# Loading data
database = n2d2.database.MNIST(data_path=path, validation=0.1)
# Initializing DataProvider
provider = n2d2.provider.DataProvider(database, [32, 32, 1], batch_size=batch_size)
# Applying Transformation
provider.add_transformation(n2d2.transform.Rescale(width=32, height=32))
# Setting the partition of data we will use
provider.set_partition("Learn")
# Iterating other the inputs
for inputs in provider:
print(inputs)