Architecture design

Design insight

This project started in 2010, main plaform design concepts were then coming from emerging Big Data and Cloud Infrastructure-As-A-Service (IAAS) technologies.

From Big Data (mainly from Hadoop) :

• cheap hardware (computing nodes and network links)
• horizontal scalability
• efficient distributed processing
• natively fault tolerant
• ease of management

From Cloud IAAS (mainly from Eucalyptus and OpenStack) :

• on-demand virtual environment
• automatic provisionning

Consider existing context

An important constraint was to not break with existing practices or software compatibility. For example, using Hadoop Distributed FileSystem as-is would have required somes changes in softwares or user practices, it was not possible. Here is a non exhaustive list of constraints :

• User practices matter
• NFS like access to the data
• Keep existing sofware compatible with the new platform
• Day to day processing environments switch or cohabitation