Eucalyptus
Projet intéressant car il permet de créer son propre cloud, en supportant les api EC2 d’Amazon. Le projet permet de manager un cloud, et le déploiement d’applis. Il devient ainsi possible de créer son propre cloud compatible EC2, et de décider de migrer ses applications vers EC2 si besoin, ou dans le sens contraire, rapatrier à la maison, une application tournant sur EC2.
Je n’ai pas encore eu le temps creuser plus loin, et ne sais donc pas si on peut ou pas gérer des clouds mixte, capable de s’étendre sur un cloud extérieur en cas de besoin, mais c’est sur que se sera la norme demain…
Présentation : Eucalyptus : Open Source for Cloud Computing
Site web : http://eucalyptus.cs.ucsb.edu
Canonical annonce le support d’Eucalyptus dans la version 9.10 d’Ubuntu, a lire sur le sujet un papier d’O’Reilly Radar
Le support d’Eucalyptus par Canonical en fait un produit à suivre, il est évident qu’un standard est en préparation dans le coin…
AppScale
Port de google app engine sur EC2 et Eucalytus
Aucune release, mais les sources sont disponible en lecture.
Site web : http://code.google.com/p/appscale/
Open Nebula
OpenNebula transforms a physical cluster into a flexible virtual infrastructure which dynamically adapts to the changing demands of a service workload. OpenNebula leverages existing virtualization platforms to create a new virtualization layer between the service and the physical infrastructure. This new layer supports the execution of the services on a physical cluster, extending the benefits of VMMs (Virtual Machine Monitors) from a single physical resource to a cluster of resources. OpenNebula effectively decouples a server (deployed as a pre-configured VM) not only from the physical infrastructure but also from its physical location.
http://www.opennebula.org
Des projets intéressants dans la section eco-system
Ganeti
Ganeti provides the following features for managed instances :
- Support for Xen virtualization :
- Full support for paravirtualized instances
- Experimental support for fully-virtualized (HVM) instances
- Experimental live migration support
- Recommended cluster size 1-25 physical nodes
- Disk management using either plain LVM volumes or across-the-network raid1 (using DRBD) for quick recovery in case of physical system failure
- Export/import mechanism for backup purposes or migration between clusters
http://code.google.com/p/ganeti/
DRBD
DRDB refers to block devices designed as a building block to form high availability (HA) clusters. This is done by mirroring a whole block device via an assigned network. DRBD can be understood as network based raid-1.
Les classiques…
Google AppEngine
Google App Engine enables you to build web applications on the same scalable systems that power Google applications. App Engine applications are easy to build, easy to maintain, and easy to scale as your traffic and data storage needs grow. With App Engine, there are no servers to maintain: You just upload your application, and it’s ready to serve to your users.
http://code.google.com/intl/fr-FR/appengine/
Amazon EC2
Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers.
Amazon EC2’s simple web service interface allows you to obtain and configure capacity with minimal friction. It provides you with complete control of your computing resources and lets you run on Amazon’s proven computing environment. Amazon EC2 reduces the time required to obtain and boot new server instances to minutes, allowing you to quickly scale capacity, both up and down, as your computing requirements change. Amazon EC2 changes the economics of computing by allowing you to pay only for capacity that you actually use. Amazon EC2 provides developers the tools to build failure resilient applications and isolate themselves from common failure scenarios.
http://aws.amazon.com/ec2/
Et pour finir
Hadoop
Le meilleur pour la fin (à mon humble avis…), hadoop, la techno de cloud open source à suivre, qui s’inspire des clouds de google. Basé sur Map Reduce et HDFS, son file system distribué permettant de stocker les metadatas et de traiter les les requêtes decoupés, comme mutli-threadé.
Le site du projet
http://hadoop.apache.org/
Les principaux éléments du projet, extrait de la page d’accueil du projet
Hadoop Core, our flagship sub-project, provides a distributed filesystem (HDFS) and support for the MapReduce distributed computing metaphor.
HBase builds on Hadoop Core to provide a scalable, distributed database.
Pig is a high-level data-flow language and execution framework for parallel computation. It is built on top of Hadoop Core.
ZooKeeper is a highly available and reliable coordination system. Distributed applications use ZooKeeper to store and mediate updates for critical shared state.
Hive is a data warehouse infrastructure built on Hadoop Core that provides data summarization, adhoc querying and analysis of datasets.
La doc
http://hadoop.apache.org/core/docs/current/
un aperçu tourné dev
http://www.us.apachecon.com/presentation/materials/54/ProgrammingHadoop-ApacheConUS08.pdf
Une explication du fonctionnement de Map Reduce
http://java.dzone.com/articles/how-hadoop-mapreduce-works
base de données au dessus d’hadoop, faite pour gerer des milliards de lignes…
http://hadoop.apache.org/hbase/
Moteur de recherche utilisant lucerne sur hadoop
http://wiki.apache.org/nutch/NutchHadoopTutorial
Howto install
http://www.michael-noll.com/wiki/Running_Hadoop_On_Ubuntu_Linux_(Single-Node_Cluster)
Improving Virtualization and Performance Tracing of Hadoop with OpenSolaris
http://opensolaris.org/os/project/livehadoop
une petite liste d’utilisateurs d’Hadoop, que des belles références…
http://wiki.apache.org/hadoop/PoweredBy