| |
corosync_overview(8) - phpMan
COROSYNC_OVERVIEW(8) Corosync Cluster Engine Programmer's Manual COROSYNC_OVERVIEW(8)
NAME
corosync_overview - Corosync overview
OVERVIEW
The corosync project is a project to implement a production quality "Revised BSD" licensed
implementation of the most recent SA Forum's Application Interface Specification. The
Application Interface Specification is a software API and policies which are used to
develop applications that maintain service during faults. The API consists of Availabil‐
ity Management Framework (AMF) which provides application failover, Cluster Membership
(CLM), Checkpointing (CKPT), Eventing (EVT), Messaging (MSG), and Distributed Locking
(DLOCK).
Currently Messaging is unimplemented.
Faults occur for various reasons:
* Application Faults
* Middleware Faults
* Operating System Faults
* Hardware Faults
The major focus of high availability in the past has been to mask hardware faults. Faults
in other components of the system have gone unsolved until AIS. AIS can mask many types of
faults in applications, middleware, operating systems, or even hardware by providing a
simple framework for allowing developers to create redundant applications. These redundant
applications can be distributed over multiple nodes such that if any one node faults,
another node can recover.
Application programmers develop applications to periodically record their state using the
checkpointing service. When an active application fails, a standby application recovers
the state of the application. This technique, called stateful application failover, pro‐
vides the fundamental difference between corosync and other systems that have come before
it. With stateful application failover, the end-application user doesn't have to reload
the application or redial a telephone. The full state is recorded, so the end-application
user sees no interruption in service.
Because programmers can now distribute applications across multiple processes or nodes, a
mechanism must exist for them to communicate. This mechanism is provided by two services.
The event service provides a publish/subscribe model for events. The messaging service
provides end to end messaging. Finally a mechanism to synchronize access is provided by
the distributed lock service.
The corosync project also provides a group messaging toolkit called EVS. The EVS service
implements a messaging model known as Extended Virtual Synchrony. This model allows one
sender to transmit to many receivers. Certain guarantees are provided for message and
membership delivery which make virtual synchrony ideal for developing distributed applica‐
tions.
QUICKSTART
The corosync executive must be configured. In the directory conf in the source distribu‐
tion are several files that must be copied to the /etc/corosync directory. If corosync is
packaged by a distro, this may be complete.
The directory contains the file corosync.conf. Please read the corosync.conf(5) man page
for details on the configuration options. The corosync project will work out of the box
with the default configuration options, although the administrator may desire different
options.
The corosync executive uses cryptographic techniques to ensure authenticity and privacy of
the messages. In order for corosync to be secure and operate, a private key must be gen‐
erated and shared to all processors.
First generate the key on one of the nodes:
unix# corosync-keygen
Corosync Cluster Engine Authentication key generator.
Gathering 1024 bits for key from /dev/random.
Press keys on your keyboard to generate entropy.
Writing corosync key to /etc/corosync/authkey.
After this operation, a private key will be in the file /etc/corosync/authkey. This pri‐
vate key must be copied to every processor in the cluster. If the private key isn't the
same for every node, those nodes with nonmatching private keys will not be able to join
the same configuration.
Copy the key to some security transportable storage or use ssh to transmit the key from
node to node. Then install the key with the command:
unix#: install -D --group=0 --owner=0 --mode=0400 /path_to_authkey/authkey
/etc/corosync/authkey
If a message "Invalid digest" appears from the corosync executive, the keys are not con‐
sistent between processors.
Finally run the corosync executive. If corosync is packaged from a distro, it may be set
to start on system start. It may also be turned off by default in which case the init
script for corosync must be enabled.
After running aisexec, a list of all processors IP addresses running the corosync execu‐
tive and configured on the same multicast address will appear. If they don't appear,
there may be a problem with multicast in the distro or hardware. If this happens, partic‐
ipation in the corosync mailing list may help solve the problem. The email address is
openais AT lists.org.
USING LIBRARIES
The corosync AIS libraries have header files which must be included in the developer's
application. Once the header file is included, the developer can reference the AIS inter‐
faces.
The corosync project recommends to distros to place include files in
/usr/include/corosync. The following include lines must be added to the application to
use each of the following services:
#include <corosync/saClm.h> For the Cluster Membership B.01.01 service.
#include <corosync/saCkpt.h> For the Checkpointing B.01.01 service.
#include <corosync/saEvt.h> For the Eventing B.01.01 service.
#include <corosync/ais_amf.h> For the AMF A.01.01 service.
The corosync project recommends to distros to place library files in /usr/lib. The fol‐
lowing link lines must be added to the LDFLAGS section of the makefile.
-lsaClm For the Cluster Membership B.01.01 service
-lsaCkpt For the Checkpointing B.01.01 service
-lsaEvt For the Eventing B.01.01 service
-lsaAmf For the AMF A.01.01 service
-lais Specify this to get access to all AIS libraries without specifying each library
individually.
IPv6
The corosync project supports both IPv4 and IPv6 network addresses. The entire cluster
must use either IPv4 or IPv6 for the cluster communication mechanism. In order to use
IPv6, IPv6 addresses must be specified in the bindnetaddr and mcastaddr fields in the con‐
figuration file. The nodeid field must also be set.
An example of this is: nodeid: 2 bindnetaddr: fec0::1:a800:4ff:fe00:20 mcastaddr: ff05::1
To configure a host for IPv6, use the ifconfig program to add interfaces: box20: ifconfig
eth0 add fec0::1:a800:4ff:fe00:20/64 box30: ifconfig eth0 add fec0::1:a800:4ff:fe00:30/64
If the /64 is not specified, a route for the IPv6 network will not be configured which
will cause significant problems. Make sure a route is available for IPv6 traffic.
ARCHITECTURE
The AIS libraries are a thin IPC interface to the corosync executive. The corosync execu‐
tive provides services for the SA Forum AIS libraries as well as the EVS and CPG
libraries.
The corosync executive uses the Totem extended virtual synchrony protocol. The advantage
to the end user is excellent performance characteristics and a proven protocol with excel‐
lent reliability. This protocol connects the processors in a configuration together so
they may communicate.
ENVIRONMENT VARIABLES
The corosync executive process uses four environment variables during startup. If these
environment variables are not set, defaults will be used.
COROSYNC_MAIN_CONFIG_FILE
This specifies the fully qualified path to the corosync configuration file.
The default is /etc/corosync/corosync.conf.
COROSYNC_AMF_CONFIG_FILE
This specifies the fully qualified path to the corosync Availability Management
Framework configuration file.
The default is /etc/corosync/amf.conf.
COROSYNC_DEFAULT_CONFIG_IFACE
This specifies the LCRSO that is used to parse the configuration file. This allows
other configuration file parsers to be implemented within the system.
The default is to use the default corosync configuration file parser which parses
the format specified in corosync.conf (5).
COROSYNC_TOTEM_AUTHKEY_FILE
This specifies the fully qualified path to the shared key used to authenticate and
encrypt data used within the Totem protocol.
The default is /etc/corosync/authkey.
SECURITY
The corosync executive optionally encrypts all messages sent over the network using the
SOBER-128 stream cipher. The corosync executive uses HMAC and SHA1 to authenticate all
messages. The corosync executive library uses SOBER-128 as a pseudo random number genera‐
tor. The EVS library feeds the PRNG using the /dev/random Linux device.
If membership messages can be captured by intruders, it is possible to execute a denial of
service attack on the cluster. In this scenario, the cluster is likely already compro‐
mised and a DOS attack is the least of the administration's worries.
The security in corosync does not offer perfect forward secrecy because the keys are
reused. It may be possible for an intruder by capturing packets in an automated fashion
to determine the shared key. No such automated attack has been published as of yet. In
this scenario, the cluster is likely already compromised to allow the long-term capture of
transmitted data.
For security reasons, the corosync executive binary should NEVER be setuid or setgid in
the filesystem.
SAFTEST COMPLIANCE
The corosync libraries are now nearly compliant with every aspect of the SA Forum's AIS
specification. The AMF service, however, is not compliant with the B.01.01 specification.
The remaining services pass most of the tests of the saftest suite against the B.01.01
specification.
BUGS
The messaging service is partially implemented and not suitable for deployment. The dis‐
tributed locking service is buggy and not suitable for deployment. The Availability Man‐
agement Framework is under development and not suitable for deployment..
SEE ALSO
corosync.conf(5), corosync-keygen(8), evs_overview(8)
corosync Man Page 2006-05-10 COROSYNC_OVERVIEW(8)
|
