This document was started on June 06, 2002 by Ryan Eatmon to explain how to set up a Jabber Server Farm using the Jabber.org Jabber Server (http://jabberd.jabberstudio.org).
This document is broken into two main sections. What is, and what will be. As of the writing of this document the Jabber Server does not have all of the pieces that are required to do full farming. There are steps that can be taken to get partial farming, and those will be covered. There are also steps that we are going to take, and those are covered to so that others can contribute and comment.
Jabber is an Open Source Instant Messaging System that uses XML as its base protcol. I could spend an entire How-To describing Jabber, so I am just going to point you to the Jabber.org web site. (http://www.jabber.org).
Copyright 2002 Ryan Eatmon
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the appendix entitled "GNU Free Documentation License".
Jabber is a Trademark of Jabber, Inc.
The Jabberd server is a reference implementation for the Jabber protocol as defined by the Jabber Software Foundation. It is written in C and is covered by both the Jabber Open Source License(JOSL) and the GNU Public License(GPL). The Jabberd project has its own documentation. (http://jabberd.jabberstudio.org/howto.html).
This is an interesting topic. When you think of a server you tend to think of a single machine running a single program. Farming is the idea of multiple machines running multiple programs that all act together to appear to be one machine/program.
Why farming? Two reasons with equal importance. Scalability and Reliability.
Scalability is the idea of allowing something to handle many many more connections/transactions at the same time. A typcial web server can handle say 1000 connections a second, but 10 web servers acting together can provide 10,000 connections per second.
Reliability is the idea that even if a piece of the system goes down, the system does not. There is no single point of failure, and there is built in redundancy to ensure that something is there to pick up the slack if something goes down. In the web server example, if one web server goes down you could not send any connections to it until it is back up, thus ensuring that your web site is always up.
Both of these ideas are crucial to a successful Farming strategy. Each wants to push the design into a different direction, but they can be reconciled into a single design that can scale and is reliable at the same time.
The first thing that we can do is to split out the c2s from the server and make it a seperate process/component. Once that is done, we can replicate the c2s processes over multiple machines. Each one would connect back to the main server running the JSM and everything should work just fine.
This is doable right now. The 1.5 version of jabberd has sought to break all of the pieces out into seperate processes. This was not done for farming specifically, but we will not complain since the jadc2s component can handle upwards of 10k users. (The 1.4.x series c2s could only handle ~1024).
Currently there is a forked version of jadc2s that works with the 1.4.2 server. It is located in the jabberd14 CVS repository on JabberStudio. The following example is running with two jadc2s boxes, and one central jabberd box. To set this up do the following:
Now that we have the c2s processes distributed across machines, we need to get them to work together. We need two things, the ability to communicate how many connections a c2s has to the others, and the ability to know who the others are.
[22:21] Jer: well, the "protocol" could just be an auth error of 3xx for redirect w/ the ip/port to connect to
Maybe we need to talk about the router layer sooner...