[Draft] Communication between server controllers

[TatharNuar]

I'd love to see this work, but in order for it to work seamlessly, server controllers need a small number of tweaks to their chat interpreters to address the fact that they can't assume they're running alone anymore. I've compiled a list of foreseeable issues that all server controllers need to address before they can work seamlessly alongside other controllers.

• All server controllers that don't use a particular command will display a "command invalid" error, even if another server controller does use that command. This cannot be completely addressed through the use of plugins, because the server controller itself is responsible for the improper error message. Key points for a solution include:
  • Server controllers must know at all times which other server controllers are running on a server. To accomplish this, when a server controller starts, it must broadcast its presence through the proposed communication protocol and ask what other server controllers are present. The other server controllers then broadcast their presence. Also, when a server controller is shut down, it should announce that via the protocol. Finally, to detect server controller crashes and improper shutdowns, each server controller must periodically send out a "ping" broadcast to tell the other controllers that they're still connected.
  • Server controllers must communicate which commands they use to each other. The proposed communication protocol would facilitate this. This can be done through the use of plugins, or directly through the server controllers' chat interpreters. When a server controller is no longer connected, its commands should then be unregistered with the other controllers' chat interpreters.
  • Upon finding out that a command is used by another server controller, each server controller's chat interpreter should register that command in its own command list and have the command either pass to an empty method or be ignored.
  • All server controllers except one must suppress their "command invalid" errors even if a command truly is invalid. Otherwise, the error message will be displayed multiple times for the same command. It is up to the server owner or the server controllers to decide which server controller will display this error message.
• Server controllers should communicate specifically whether they have a help system accessed by chat command. All server controllers except one should disable this feature, and if that controller is disconnected, then the others must then decide whether one should take its place or to have no replacement system.
• If any server controllers contain a help system that provides basic help information on a command (like ManiaLive's /man command, but not /help) then the system should be disabled, unless it is the only one enabled and it can get or generate the necessary data for the other controllers' commands. Standardization of this data collection is important, but beyond the scope of this list.
• If a controller needs to use ChatEnableManualRouting() to manipulate player chat messages, it should communicate this to the other controllers. Only one server controller should route player chat messages, or else multiple transformed messages will be displayed for every one message sent by a player.

I believe that's all that needs to be done aside from standardizing a protocol for Echo() based communication that plugins can also use. GUI window collision isn't a big deal as long as window placement is configurable, since collision detection can detect desired collisions. Besides, plugins for the same controller can have GUI collisions, and I've yet to see any server owners complain.

First, we're using the Echo method so here are a few things to know before starting:

• Messages are sent by calling Echo method on the dedicated
• Message are received by the callback ManiaPlanet.Echo
• Echo callback is always sent to everyone
• Echo method and ManiaPlanet.Echo callback have 2 arguments but there are not in the same order (which most probably won't be fixed)

I didn't know that Echo and the callback switched the argument order. I was planning on using an Echo-based protocol for my ManiaLive plugins to communicate with other server controllers, and I never would have thought that the arguments would be in reverse order. Thanks for letting us know.

EDIT: I was missing some things that server controllers and plugins need to do to be multi-controller aware. I haven't finalized it yet, but I have a draft specification to give server controllers and plugins MCA functionality. If you want a copy of what I have now, please send me a private message.

[TatharNuar]

So now, here is how it works at the moment...

Format

Code: Select all

Echo(sender, message) to send a message
ManiaPlanet.Echo(message, sender) when receiving a message

As stated before, arguments are not in the same order in the method and in the callback. Each argument is a string.

Code: Select all

Message:
    (?|!)method[:recipient]

? is used for requests while ! is used for answers when receiving a request (an answer is not mandatory for every request)
method is a single word
:recipient is optional but used when you want a single system to take care of the message (meaning other systems must ignore it)


Identifiers

Senders and recipients have to be a string, preferably unique. As an example, we have at the moment "ManiaLive 2.4" and "DedicatedManager 1.5". It's not perfect as it doesn't ensure unicity (eg. if you have 2 instances of ManiaLive 2.4) but that's why it's only a draft.


Methods

census: ask systems to identify themselves.

Code: Select all

Sent request:
    Echo( DedicatedManager 1.5 , ?census )

Received answer:
    ManiaPlanet.Echo( !census:DedicatedManager 1.5 , ManiaLive 2.4 )

stop: ask a system (or all) to stop. No answer expected

Code: Select all

Sent request:
    Echo( DedicatedManager 1.5 , ?stop:ManiaLive 2.4 )

And that's all to know (I think). Feel free to comment, contribute or implement it in your controllers!

I've been working on a slightly different (and incompatible!) protocol using Echo that addresses the issues with this one. (Technically, it is multiple protocols: one for how to use Echo, and a few built upon it that were specific to my plugins.)

• The Echo(sender, message) part contains a few issues:
  • Instead of "sender", the first argument should be reserved for metadata, which includes the following with a space delimiter:
    1. An abbreviation for the top-level protocol being used. This reduces conflicts with other protocols using Echo. In my draft, I decided on "XCTRL" (pronounced "cross control") for the Echo framework protocol and its control messages, "XTALK" for the protocol built on top of the XCTRL protocol for plugins to share generic data with each other, "DRGN" for the protocol for my Dragon Console plugin to tell other controllers' plugins to move, hide, or show their GUI windows, as well as related commands, and "XCMD" for the protocol to communicate chat command registration data with the other controllers present. Keep in mind that many other protocols (and some file formats) also do this for the same reason.
    2. An 8-digit hexidecimal number generated by the sender in sequential order. It doesn't matter if other senders use this number, but the same sender must not use the same number twice. If the number reaches FFFFFFFF, then the next generated number should be 00000000. This is most useful for ACK return messages to ensure that all recipients got the message, but can be used to identify the specific message for other reasons too.
    3. The sender's ID. Unlike your draft, I felt the need to identify plugins as well. The sender ID was in the form "ControllerName\PluginAuthor\PluginName\InstanceNumber" and was in PascalCase with no spaces. For server controllers, ControllerName has "!" added to the end, and PluginAuthor and PluginName can be omitted, but should be used to identify the specific part of the controller used. The InstanceNumber is a random 4-digit hexidecimal number generated when the sending plugin or controller is loaded. In the very rare event of a sender ID collision, the controller already using that ID would send out an XCTRL control message and both controllers would generate a new InstanceNumber and announce their new sender ID. I specifically left out version numbers as they could contain spaces or make the metadata too long, and if anything needed to know a specific sender ID's version number, it could be obtained through an XCTRL control message. Also, anything starting with "Broadcast" is reserved for the next part and cannot be used as an ID.
    4. The recipients of the message. This can either be a space-delimited list of recipient IDs, a space-delimited list of partial recipient IDs, "Broadcast", or anything starting with "Broadcast\". Broadcast IDs indicate all connected controllers are the recipients, and partial recipient IDs match to all recipient IDs that start with the given partial ID. This is most useful for telling all instances of a server controller or plugin the same thing.
  • The message can be any string, depending upon the top-level protocol's rules. This is important, since non-compliant and badly-formed Echo calls could send any message they want. An draft list of XCTRL control messages include:
    • "PING (Timestamp) (TimeoutInSeconds)" - Always sent to "Broadcast\Ping" and announces that the sender is (still) connected. Should be sent when an XCTRL-compliant sender connects to the server and periodically after. "Timestamp" is the same as PHP's time(), and server controllers should assume that the sender is connected to the server until "TimeoutInSeconds" seconds after "Timestamp". Generally, the sender should send out at least one more PING before this time expires, plus a buffer to allow time for other controllers to receive the next PING. When receiving a PING, each server controller should send an ACK message or a CONFLICT message, depending on whether that controller uses the sender's ID or not. However, if there is something wrong with the PING message, a NAK should be returned. (I'm also considering whether to have a BADSID message sent if specifically the sender ID is malformed.) This message can also be sent to poll the server for XCTRL-compliant server controllers, like your "census" method does.
    • "ACK (MessageID)" - Positively acknowledges a message being received. "MessageID" is the same 8-digit message ID as the Echo message that was received, and along with the recipient ID in the metadata, identifies a specific message. This message must be sent to a single full recipient ID, or it will not make sense.
    • "NAK (MessageID)" - Negatively acknowledges a message being received, indicating an error or problem with the received message. "MessageID" is the same 8-digit message ID as the Echo message that was received, and along with the recipient ID in the metadata, identifies a specific message. This message must be sent to a single full recipient ID, or it will not make sense.
    • "CONFLICT" - Identifies a conflict on the sender ID for any message that wasn't sent by this controller or plugin. Sent only to "Broadcast\Conflict". When this message is received, all server controllers and plugins should consider the sender ID invalid for future messages, and all controllers or plugins using the sender ID should generate a new InstanceNumber.
    • "STOP" - Tells the recipient IDs to disconnect from the server and shut down. Any number of parameters can be used to authenticate the sender, but other server controllers can see that, so it shouldn't be used to stop rogue server controllers. Recipients should first send an ACK and a STOPPING message (or a NAK if the STOP metadata or message is malformed) before disconnecting. This allows the sender of this message to confirm that the recipient has indeed disconnected. This is equivalent to your stop method, except it allows for confirmation that the recipient is shutting down.
    • "STOPPING" - Tells all server controllers that the sender is disconnecting. Must be sent to "Broadcast\Stopping". No ACK response is needed.
    • "PGET" - Asks the recipient which custom XCTRL-compatible protocols it supports. The recipient responds with an ACK followed by a PNUM message, a list of PSUP messages, and a PEND message.
    • "PNUM (NumberOfProtocols)" - Tells the recipient how many custom protocols it supports. This must be followed by the appropriate number of PSUP messages and a PEND message, or the recipient should send a NAK for the PNUM message. Requires ACK response from the recipient, even if a NAK is sent later.
    • "PSUP (ShortName) (AuthorLogin) (LongName)" - Tells the recipient about one of the custom protocols supported by the sender. "ShortName" is the protocol's identifier as used in its metadata. "AuthorLogin" is the ManiaPlanet login of the protocol's author, or the current lead author's login if it was created by multiple people. If the protocol was created by Nadeo, then "AuthorLogin" should be "Nadeo", case sensitive, otherwise the login should be all lowercase. "LongName" is the full name and version number of the protocol. This typically follows a PNUM message or another PSUP message, but can be used to ask a recipient whether it is compatible with the sender's protocol. The recipient should send an ACK message if it is known compatible with the protocol, a PCONFLICT message if it is not, and a NAK if there is something wrong with the PSUP message.
    • "PCONFLICT (ShortName) (AuthorLogin) (LongName)" - Tells the recipient that its "ShortName" protocol is believed to be incompatible with the sender's version of that protocol. The recipient should send an ACK message if it is compatible with the protocol, a PCONFLICT message if it is not, and a NAK if there is something wrong with the PCONFLICT message. If an ACK is sent and the protocols use commands, then both protocols should ignore any commands that are not found in themselves.
    • "PEND" - Tells the recipient that it is done sending PSUP messages. This must follow a PNUM or PSUP message, but can be omitted if no protocols are used. Requires an ACK response to ensure it is received.
• The question and answer approach is unnecessary, and presents issues that are addressed via my ACK and NAK control messages. Namely, with ? and !, there is no way to properly handle problems with the received message.
• I'm leaving out some error handling information since I haven't finished it yet, but error handling will be included.
• I believe my XCTRL protocol adequately covers what you're trying to accomplish, and it includes essential features for server controllers to be multi-controller aware. I designed it based on how TCP works, so it should be suitable for your needs.

IMPORTANT REVISION: To address a pair of issues with the above specification, I need to make a few changes to the metadata part of the above specification. Prior to the protocol designation at the start of the metadata, all XCTRL-compliant metadata strings must start with "X! " or "X? ", excluding quotes, but including the space. Non-compliant protocols must not send messages with the metadata string starting with "X! " or "X? ", or else servers will be unable to use those protocols with XCTRL. Also, protocol designations must be entirely alphanumeric to be valid, or a NAK message should be sent. That makes XCTRL-compliant metadata strings follow the following format: "X(!|?) ProtocolName MessageID SenderID RecipientID1 RecipientID2 ... RecipientIDn". The difference between "X! " and "X? " is that "X! " is used for messages that don't require an ACK response from the recipients, while "X? " is used for messages that do.

The message string still has no restrictions, except as required by the protocol designated in ProtocolName. Protocols should specify when server controllers and plugins should expect an "X! " message or an "X? " message so they can better handle badly formed metadata.

As for the other protocols mentioned in this post, XTALK places no restrictions on the message string, so it is best used for sending raw data to other controllers and plugins. DRGN and XCMD are not finished yet, though XCMD will go through regular revisions to satisfy each server controller's help systems' requirements, and XCTRL's protocol support commands will be very useful for keeping up with the changes.

REVISION 2: I am changing the AuthorName part of the sender and recipient IDs to fix an issue that could come up later. The AuthorName can be in any of the following formats:

• "login.loginname" - The Maniaplanet login of a single author. For group authors, the lead developer should be used. The login name itself must be alphanumeric and all lowercase. Example: "login.tathar".
• "tmlogin.loginname" - The Trackmania 1 login of a single author. For group authors, the lead developer should be used. The login name itself must be alphanumeric and all lowercase. Example: "tmlogin.tatharnuar".
• "manialink.linkname" - A Manialink page owned by the author and registered with Maniaplanet, preferably the one associated with the server controller or plugin. Example: "manialink.dragon".
• "tmlink.linkname" - A Manialink page owned by the author and registered with Trackmania 1, preferably the one associated with the server controller or plugin. Example: "tmlink.example".
• "www.tld.domain.subdomain" - A domain name or subdomain owned by the author, preferably the one used to download the server controller or plugin. The domain name must be displayed in reverse order. Websites using free hosts cannot use this format unless they can be accessed from the root folder of a subdomain. Example: "www.com.xaseco.www".
• "project.tld.domain.subdomain.projectname" - The server controller or plugin's project on a source control website, such as Google Code, github, or Sourceforge. The source control website must be displayed in reverse order, followed by the project's name from its homepage URL. Ideal for open-source projects by multiple authors. Example: "project.com.google.code.manialive".
• "email.tld.domain.username" - An email address owned by the author. This should be used only if each of the above options can't be used, since if an author wants to be contacted by email for support, they can communicate that in better ways. Example: "email.com.gmail.example".
• "alias.authorname" - A name used to identify the author of a plugin with the plugin's server controller. This is useful for server controllers to convey XCTRL compliance to their plugins, but this should only be used as a default value in case the plugin does not specify a different AuthorName. Example: "alias.DragonTeam".
• "name.surname.firstname" - The name of a single author, in reverse word order. This should not be used unless all other options are unavailable and the author cannot sign up for a service that would make one of the other options available. It should never be used by anyone who values their privacy. No example for this one, for obvious reasons.

REVISION 3: OK, my memory of how TCP works is definitely rusty. The above Echo protocol won't work as intended, but it does describe which features need to be maintained for an Echo communication protocol to work properly. I'll work on my draft protocol a bit more and post it when I'm more confident that it would work. I've already completed the specification that describes what server controllers and plugins need to do so they can be multi-controller aware, regardless of which Echo communication protocol they use. Please send me a private message if you want a copy of that specification.

[Bueddl]

In my opinion, this is far to complex. It has not to be that complex, so why make it?

Keep in mind, that is not very performant to flood the server with echo commands as they get delivered to every (xml-rpc) client connected.

I think there should be more a focus on a smart, not redundant and easy to implement (and for sure use) protocol.

I think the initial post was a good start, but should be extended with some features.
e.g. there should be a method to ask the controller about the supportet methods.

May be we should define our targets first and build the protocol up from this.

Feel free to complete this list:

Targets:

• simple communication/data exchange
• "remote" execution of commands (e.g. stop a controller)
• sharing custom collbacks (e.g. controller is stopping)
• easy implementation
• ...

What this protocol is not designed for:

• authentication or permission watching
• access to methods that are not addressed to a controller (keep server features on xmlrpc, not on this!)
• ...

Cheers,
Bueddl

[TatharNuar]

The initial post is incompatible with protocols introduced by other authors, unfortunately. There is nothing in that protocol that keeps third-party echo methods separate from Nadeo's methods, and there is no collision control.

While I agree that some parts of my draft are unnecessary, like the length of the sender IDs, it is just a draft, and I haven't done anything to optimize it. However, the majority of XCTRL's functions are in fact necessary, due to the shortcomings apparent in Nadeo's protocol and the others here. More than anything, what I'm trying to do is allow for additional protocols to be defined and used alongside XCTRL that would be more lightweight or offer additional, custom features.

Unfortunately, the nature of Echo is such that, unless you reintroduce those features from TCP/IP, you would be unable to tell who is sending what to where, and whether the recipient is even listening. This necessitates many of the control commands in XCTRL, since you typically need to ensure that the data is delivered to the recipient and acted upon. One of the first optimizations I would make would be to change the control commands to binary flags in the metadata so there's less data being sent to everything.

Believe me, I really don't like that Echo is always broadcast to everything on the server. This isn't secure, and it slows down any communication protocol. However, we can't do anything about this, so I'm working with what I can. Anyway, I'm going to wait until I'm actually looking to implement a later XCTRL draft in my plugins before I make the optimizations that would address your concerns.

EDIT: What I mean by the initial protocol being incompatible is that third-party methods using that protocol can easily conflict with each other or Nadeo's methods. I managed to make XCTRL mostly capable of working alongside the Nadeo protocol in a recent revision (can't remember which), but the Nadeo protocol's structure is unable to function properly in many of the circumstances that make multi-controller awareness so powerful.

The key here is not to make a protocol that's trivial to implement. First, you need it to be robust and capable of meeting the needs of the use cases that make multi-controller awareness useful in the first place. Of course, it's best if a server controller manages any Echo protocol's use so that plugins don't individually have to implement the protocol. Failing that, a single plugin that other plugins make use of can perform this function.

[Bueddl]

This isn't secure,

Aggree here.

and it slows down any communication protocol.

In my opinion, thats not the point of that procotol to be secure robust yes, but not secure in any meaning of security.

The key here is not to make a protocol that's trivial to implement. First, you need it to be robust and capable of meeting the needs of the use cases that make multi-controller awareness useful in the first place.

Didnt say that. But you have much overhead and it is far too complex. It simply has not to be that complex. And for sure it is a big goal to make it easy to implement as this also implys that the protocol isn't that complex to understand and work with.

[TatharNuar]

In that case, could you list which parts are unnecessary? I'd like to comment on each individually since, admittedly, the purpose behind some of the features is not immediately apparent. Also keep in mind that the protocol I mentioned is deficient in several areas that I'm fixing, so I'll mention that if you list any parts that I'm fixing up.

For example, I really don't like how the sender IDs are so long, but having a hierarchical sender ID allows recipients to identify a sender based on any of: instance number, plugin name, plugin author, and server controller. I'd like to preserve this function since it is a necessary part of the software supporting a Maniaplanet server cluster. Being able to use multiple server controllers together is a boon for any server owner, but large servers (read: your servers) would get an even greater benefit from being able to load balance plugins' work across multiple machines. I'm looking into shrinking the sender IDs into an IP address-like identifier, though I don't have specifics yet.

Also I'm splitting up XCTRL so it's not a single monolithic protocol. Not everything needs to fully implement XCTRL to be compatible, so I'm identifying what plugins and server controllers need to implement individual parts and designing a protocol for each set of parts that are used by the same group of plugins and server controllers. This is something I've adapted from the TCP/IP suite to reduce overhead and make it easier to implement XCTRL. All plugins and server controllers need to follow a certain set of behavior rules to be multi-controller aware (MCA), however, or else they will cause unexpected behavior. I have the MCA part of the next draft completed already, but it's on another forum that requires registration and manual verification to view.

Anyway, I'm trying to make XCTRL suitable for three major use cases:

• Being able to use multiple server controllers on the same server, without the server controllers or their plugins interfering with each others' functions. This is good for server controller and plugin authors who aspire to have their works used by a larger percentage of servers.
• Being able to use multiple versions of the same server controller, in case a new version of the server controller breaks a plugin's functionality that server owners like. This is something I'm working on mainly for ManiaLive's benefit, since it has a lot of plugins that don't work with newer versions. It's a side benefit for other server controllers with plugin support.
• Being able to use multiple instances of the same server controller and version, typically across multiple server machines, to load balance the work between each of them. This is huge for large servers, since under normal circumstances, a server with many players connected would experience performance issues associated with the server controller maintaining numerous plugin windows for each player and spectator.

For this to work, XCTRL and the underlying MCA design rules have to be designed to be robust and extendable enough to work reliably. Ease of implementation is a factor to consider for how easily the protocols will be adopted, but it has to be prioritized after the traits necessary to accomplish these three goals.

EDIT: What I'm trying to emphasize about Echo is that broadcasting everything reduces performance whenever you're trying to send information to one or few recipients. Security would be useful in certain circumstances, but really there isn't anything secure about the server API. Still, security isn't my main complaint with Echo.

Also, I need to clarify that I'm only designing these protocols to use Echo for communication. I'm not sure whether other XML-RPC calls get broadcasted anyway, and I'm not establishing any protocols for their use. I'm a plugin author, not a server controller author, so I can't speak to how XML-RPC calls are actually made. If a plugin needs to tell the other server controllers that it did something, then it should probably use Echo to broadcast that anyway. XCTRL will support custom protocols that define a standardized way to announce such actions, especially since having two Dedimania plugins write the same record to the Dedimania server is a bad idea, and they need to be able to communicate with each other reliably to decide which one will perform Dedimania writes for the server.

Anyway, I want to make sure that everyone knows that the XCTRL draft posted above is not a final draft, or even developed enough for server controllers to implement. In fact, I'm telling server controller authors not to implement the protocol as written above, but to wait for me to finalize the protocol before implementing it. However, it is an important first step toward having a protocol available that meets the needs of the use cases I listed here. The other proposed protocols do not meet these needs.

I have to study the TCP/IP protocols again to refresh my memory on how they work, so it could be a while before I'm ready to take another shot at revising the protocol draft. Since TCP/IP is well established, it's a good starting point for how to make a set of Echo protocols that are robust and extensible. I'm also prioritizing my plugin suite's development since there isn't a pressing need to have this done right away. If other developers contact me about getting a draft together, then that could change, but otherwise I'm waiting until I've finished the basic plugins in my plugin suite and can release something to the public.

[askuri]

To all controller developers: which protocol do you use for echo in your controller? I'm currently implementing this in UASECO, and want to rely on the most commonly used one. It seems like everybody is inventing his own protocol here :/
Thanks

[oliverde8]

eXpansion being base on Ml uses the same protocol. We used it at one point to have multiple isntances of eXp running for various tasks but at the moment the only usage we have is to prevent multiple instances of the controller to run together.

I would think of a library actually to handle those communications in a transparent manner whatever the controller.