[Scripts] Keeping your servers up without affecting players

[Harest]

Hi,

Quite a long time ago i noticed the issue we all have with our servers : disconnection from main server after 20+ days +/-. I assume depending on who is managing your server, you may not even be aware of it. This thread will mainly be for anyone having his/her own dedicated server or VPS with TM²/SM servers running on it.

I'm not using the most up-to-date stuff but if it can help you a bit, that's nice.
Until then, i was only using a failsafe script (put in a cron task executed every 10 minutes) to check if a server was down and if yes, to restart it:

Code: Select all

#! /bin/bash

logfile="/var/log/trackmania2/failsafe.log"

function checkIt()
{
 ps -fu tm2launcher | grep "$1" > /dev/null

 if [ $? != 0 ]
 then
   echo "[ERROR]$(date '+%Y-%m-%d %H:%M:%S'): TM2 $1 server not found, restarting it." >> $logfile;
   if [ $1 == 'stadium' ]; then
     /etc/init.d/trackmania2 restart >> $logfile
   elif [ $1 == '\.rpg\.' ]; then
     /etc/init.d/trackmania2_rpg restart >> $logfile
   elif [ $1 == 'rpgbeg' ]; then
     /etc/init.d/trackmania2_rpg restart beg >> $logfile
   elif [ $1 == 'rpgint' ]; then
     /etc/init.d/trackmania2_rpg restart int >> $logfile
   elif [ $1 == 'rpgadv' ]; then
     /etc/init.d/trackmania2_rpg restart adv >> $logfile
   elif [ $1 == 'rpgexp' ]; then
     /etc/init.d/trackmania2_rpg restart exp >> $logfile
   elif [ $1 == 'rpgprv' ]; then
     /etc/init.d/trackmania2_rpg restart prv >> $logfile
#   elif [ $1 == 'ccpall' ]; then
#     /etc/init.d/trackmania2_ccp restart all >> $logfile
#   elif [ $1 == 'ccpesl' ]; then
#     /etc/init.d/trackmania2_ccp restart esl >> $logfile
#   elif [ $1 == 'ccpalp' ]; then
#     /etc/init.d/trackmania2_ccp restart alp >> $logfile
   fi;

 else
   echo "[OK]$(date '+%Y-%m-%d %H:%M:%S'): TM2 $1 server still running." >> $logfile;
 fi;
}

checkIt "stadium";
checkIt "\.rpg\.";
checkIt "rpgbeg";
checkIt "rpgint";
checkIt "rpgadv";
checkIt "rpgexp";
checkIt "rpgprv";
#checkIt "ccpall";
#checkIt "ccpesl";
#checkIt "ccpalp";

I'm now using XML-RPC to do it a bit more "properly". Failsafe cron task is still running in case of an abnormal crash but now i'm checking if the server is running since 20+ days and if there's any player(s) on it. If it runs since 20+ days and no player is on it, a restart is done. This is running as a cron task every few hours. Depending on how popular your servers are, you might want to have it runs more frequently.
Here's the script:

Code: Select all

#! /bin/bash

logfile="/var/log/trackmania2/servercheck.log"

function checkIt()
{
 serverdata=$(php /home/tm2launcher/Scripts/RPCUtils/sendcmd.php "$1" GetNetworkStats)
 uptime=$(echo "$serverdata" | jq '.Uptime')
 uptimedays=$(($uptime/86400))
 uptimelimit=1728000
 nbplayers=$(echo "$serverdata" | jq '.PlayerNetInfos | length')

 if [ "$uptime" -ge $uptimelimit ]
 then
   if [ "$nbplayers" -eq 0 ]
   then
     echo "[RESTART]$(date '+%Y-%m-%d %H:%M:%S'): TM2 $1 server started 20+ days ago ($uptimedays). $nbplayers player on it, restarting it." >> $logfile;
     if [ $1 == 'stadium' ]; then
       /etc/init.d/trackmania2 restart >> $logfile
     elif [ $1 == 'rpg' ]; then
       /etc/init.d/trackmania2_rpg restart >> $logfile
     elif [ $1 == 'rpgbeg' ]; then
       /etc/init.d/trackmania2_rpg restart beg >> $logfile
     elif [ $1 == 'rpgint' ]; then
       /etc/init.d/trackmania2_rpg restart int >> $logfile
     elif [ $1 == 'rpgadv' ]; then
       /etc/init.d/trackmania2_rpg restart adv >> $logfile
     elif [ $1 == 'rpgexp' ]; then
       /etc/init.d/trackmania2_rpg restart exp >> $logfile
     elif [ $1 == 'rpgprv' ]; then
       /etc/init.d/trackmania2_rpg restart prv >> $logfile
#     elif [ $1 == 'ccpall' ]; then
#       /etc/init.d/trackmania2_ccp restart all >> $logfile
#     elif [ $1 == 'ccpesl' ]; then
#       /etc/init.d/trackmania2_ccp restart esl >> $logfile
#     elif [ $1 == 'ccpalp' ]; then
#       /etc/init.d/trackmania2_ccp restart alp >> $logfile
     fi;
  else
    echo "[PENDING]$(date '+%Y-%m-%d %H:%M:%S'): TM2 $1 server started 20+ days ago ($uptimedays). $nbplayers player(s) on it, restart delayed." >> $logfile;
  fi;
 else
   echo "[OK]$(date '+%Y-%m-%d %H:%M:%S'): TM2 $1 server is running for less than 20 days ($uptimedays)." >> $logfile;
 fi;
}

checkIt "stadium";
checkIt "rpg";
checkIt "rpgbeg";
checkIt "rpgint";
checkIt "rpgadv";
checkIt "rpgexp";
checkIt "rpgprv";
#checkIt "ccpall";
#checkIt "ccpesl";
#checkIt "ccpalp";

It's still using the same init.d script i'll put at the end of this post but there's a bit more stuff. First i'm using RPCUtilities (first link in the list), more specifically we're using the sendcmd.php script to send a XML-RPC method that'll give us the info we need. And then i'm using jq, a command-line JSON processor that's helping us parsing the data.
I modified sendcmd.php a bit since i've several servers, it's not just a single port for instance. So i just basically used the 2nd argument as the server and the rest didn't change. Also changed what the showIt() function returns. I mentioned jq, so yes, we're now returning json data. And about what we're doing with the server argument, we're just using a switch ($currentServer) in rpcconfig.inc.php file to set the proper variables needed (port, ...) for each server.

Here's my main init.d script. Note that as i said in the beginning of this thread, it's a bit old. Nowadays, you probably want to use "service". Also here it's for the RPG servers i've. You may want to strip some stuff like the SRV_SUFFIX. Also using PyPlanet as my controller here.

Code: Select all

### BEGIN INIT INFO
# Provides:          TM2 RPG Server
# Required-Start:    $remote_fs $syslog
# Required-Stop:     $remote_fs $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start TM2 daemon at boot time
# Description:       Enable service provided by daemon.
### END INIT INFO
    #!/bin/bash

    TM_DIR="/home/yourdirectory"
    TM_USER="youruser"
    SRV_SUFFIX=${2:-}
    RUN_SCRIPT="manage.py"

    start_tm() {
    echo "Lancement du server RPG${SRV_SUFFIX} TrackMania"
    su -l $TM_USER -c "cd $TM_DIR; $TM_DIR/ManiaPlanetServer /dedicated_cfg=dedicated_cfg.rpg${SRV_SUFFIX}.txt /game_settings=MatchSettings/RPG${SRV_SUFFIX}DefaultMaps.txt; pyenv activate pyplaenv; $TM_DIR/rpg${SRV_SUFFIX}_server/$RUN_SCRIPT start --detach --pid-file=$TM_DIR/rpg${SRV_SUFFIX}_server/pyplanet.pid" > /dev/null &
    }

    stop_tm() {
    echo "Arrêt du serveur RPG${SRV_SUFFIX} TrackMania"
    for pid in $(ps -fu "$TM_USER" | grep --regexp "[\._]rpg${SRV_SUFFIX}[\._]" | awk '{print $2}'); do kill -9 $pid; done > /dev/null
    }

    start_pypla() {
    echo "Lancement du contrôleur PyPlanet"
    su -l $TM_USER -c "cd $TM_DIR; pyenv activate pyplaenv; $TM_DIR/rpg${SRV_SUFFIX}_server/$RUN_SCRIPT start --detach --pid-file=$TM_DIR/rpg${SRV_SUFFIX}_server/pyplanet.pid" > /dev/null &
    }

    stop_pypla() {
    echo "Arrêt du contrôleur PyPlanet"
    for pid in $(ps -fu "$TM_USER" | grep -E "python.*[\/]rpg${SRV_SUFFIX}_" | awk '{print $2}'); do kill -9 $pid; done > /dev/null
    }

    case "$1" in
    start)
    start_tm
    ;;
    stop)
    stop_tm
    ;;
    restart)
    stop_tm
    sleep 2
    start_tm
    ;;
    pystart)
    start_pypla
    ;;
    pystop)
    stop_pypla
    ;;
    pyrestart)
    stop_pypla
    sleep 2
    start_pypla
    ;;
    *)
    echo "Usage du script : $0 {start|stop|restart|pystart|pystop|pyrestart}"
    ;;
    esac

Hope it can help anyone. Feel free to ask any question, or if you have any feedback, i'm nowhere near an expert.

[Miss]

Nice one!