Configure your own Syslog Server

Rsyslog is a rocket-fast system for log processing and is commonly used for any kind of system logging. For more informations about rsyslog, visit http://www.rsyslog.com. We use a Ubuntu server 14.04 LTS distribution to show you how to configure your own syslog server to receive your CDN logs in real time.

Syslog Server Installation

Update the packages list and install the latest version of rsyslog.

  1. apt-get update
  2. apt-get install rsyslog

Syslog Server Configuration

Configure rsyslog to receive UDP logs and define a filter where you want to store the logs.

  1. Open the rsyslog conf file and add the following lines
    vi /etc/rsyslog.conf
    
    # provides UDP syslog reception
    module(load="imudp")
  2. Create and open your custom config file.
    vi /etc/rsyslog.d/00-custom.conf
    # Templates
    template(name="ReceiveFormat" type="string" string="%msg:39:$%\n")
    
    # UDP ruleset mapping
    input(type="imudp" port="514" ruleset="customRuleset")
    
    # Custom ruleset
    ruleset(name="customRuleset") {
        if ($msg contains '366c3df6-93dd-4ec0-a218-aec9d191c59e') then {
            /var/log/cdn.log;ReceiveFormat
            stop
        }
    }
    

    Replace 366c3df6-93dd-4ec0-a218-aec9d191c59e with your own custom token. Your token values must be separated by four dashes “-” and must not exceed 45 characters. A few other valid token examples include:

    1a-2b-3c-4d-5e
    your-sec-token-key-101
    1abc-efg2-3hig-4lmn-5opq
    

    Use the following regex expression with regex101 to validate the token value you define:

    ^[a-z0-9]+-[a-z0-9]+-[a-z0-9]+-[a-z0-9]+-[a-z0-9]+$

    regex token validation

    You may use any letters from a-z and numbers from 0-9 when creating your token.
  3. Adjust your firewall rules. Use the previously defined UDP port and syslog sender IP which are found in the KeyCDN dashboard.
  4. Restart the rsyslog process
    service rsyslog restart
  5. Configure your syslog server within the KeyCDN Dashboard: Account Settings -> General -> Real-time Log Forwarding (syslog) syslog settingsThe log forwarding starts within 5 minutes after you save the configuration.
  6. Verify if you are receiving the logs
    tail -f /var/log/cdn.log

Troubleshooting Commands

  • service rsyslog status Verify that rsyslog is running.
    # service rsyslog status
    rsyslog start/running, process 26527
  • netstat -na | grep ":<defined port>" Is rsyslog listening on the right port?
    # netstat -na | grep :514
    udp        0      0 0.0.0.0:514             0.0.0.0:*
  • tcpdump port <defined port> Are you receiving any packet on the defined port?
    # tcpdump port 514
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
    11:20:53.066938 IP keycdn-syslog.37960 > your-server.syslog: [|syslog]
    ^C
    1 packet captured
    1 packet received by filter
    0 packets dropped by kernel
  • tail -f /path/to/your/logfile Check if you get new log entries.
    # tail -f /var/log/cdn.log 
     1421338853.058|defr|115.81.56.12|200|439|1|6976|cdn-1.kxcdn.com|HIT|"HEAD /lorem.jpg HTTP/1.1"|[15/Jan/2015:17:20:53 +0100]|"-"|"curl/7.30.0"|http|CH|Switzerland|Winterthur|25|47.5000|8.7251|"AS6830 Liberty Global Operations B.V."^C