Configure your own Syslog Server

Rsyslog is a rocket-fast system for log processing and is commonly used for any kind of system logging. For more informations about rsyslog, visit http://www.rsyslog.com. We use a Ubuntu server 14.04 LTS distribution to show you how to configure your own syslog server to receive your CDN logs in real time.

Syslog Server Installation

Update the packages list and install the latest version of rsyslog.

  1. apt-get update
  2. apt-get install rsyslog

Syslog Server Configuration

Configure rsyslog to receive UDP logs and define a filter where you want to store the logs.

  1. Open the rsyslog conf file and uncomment the following lines
    vi /etc/rsyslog.conf
    
    $ModLoad imudp
    $UDPServerRun 514

    You could change the default port 514 to something else if you want.

  2. Create and open your custom config file
    vi /etc/rsyslog.d/10-custom.conf
    $template cdnlogs,"%msg%\n"
    :msg, contains, "|uid<userId>|" /path/to/your/logfile;cdnlogs
    & ~
    

    Replace <userId> with your userId (decimal): KeyCDN Dashboard -> Account Settings -> Account Details -> User ID

  3. Adjust your firewall rules. Use the previously defined UDP port and syslog sender IP which are found in the KeyCDN dashboard.
  4. Restart the rsyslog process
    service rsyslog restart
  5. Configure your syslog server: KeyCDN Dashboard -> Account Settings -> Real-time Log Forwarding (syslog)
    KeyCDN syslog settings The log forwarding start within 5 minutes after you saved the configuration.
  6. Verify if you are receiving the logs
    tail -f /path/to/your/logfile

Troubleshooting Commands

  • service rsyslog status. Verify that rsyslog is running.
    # service rsyslog status
    rsyslog start/running, process 26527
  • netstat -na | grep “:<defined port>”Is rsyslog listening on the right port?
    # netstat -na | grep :514
    udp        0      0 0.0.0.0:514             0.0.0.0:*
  • tcpdump port <defined port>Are you receiving any packet on the defined port?
    # tcpdump port 514
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
    11:20:53.066938 IP keycdn-syslog.37960 > your-server.syslog: [|syslog]
    ^C
    1 packet captured
    1 packet received by filter
    0 packets dropped by kernel
  • tail -f /path/to/your/logfileCheck if you get new log entries.
    # tail -f /var/log/cdnlog 
     1421338853.058|defr|115.81.56.12|200|439|1|6976|cdn-1.kxcdn.com|HIT|"HEAD /lorem.jpg HTTP/1.1"|[15/Jan/2015:17:20:53 +0100]|"-"|"curl/7.30.0"|http|CH|Switzerland|Winterthur|25|47.5000|8.7251|"AS6830 Liberty Global Operations B.V."^C