Willing and not afraid to challenge the status quo.

pfSense 2 log visualization with glTail on OSX Lion

written by Ruan
at 5:58 pm
on December 5, 2011
in Development, Open Source, Programming, Security, Technology
no comments

pfSense has changed considerably since the first generation of the fantastic FreeBSD based firewall platform and with that, 3rd party utilities such as glTail require updated configs and log parsers to restore the ability to visualize pfSense 2 firewall logs since the logging system in pfSense 2 has been revised considerably.

glTail rendering of logs generated by pfSense 2 on OSX

Necessary changes:
A commit to the glTail repo by pfSense developer JimP for an updated logging parser for pfSense 2 (https://github.com/Fudge/gltail/pull/14) included a comment about an additional and mandatory updated log output command used by pfSense 2 to be added to config.yaml called /usr/local/bin/filterpaser.php

In trying out the commit for the updated pfsense2.rb logging processor, glTail output was broken since the updated pfSense 2 logging parser had been revised considerably with a new block naming convention and as well as dropping blocks that were previously used in the pfSense 1 configuration. This resulted a config.yaml revision to use the new block naming convention and depreciated block names.

Additionally updates were needed to the hostwithport value processing for IPv4 traffic in the pfsense2.rb logging processor to parse out the host and port combinations from the old xxx.xxx.xxx.xxx.zzz format from pfSense 1 to the xxx.xxx.xxx.xxx:zzz format now used by pfSense 2. The committed parser did not yet include this necessary change.

Installation of glTail on OSX Lion:
Provided here are some simple steps to take to get glTail installed on OSX. This as been tested on Lion only.

wget https://github.com/Fudge/gltail/zipball/master
unzip Fudge-gltail-9d2b843.zip 
cd Fudge-gltail-9d2b843
sudo gem install net-ssh ruby-opengl file-tail net-ssh-gateway chipmunk -r
mv config.yaml config.yaml.old

Configuration of glTail for pfSense 2:
The following updated config and logging parser include all the fixes necessary to run glTail against pfSense 2 generated log output from /usr/local/bin/filterpaser.php.

Download and save the following config.yaml to the root of your glTail install and update with the host value with the IP of your pfSense 2 server.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
servers:
    # Example 1: Connect directly to a pfSense router
    pfsense1:
        host: 192.168.1.1
        user: root
        password: 
        command: /usr/sbin/clog -f /var/log/filter.log | /usr/local/bin/filterparser.php
        files: /var/log/filter.log
        parser: pfsense2
        color: white
 
    # Example 2: Logs forwarded to a syslog host 
    # pfsense2:
    #     host: 192.168.1.2
    #     user: logview
    #     password: logviewpassword
    #     command: /usr/bin/tail -f -n0
    #     # Adjust this based on where you have syslog direct the output
    #     files: /var/log/hosts/pfsense.log
    #     parser: pfsense
    #     color: 0.2, 1.0, 0.2, 1.0
 
config:
    dimensions: 1024x700
    min_blob_size: 0.004
    max_blob_size: 0.02
    highlight_color: orange
    bounce: true
    left_column:
        size: 45
        alignment: -0.99
        blocks:
            action:
                order: 1
                size: 5
                color: red
            ipprotocol:
                order: 2
                size: 5
                color: magenta
            int:
                order: 3
                size: 5
            sourcedestination:
                order: 4
                size: 10
                color: pink
 
    right_column:
        size: 45
        alignment: 0.99
        blocks:
            destinationhost:
                order: 1
                size: 15
            destinationport:
                order: 2
                size: 15
                color: cyan
            sourcehost:
                order: 3
                size: 15
            sourceport:
                order: 4
                size: 15
                color: blue
resolver:
    reverse_ip_lookups: true
    reverse_timeout: 0.5

Download and save pfsense2.rb to lib/gl_tail/parsers

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
# gl_tail.rb - OpenGL visualization of your server traffic
# Copyright 2007 Erlend Simonsen ([email protected])
#
# Licensed under the GNU General Public License v2 (see LICENSE)
#
 
# Parser for pfSense PF Logs, specifically those from pfSense (2.0)
# Jim Pingle ([email protected])
 
# Available Blocks
#action: block|pass
#rule: Rule number matched
#ipprotocol: carp|icmp|tcp|udp|ah|igmp|esp|gre you get the idea..
#int: This will be the actual interface (fxp0, vlan2, em1, etc) as the 'friendly' name is not put in the logs.
#sourcehost: source host/IP
#sourceport: source port
#destinationhost: destination host/IP
#destinationport: destination port
#sourcedestination:  source host and port > destination host and port
 
# Use with command:  /usr/sbin/clog -f /var/log/filter.log | /usr/local/bin/filterparser.php
 
class PFSense2Parser < Parser
  require 'date'
 
  def getipandport(hostwithport)
 
    # Test for IPv6
    if (hostwithport.count(':') > 2)
      if (hostwithport.count('.') == 1)
        thisport = hostwithport.split('.')[1].to_s
        thishost = hostwithport.split('.')[0].to_s
      else
        thishost = hostwithport
        thisport = "none"
      end
    else
      # IPv4
      if (hostwithport.count('.') == 3 && hostwithport.count(':') == 1)
        thisport = hostwithport.split(':')[-1,1].to_s
        thishost = hostwithport.split(':')[0,1].to_s
      else
        thishost = hostwithport
        thisport = "none"
      end
    end
 
    if thisport.include?(':')
      thisport = thisport.split(':')[0]
    end
    if thisport.include?(' ')
      thisport = thisport.split(' ')[0]
    end
 
    return [thishost, thisport]
  end
 
  def getport(thisport)
    if thisport == "none"
      return ""
    else
      return ":" + thisport.to_s
    end
  end
 
  def parse( line )
    lmonth, lday, ltod, action, int, ipprotocol, src, dst = line.split(' ')
    ltime = [ lmonth, lday, ltod ].join(' ')
 
    # Assume the server is in the same time zone as the viewing client.
    timewithoffset = ltime.to_s + DateTime.now().zone()
 
    # Alternately, just set it this way to assume UTC/GMT
    #timewithoffset = ltime.to_s
 
    hours,minutes,seconds,frac = Date.day_fraction_to_time(DateTime.now() - DateTime.parse(timewithoffset))
 
    # When connecting directly, there is no way to only view the end of the log. The clog program to view
    # circular logs will dump the entire log to the parser, then will tail it showing new messages.
    # Therefore, we can run a simple time check and only view entries from the last 5 minutes, or the
    # "future". On some systems, I have seen the clock show negative (-1hr 59mins) instead of 0, so we
    # can allow "future" messages just to be safe.
    if ((hours == 0) and (minutes < 5)) or (hours < 0)
      # Debug
      # printf("Adding entry from %s hours, %s minutes ago\n", hours.to_s, minutes.to_s)
 
      sourcehost, sourceport = getipandport(src)
 
      destinationhost, destinationport = getipandport(dst)
 
      add_activity(:block => 'action',  :name => action.to_s)
      add_activity(:block => 'int',     :name => int.to_s)
      add_activity(:block => 'ipprotocol',   :name => ipprotocol.to_s)
      add_activity(:block => 'sourcehost', :name => sourcehost.to_s)
      if sourceport != "none"
        add_activity(:block => 'sourceport', :name => sourceport.to_s)
      end
      add_activity(:block => 'destinationhost', :name => destinationhost.to_s, :type => 5)
      if destinationport != "none"
        add_activity(:block => 'destinationport', :name => destinationport.to_s, :type => 5)
      end
      add_activity(:block => 'sourcedestination',  :name => sourcehost.to_s + getport(sourceport) + " > " + destinationhost.to_s + getport(destinationport) + " (" + ipprotocol.to_s + ")")
    else
      # Debug
      # printf("Not adding entry from %s hours, %s minutes ago\n", hours.to_s, minutes.to_s)
    end
  end
end

Executing:
Executing glTail is done from the root of the glTail installation folder location.

./bin/gl_tail config.yaml

Notes:
This has not been tested against IPv6 traffic yet.


SANS Report: 60% Of All Attacks Hit Web Applications, Most in the U.S.

written by Ruan
at 9:41 am
on September 15, 2009
in Development, Programming, Security, Technology
no comments

Most organizations are focusing their patching efforts and vulnerability scanning on the operating system — but 60 percent of the total number of attacks occur on Web applications, and many attacks are aimed at third-party applications such as Microsoft Office, and Adobe Flash and other tools, according to actual attack data gathered for the report. Meanwhile, enterprises are taking twice as long to patch their applications than to patch their operating systems, the report says.

More at darkREADING


OpenBSD 4.5 Released

written by Ruan
at 10:56 am
on May 1, 2009
in Data Centers, Development, Open Source, Programming, Security, Technology
no comments

OpenBSD 4.5 has been released today. This release includes OpenSSH 5.2 as well as various tweaks, bugfixes, and enhancements. New and extended platforms include sparc64, and added device drivers. See the announcement page for a full list


New rogue DHCP server malware

written by Ruan
at 10:00 pm
on March 16, 2009
in Data Centers, Development, Programming, Security, Telecommunications
no comments

A bold new type of malware has been identified.  Its attack vector is based on hijacking the DNS settings for devices on a local area network. Any device regardless of operating system that depends on an internal or external name server can be affected.

The trojan configures and runs a rogue DHCP daemon on the infected host. Other devices on the same LAN are misled into using name servers settings provided by the trojan DHCP daemon for DNS lookups instead of using the origional configured name servers.

Devices on the network are then sent to fraudulent websites that can be more difficult to identify as imposters since the DNS lookups appear correct.

This is a more advanced attack of a well known vector of attacking a systems hosts file, but by being system agnostic and using the familiar DNS protocol, it is much more effective.

More details can be found at SANS


Valve Announcing The Upcoming Roll-Out Of Steam Cloud

written by Ruan
at 11:53 pm
on November 4, 2008
in Business, Development, Games
no comments

Steam Cloud support, which was announced last May, will be released in tangent with Valve’s co-op zombie shooter Left 4 Dead demo later this week. Steam Cloud is expected to be compatible with all other Valve Steam releases (including the back catalog) after receiving support updates. Valve has also been providing other developers with the tools to make their own titles compatible free of charge.

Steam Cloud support is seamless, meaning any changes to a users game configuration such as graphics options and key settings, and best of all, save games, will propagate to the Cloud automatically. Upon logging into your Steam account from another PC, these settings will be synchronized down from the Cloud and automatically loaded by the game. Any changes to the configuration on this second machine are then synced to the Cloud for future sessions.

The simplicity from the perspective of the user of this new capability is quite brilliant.


NSA Open Sources Tokeneer Research Project

written by Ruan
at 3:25 pm
on October 6, 2008
in Development, Miltary, Open Source
no comments

The Tokeneer research project has been released to the open source community by the US National Security Agency. The main goal of this project was to show how highly secure software can be developed cost-effectively.

“Tokeneer has been written in SPARK Ada, a high level programming language designed for high-assurance applications. Originally a subset of the Ada language, it is designed in such a way that all SPARK programs are legal Ada programs. Ada is the natural choice for mission-critical, high-integrity systems due to its combination of flexibility, reliability and ease of use, and SPARK further adds a static verification toolset that combines depth, soundness, efficiency and formal guarantees.”

Via Slashdot


Apple Macs now account for 8% of computer market.

written by Ruan
at 12:00 pm
on October 3, 2008
in Business, Design, Development
no comments

In a survey conducted by Net Applications on 40,000 sites, the results show that more than 8% of computers accessing the web were using Apple Macs. These numbers are only indicative of Macs that are used to access the internet. 8% may seem like a small number, but consider that the current estimated internet usage statistics pin a number around 1463632361 users.  That places Macs used to access the internet around 117090588 machines.

These numbers only highlight the importance of testing Apple Mac as a platform against your external facing websites.


Congress Plugs Open Source For US Military

written by Ruan
at 10:51 am
on September 26, 2008
in Development, Miltary, Open Source, Security, Technology
no comments

In an important milestone for open source, a draft defense authorizing act in Congress includes wording plugging open source software. Both cost and software security appears to be considerations.

“It’s rare to see a concept as technical as open-source software in a federal funding bill. But the House’s proposed National Defense Authorization Act for Fiscal Year 2009 (H.R. 5658) includes language that calls for military services to consider open-source software when procuring manned or unmanned aerial vehicles.”


Official Support For PHP 4 Ends

written by Ruan
at 5:35 pm
on August 11, 2008
in Development, Programming
1 comment

Via Slashdot From ComputerWorld:

“For a technology that has been in stable release since May 22, 2000, PHP 4 has finally reached the end of its official life. With the release of PHP 4.4.9, official support has ended and the final security patch for the platform issued. …With eight years of legacy code out there, it is likely that there are going to be a fairly large number of systems that will not migrate to PHP 5 in the near future, and a reasonable proportion of those that will not make the migration at all. For those who are not able to migrate their systems to the new version of PHP, noted PHP security expert Stefan Esser will continue to provide third party security patching for the PHP 4 line through his Suhosin product.”


Setting up an AMP stack on FreeBSD

written by Ruan
at 8:24 pm
on August 2, 2008
in Databases, Design, Development, Programming, Security, Technology
no comments

Linux.com has a detailed step by step installation procedure for installing Apache, MySQL and PHP on FreeBSD using the ports collection.  Some extra steps are taken to secure the installation and you end up with a nice multifunctional FreeBSD+AMP setup with very little effort.


 
« Previous Entries  

about this

Ruan is a resolute technophile that is currently devoted to the professional practice of Information Technology Management. In his free time Ruan pursues various interests including the study of Information Security practices and the exploration of visual culture through contemporary photography and communication design.


fineprint
entire contents © 2012 Ruan Müller