For the vast majority of home users the GUI in the OS X System Preferences application for managing your Firewall is all you’ll ever need, however, once you start wanting to have different rules for different hosts you have no choice but to either fork-out cash for something like Brickhouse or to get your hands dirty and write your own firewall rules with IPFW, the firewall that is part of OS X.

In this article I’m going to walk you through the right way to set up your mac to use custom IPFW rules, I am NOT going to give a tutorial on IPFW, I will be assuming that you are familiar with it and basic firewalling concepts. If you don’t know what you are doing with IPFW then DO NOT SET UP YOUR OWN CUSTOM RULES, you are playing with fire here people, if you mess them up you can actually stop your computer from working or leave it open to attack, neither are good!

The first step in setting up a custom firewall is to create a Startup item to manage it as a service. There are many different scripts out there to do this but most do this in a very poor way that does not use the standard files and is not controllable in the standard BSD way. The script I give here DOES do things the Right-Way_(tm).

As root you will need to make a folder /Library/StartupItems/Firewall, in this folder you will have to create two files. The first is the script to control your firewall, the second is the list of parameters the OS needs about your firewall service.

Firstly, the script to control your firewall, this file MUST be named identically to the startup item so in this case Firewall. The script should contain the following:

#!/bin/sh

##
# Firewall
##

. /etc/rc.common

StartService ()
{
  if [ "${FIREWALL:=-NO-}" = "-YES-" ]
  then
    ConsoleMessage "Starting Firewall"
    sh /etc/rc.firewall > /dev/null
  fi
}

StopService ()
{
  ConsoleMessage "Stopping Firewall"
  /sbin/ipfw -f -q flush
}

RestartService ()
{
  StopService
  StartService
}

RunService "$1"

This script has the advantage that it allows your firewall to be started, stopped and restarted like any OS service and it also allows the firewall to be enabled and disabled from /etc/hostconfig (without this you would have to delete the startup item to disable it). As you can see this file does not actually contain our firewall rules, those should be defined in /etc/rc.firewall.

The second file MUST be called StartupParameters.plist and should contain:

{
  Description     = "Firewall";
  Provides        = ("Firewall");
  Requires        = ("NetworkExtensions","Resolver");
  OrderPreference = "Late";
  Messages =
  {
    start = "Starting firewall";
    stop  = "Stopping firewall";
  };
}

The next step is to create your custom firewall rules and store them in /etc/rc.firewall. As I said I am not going to go into a discussion on creating firewall rules in this article, I am assuming that you know what you are doing when it comes to creating firewalls with IPFW! Hence I am going to skip straight on to enabling the service in /etc/hostconfig. This is exceptionally simple, all you have to do is add the line:

FIREWALL=-YES-

to the end of the file.

That’s it, you now have a custom IPFW firewall running correctly on OS X. If at any stage you want to temporarily stop using your custom firewall all you have to do is stop the service with /Library/StartupItems/Firewall/Firewall stop and then set the entry for your firewall in /etc/hostconfig to FIREWALL=-NO- to disable it.

I may well do a follow-up article to this one discussing firewall rules for OS X and/or logging but I’ll make no promises!