The Author – Bart Busschots
Creative Commons
Unless otherwise stated, the content on this site is licensed under Creative Commons Attribution, Noncommercial, No Derivative Works 3.0 License
Jan
27
Some Simple Little Linux Commands You Can Never Remember When You Need Them
Filed Under Computers & Tech, System Administration on January 27, 2006 | Leave a Comment
I’m sure you know the feeling, you’re on a server, some form of excrement has just hit some kind of spinning thing and know that netstat will do what you want if only you could remember the bloody set of options you needed. Well, from now on I’m adding all those commands to this Blog entry so next time that happens I’ll know here to look!
This page will be continually updated but I’m gonna start with a few of those commands I’ve needed in the last few days that are currently stuck to my display in little yellow PostIt Notes.
What Processes are Listening on What TCP Ports
When you need to easily see what a server is serving in a hurry use:
netstat -lntp
Creating and Extracting tar.gz Files Just Using tar
I used to always create and extract tar.gz files by using both tar and gzip and sticking them together with a pipe (I’m sure that nasty Sun E450 was in some way to blame for me getting into that habit). This command was always long and I never remembered the syntax. So here’s the easy way to create a tar.gz archive:
tar -pczf name_of_your_archive.tar.gz /path/to/directory
(You should leave out the p if you don’t want to preserve the file permissions). And here is how to extract one:
tar -xzf file.tar.gz
Dec
8
Breezy Badger release of KUbuntu continues to dissapoint
Filed Under Computers & Tech, System Administration on December 8, 2005 | Leave a Comment
Following on from my last article about Breezy I’m afraid things have not gotten any better. KUbuntu is just broke! At first I thought the mac style control panel was really cool until I discovered that I can’t get into administrator mode on any of the dialogs. I click the button, enter the password, the border goes red, the panel goes blank, stays blank for some timeout period and then brings me back to the non-admin display. I can click that button all I want but I can never edit any settings that you need to be root for. Yet again not acceptable from a Linux that is supposed to be for "human beings".
Dec
7
SSH via a Socks proxy on OS X with connect.c
Filed Under Computers & Tech, System Administration on December 7, 2005 | 15 Comments
NOTE: tested on OS X 10.7 Lion, works fine!
These instructions are for setting up a mac to use connect.c to get SSH through a SOCKS proxy. If you are not using OS X this may still be of some use to you because connect.c will compile on Windows and *nix as well. If you’re in the NUIM oncampus accommodation and are having problems SSHing this could be the answer to your problems!
The first step is to get a copy of connect.c and compile it. The website contains instructions for doing this on other platforms but for the mac use:
gcc connect.c -o connect -lresolv
This will spool out a ream of warnings but don’t worry about that.
Then you have to copy this to a folder in the path and set up the correct permissions:
sudo cp connect /usr/bin sudo chmod 555 /usr/bin/connect sudo chown root:wheel /usr/bin/connect
At this stage connect.c is installed, you now need to tell SSH to use this proxy for any servers you want to connect to that is outside the campus.
To do this you need to add lines of this form to ~/.ssh/config:
Host xxx.yyy.com ProxyCommand connect -a none -S socks.yyy.com %h %p
The example above is for connection to xxx.yyy.com, you’ll need pairs like this for each host you want to connect to. You should separate the pairs with a bank line.
That’s it, you can now ssh as normal and ssh will use the SOCKS proxy.
ssh [email protected]
Dec
7
IPFW Firewall Script (Suitable for OS X)
Filed Under Computers & Tech, Security, System Administration on | Leave a Comment
I posted an article the other day about how to configure OS X to use a custom firewall script but explicitly didn’t discuss the actual rules, in this article I’m going to focus only on the firewall rules and assume that you are using an IPFW configuration that is working correctly on some OS that uses IPFW. Please note that I have tested these rules on OS X only but I can’t see why they wouldn’t work on any other IPFW enabled system.
So, without further ado, here is my /etc/rc.firewall (with all specific IP addresses blanked out of course!):
#!/bin/sh
#
# Declare variables
#
IPFW='/sbin/ipfw'
LOOPBACK_INTERFACE=lo0
PRI_DNS=XXX.XXX.XXX.XXX
SEC_DNS=XXX.XXX.XXX.XXX
#
# Flush the rules so we start with an empty pallet
#
$IPFW -f -q flush
#
# Do some house keeping
#
# allow through all genuinely local packets
$IPFW add 00001 allow ip from any to any via $LOOPBACK_INTERFACE
# deny all spoofed local packets
$IPFW add 00010 deny ip from 127.0.0.0/8 to any in
$IPFW add 00011 deny ip from any to 127.0.0.0/8 in
$IPFW add 00012 deny ip from 224.0.0.0/3 to any in
$IPFW add 00013 deny tcp from any to 224.0.0.0/3 in
# allow through established connections
$IPFW add 00020 allow ip from any to any established
# filter ICMP packets to only allow through some types
$IPFW add 00030 allow icmp from any to any icmptypes 0,3,4,8,11,12
$IPFW add 00031 deny icmp from any to any
# allow through DNS
$IPFW add 00040 allow udp from me to $PRI_DNS 53 keep-state
$IPFW add 00041 allow udp from me to $SEC_DNS 53 keep-state
# allow myself to make outgoing connections
$IPFW add 00050 allow tcp from me to any keep-state
$IPFW add 00051 allow udp from me to any keep-state
# allow through all DHCP packets
$IPFW add 00060 allow udp from any to any 67
$IPFW add 00061 allow udp from any to any 68
# deny UDP
$IPFW add 00070 deny udp from any to any
#
# Allow services
#
# ssh just from within my departments range
$IPFW add 00100 allow tcp from XXX.XXX.XXX.0:255.255.255.0 to me 22
#
# Allow in exceptions for me
#
# allows the FTP server to connect back to me for active FTP
$IPFW add 00200 allow tcp from XXX.XXX.XXX.XXX 20 to me
#
# end with deny all
#
$IPFW add 65534 deny ip from any to any
OK, so now I’m going to go through this bit by bit to explain why I’m doing things the way I am. There is logic behind each line and I’ve spent a lot of time and effort this week getting this right. I’ll also mention some pitfalls to look out for along the way.
I should also point out what I was trying to achieve with these rules. The first thing is that these rules allow me to go out on any port to anywhere I want, they are not designed to stop me doing stuff. Secondly, these rules are designed to expose as few as possible services on this machine and any services that are exposed should be exposed to as few hosts as possible. Finally, these rules are also designed to filter out a lot of the UDP rubbish floating round on the network I work on because of all the Windows boxes on it.
So, into the rules, the way I have it set up is that the only parts that you should need to edit are the top bit that defines the variables and the bottom bit that defines the services you want to open up, the bits in between that should not need to be edited as there are no specific IP addresses anywhere in them.
The first line is obviously a shebang line, if you don’t know what that is please stop reading now because you are not ready to set up a custom firewall script! So, the next bit is the definition of some variables:
IPFW='/sbin/ipfw'
LOOPBACK_INTERFACE=lo0
PRI_DNS=XXX.XXX.XXX.XXX
SEC_DNS=XXX.XXX.XXX.XXX
The first one is obviously defining the location of the IPFW binary and the second line is the definition of the loop-back interface. Be careful here, some scripts you see use lo* here and that is very dangerous, using it made my machine un-usable till I changed it back to lo0. The next two lines define the IP addresses for the DNS servers that the firewall should allow the machine to use.
The next thing you have to do is to flush all the rules that may be in the firewall already out so you know exactly what will be in there when your script ends, to do this we use the flush command but we must also use the -f flag to force it to do the flush without asking us if we are sure.
$IPFW -f -q flush
The next stage is to ensure that all loopback traffic can flow but that no one can spoof the localhost address on another interface, anyone doing so is definitely up to no good!
# allow through all genuinely local packets
$IPFW add 00001 allow ip from any to any via $LOOPBACK_INTERFACE
# deny all spoofed local packets
$IPFW add 00010 deny ip from 127.0.0.0/8 to any in
$IPFW add 00011 deny ip from any to 127.0.0.0/8 in
The next two lines are standard in all firewalls OS X’s firewall interface generates so I have added them to my rules too. They block certain types of multi-cast traffic.
$IPFW add 00012 deny ip from 224.0.0.0/3 to any in
$IPFW add 00013 deny tcp from any to 224.0.0.0/3 in
The next thing to do is to allow packets belonging to contections that have already been established to pass through the firewall without needing to be checked off all the rules again. This a great optimisation but is only any good if you have it close to the start of your rules.
$IPFW add 00020 allow ip from any to any established
At this point the next thing we are going to deal with is ICMP packets. Many people block all of these packets but I am not a fan of that. ICMP is there for a good reason and blocking it will probably make your system administrator grumpy! The way I filter ICMP I can ping people, people can ping me, I can traceroute to people and people can traceroute to me. I am however still blocking loads of other potential ICMP traffic that is just not needed (note that to get traceroute working you also need UDP to be allowed out).
$IPFW add 00030 allow icmp from any to any icmptypes 0,3,4,8,11,12
$IPFW add 00031 deny icmp from any to any
Next I allow through DNS to my specific DNS servers. These rules are technically not needed if you decide to allow yourself to send out traffic on all UDP ports using a state-full rule (see rule 00051) but I like to put these in explicitly because I sometimes like to just kill all UDP connections later in the rules.
$IPFW add 00040 allow udp from me to $PRI_DNS 53 keep-state
$IPFW add 00041 allow udp from me to $SEC_DNS 53 keep-state
The next thing I do is allow myself to make outgoing connections to any host on any TCP port and to send UDP to any port on any host. If you want to be able to use traceroute you have to allow UDP traffic out. Again, if you allow all UDP out using a state-full rule like the one below you don’t need to explicitly allow DNS so you can leave out the two lines above if you wish.
$IPFW add 00050 allow tcp from me to any keep-state
$IPFW add 00051 allow udp from me to any keep-state
One more thing to bear in mind is that if you do not include rule 00051 above to allow out and back all UDP ports then you would need to explicitly allow NTP if you wanted to use it.
The next thing to do is to make sure that you let all DHCP packets through. If you don’t use DHCP you can leave these two lines out.
$IPFW add 00060 allow udp from any to any 67
$IPFW add 00061 allow udp from any to any 68
At this point I like to deny all UDP traffic because there is just so much of it on our network but you can leave this line out if you wish, the last line will catch these packets too. The reason I put this rule here is for efficiency.
$IPFW add 00070 deny udp from any to any
Then we get to the second last part of the script where we define the services on our machine that we will allow be visible (to part of) the outside world. I like to tie things down very tightly so that I expose as few services as possible to as as few people as possible, hence, I only have two entries in this part of the script (and one is not technically a service). The first thing I do is allow SSH access to my machine but ONLY from within the subnet used by people within my department, not to anyone else on campus or beyond.
$IPFW add 00100 allow tcp from 149.157.4.0:255.255.255.0 to me 22
The second entry I have in here is needed to allow me to use active FTP to communicate with a web server I have to publish stuff to.
$IPFW add 00200 allow tcp from ccintranet.nuim.ie 20 to me
Finally, AND MOST IMPORTANLY, you must end with a rule to deny all packets not yet accepted. If you don’t your rule set will be utterly pointless and your machine will be pretty much completely open.
$IPFW add 65534 deny ip from any to any
And there you have it, my full set of Firewall rules for OS X. Now, please note that although I’ve been working with IPFW rules for many years both on OS X and FreeBSD I would still not consider myself an expert, hence, these rules come with absolutely no guarantees. I’m confident they are an excellent set of rules that combine the best bits of the other scripts I found on the web for OS X but you will need to make your own judgement call before using them.
My final bit of advice is to NEVER put a line onto your rules that you do not understand 100%, if you’re not sure what a rule does it should not be in your script! When you are done with your rules you should then check them by getting someone to nmap your machine or by using a free online service like ShieldsUp! from Gibson Research Corporation. Happy firewalling!
Dec
5
The RIGHT way to set up a Custom Firewall on OS X
Filed Under System Administration, Computers & Tech, Security on December 5, 2005 | Leave a Comment
For the vast majority of home users the GUI in the OS X System Preferences application for managing your Firewall is all you’ll ever need, however, once you start wanting to have different rules for different hosts you have no choice but to either fork-out cash for something like Brickhouse or to get your hands dirty and write your own firewall rules with IPFW, the firewall that is part of OS X.
In this article I’m going to walk you through the right way to set up your mac to use custom IPFW rules, I am NOT going to give a tutorial on IPFW, I will be assuming that you are familiar with it and basic firewalling concepts. If you don’t know what you are doing with IPFW then DO NOT SET UP YOUR OWN CUSTOM RULES, you are playing with fire here people, if you mess them up you can actually stop your computer from working or leave it open to attack, neither are good!
The first step in setting up a custom firewall is to create a Startup item to manage it as a service. There are many different scripts out there to do this but most do this in a very poor way that does not use the standard files and is not controllable in the standard BSD way. The script I give here DOES do things the Right-Way(tm).
As root you will need to make a folder /Library/StartupItems/Firewall, in this folder you will have to create two files. The first is the script to control your firewall, the second is the list of parameters the OS needs about your firewall service.
Firstly, the script to control your firewall, this file MUST be named identically to the startup item so in this case Firewall. The script should contain the following:
#!/bin/sh
##
# Firewall
##
. /etc/rc.common
StartService ()
{
if [ "${FIREWALL:=-NO-}" = "-YES-" ]
then
ConsoleMessage "Starting Firewall"
sh /etc/rc.firewall > /dev/null
fi
}
StopService ()
{
ConsoleMessage "Stopping Firewall"
/sbin/ipfw -f -q flush
}
RestartService ()
{
StopService
StartService
}
RunService "$1"
This script has the advantage that it allows your firewall to be started, stopped and restarted like any OS service and it also allows the firewall to be enabled and disabled from /etc/hostconfig (without this you would have to delete the startup item to disable it). As you can see this file does not actually contain our firewall rules, those should be defined in /etc/rc.firewall.
The second file MUST be called StartupParameters.plist and should contain:
{
Description = "Firewall";
Provides = ("Firewall");
Requires = ("NetworkExtensions","Resolver");
OrderPreference = "Late";
Messages =
{
start = "Starting firewall";
stop = "Stopping firewall";
};
}
The next step is to create your custom firewall rules and store them in /etc/rc.firewall. As I said I am not going to go into a discussion on creating firewall rules in this article, I am assuming that you know what you are doing when it comes to creating firewalls with IPFW! Hence I am going to skip straight on to enabling the service in /etc/hostconfig. This is exceptionally simple, all you have to do is add the line:
FIREWALL=-YES-
to the end of the file.
That’s it, you now have a custom IPFW firewall running correctly on OS X. If at any stage you want to temporarily stop using your custom firewall all you have to do is stop the service with /Library/StartupItems/Firewall/Firewall stop and then set the entry for your firewall in /etc/hostconfig to FIREWALL=-NO- to disable it.
I may well do a follow-up article to this one discussing firewall rules for OS X and/or logging but I’ll make no promises!
Nov
19
Synergy on OS X – the easy way
Filed Under Computers & Tech, System Administration on November 19, 2005 | 6 Comments
If you have multiple machines on your desk and you are fed up of having a mess of keyboards and mice all over the place then Synergy is just the thing you need. What makes Synergy even better is that it works cross-platform so you can share a single keyboard and mouse between Linux, OS X, any Unix and even crappy old Windows!
To find out more about installing synergy on non-OS X platforms or
about the practicalities of setting up a Synergy server checkout
Synergy’s home page: http://synergy2.sourceforge.net/
The synergy home page allows you to download binaries for OS X but that package literally gives you just the executable file for the server and client and nothing more. If you want Synergy to be easier to use and install on OS X don’t get it from the Synergy website, get Synergy KM instead.
SynergyKM gives you a nice Synergy icon on the Menu Bar to allow you to easily see and change Synergy’s state (see screen first shot below). SynergyKM also adds an extra panel to your System Preferences to allow you to configure Synergy both as a client and a server (see second screen shot).
You can get SynergyKM here: http://software.landryhetu.com/synergy/
Addendum – 24-11-05
For some reason when you are using Synergy to connect your Mac to another keyboard and mouse and you have a hot-corner set up to lock the screen it doesn’t work while Synergy is on. If you turn-off synergy it works fine. for the last week I’ve been turning off Synergy each time I wanted to lock my computer and that’s just not ideal at all so I did some Googling on the matter and found a good solution.
You can get a padlock icon to appear in your Menu Bar and when you click on that one of the options in the menu is "Lock Screen" (see first figure below). This will allow you to lock your screen even while using Synergy. Getting this padlock is a little counter intuative but here goes:
- Open the "Keychain Access" program in your Applications/Utilities folder
- Open the prefferences for this app (Keychain Access -> Preferences)
- I nthe "General" pane check the checkbox labled "Show Status in Menu Bar"
Simple as that!
Oct
4
The CORRECT way to install Sun Java on Debian/Ubuntu
Filed Under Computers & Tech, System Administration on October 4, 2005 | Leave a Comment
This little guide is just a greatly padded out version of instructions I got from Misha but since we have quite a few Ubuntu users and quite a few Java programmers in MiNDS> this should come in quite useful.
Basically this is the Debian way of installing the Sun JSDK so that it can be managed with dpkg and hence easily upgraded or removed at a later date.
Firstly, since Ubuntu IS a distribution of Debian really I will just refer to Debian in these instructions but the instructions also apply to Ubuntu and in fact it was on Ubuntu that I tested this. Also, these instructions assume that you are logged in as a user who has access to root via sudo.
The way this will work is that we will use a Debian package called java-package to turn the binary Linux installer we get from Sun (or IBM and others too) into a proper Debian package (.deb) and then we will use Debian’s package manager (dpkg) to install that Debian package.
The first step in the process is to install the java-package program that will allow us to create the .deb file. To do this simply type:
sudo apt-get install java-package
If you are asked for a password it is for sudo and you should enter your own login password.
The next step is to go to java.sun.com and download the Linux binary installer for the JSDK that you want to install. DO NOT DOWNLOAD THE LINUX RPM FILE!
The next step is to use java-package to turn this binary file into a Debian package, to do this move into the folder where you downloaded the binary file from java.sun.com.
NOTE, if you are doing this in the college on your machine for your 4th year project you will have to copy the binary file to /tmp and work from there because local root does not have access to your NFS mounted home directory.
For this step you will need fakeroot, if you don’t have it installed install it with apt-get like so:
sudo apt-get install fakeroot
Once you have fakeroot installed and you are in the right directory start the packaging with the command (replacing file_from_sun.bin with the actual name of the file you got from Sun):
fakeroot make-jpkg file_from_sun.bin
This will now appear to run the Sun installer but rather than installing it into your system it is extracting it to a temporary fake root file system and then creating the .deb file from the files in that fake file system. During this stage you will be asked to agree to the Java license agreement.
When the above command completes (will take a few minutes) a .deb file will have been created in your current folder. This is the file that we will now install with dpkg as follows (replacing name_of_deb_file.deb with the actual name of the .deb file created):
sudo dpkg -i name_of_deb_file.deb
And hey presto you are finished, when ever you want to get rid of that JDK all you have to do is (replacing name_of_deb_file_without_The_extension with the name of the generated .deb file but with the .deb extension left out, e.g. sun-j2sdk1.5):
dpkg --purge name_of_deb_file_without_The_extension
You can then install a new JSDK in the same was as described above or just leave your system Java free (heaven knows why you’d do something mad like that!)
Oct
3
The easiest way to turn a Mac into a Mail Server
Filed Under Computers & Tech, System Administration on October 3, 2005 | Leave a Comment
So, you have a mac server running OS X and your boss tells you it needs to act as a mail server too. The HARD way to do things is to compile your own postfix and then manually configure it. If you are a Unix whiz then this will of course be the kind of stuff you do in your sleep but if you’re just a pleb like the rest of us then you will be interested in this neat bit of software.
Using Post Fix Enabler it took me a whopping 30 seconds to get my machine running properly as a mail server (note things were made quite a bit quicker by the fact that I had to use another server as a smarthost). Yes, it costs just under $10 but it is money well spent!
Oct
3
Beating Bugzilla into submission on OS X
Filed Under System Administration, Computers & Tech on | Leave a Comment
I had been warned that Bugzilla was a pain to install but when I read the instructions on their webpage I thought it all looked pretty simple, turns out it WAS a pain to install!
The trouble was caused by two things:
- CPAN modules refusing to build on OS X
- Bugzilla making ridiculous assumptions that just don’t stand up to reality, namely:
- You will always want to use /usr/bin/perl
- sendmail will always be in /usr/lib/sendmail
For details on how I resolved these issues read on.
Firstly, getting Perl to behave itself. The simplest solution to this problem is not to use Perl that comes with OS X but to use DarwinPorts or Fink to get Perl in order. I used DarwinPorts because I have had bad experiences with Fink and really like DarwinPorts. Anyway, it worked like a charm, all the modules I needed were available via DarwinPorts and installed with no issues at all. One thing to note is that this does not patch up the Perl in /usr/bin but rather installs a whole new Perl in /opt/local/bin. One slight annoyance was that to get DBI::MySQL I also had to install a copy of MySQL from DarwinPorts. This is fine because you don’t actually have to USE it, just have it there. As it happens the guys at MySQL have an excellent binary distribution of the MySQL server these days so there is no need to get it from DarwinPorts. I also already had other databases installed and running on my MySQL install so I was in no humor to have a second MySQL server running just for Bugzilla! The one complication here is that the default location for the MySQL socket in the DarwinPorts Perl is the location the DarwinPorts MySQL server uses and not the one the default MySQL server uses. Since I am not using this MySQL I had to tell Bugzilla not to use the default socket but the one I was actually using. You do this by editing localconfig. In my case I had to set the following:
$db_sock = '/private/tmp/mysql.sock';
Now, you can probably imagine how using DarwinPorts’ Perl caused the first of the assumptions made by the Bugzilla installer to break down. The Perl that comes with OS X is at /usr/bin/perl and is used by loads of stuff on the system (so I can’t replace it with a symlink to the right Perl), the Perl from DarwinPorts is at /opt/local/bin/perl. Although I explicitly ran the installer for Bugzilla with this Perl in the hope that it would be smart and realise that it should use that Perl for everything, it still set all the shebang lines to #!/usr/bin/perl and hence although Bugzilla passed the installer’s tests with flying colours, it crashed in a heap when you tried to use it because it went running off to the wrong Perl!
I figured that if the shebang lines were not generated by setting them to the Perl running the installer then there MUST be a configuration option to allow you to specify what Perl should be used. I tried looking really hard but with no joy. I looked on the online documentation, no joy, I tried their mailing list, not even a reply so I took the bull by the horns and wrote my own Perl script to change all the shebang lines for me:
#!/opt/local/bin/perl
use strict;
print "\n\n";
# get the files in the directory
opendir(THISDIR, ".");
my @files = readdir(THISDIR);
closedir(THISDIR);
#loop through the files
foreach my $file (@files){
# skip files that are not cgi files
next unless $file =~ m/\.cgi$/;
print "Processing file: $file\n";
# get the contents
open(CGIFILE, "$file");
my @contents = <CGIFILE>;
close(CGIFILE);
# write it back out but with the correct shebang line
open(CGIFILE, ">$file");
if($contents[0] =~ m/^\#\!/){
print CGIFILE "#!/opt/local/bin/perl\n";
}else{
print CGIFILE $contents[0];
}
for(my $i=1;$i < scalar(@contents); $i++){
print CGIFILE $contents[$i];
}
close(CGIFILE);
}
print "\n\n";
This got Bugzilla up and running and talking ot the DB but desepite the fact that I had sendmail up and running on the server Bugzilla now had a hissy-fit each time you tried to add a bug or do anything that caused it to try to send an email. This is where the second retarded assumption made by the Bugzilla peeps was the culprit. firstly, their error reporting sucks donkey dick and was of no use at all so into the code I dived. After some poking around inside the source tree I figured the problem must lie within Bugzilla/BugMail.pm and sure enough in the last function of that file (line 868 of 876) I found this:
open(SENDMAIL, "|/usr/lib/sendmail $sendmailparam -t -i")
Yup, they had hard-coded in the location of sendmail! Again, there is no way to set the location of sendmail in any config file I could find so I had to manually edit this line to look like this:
open(SENDMAIL, "|/usr/sbin/sendmail $sendmailparam -t -i")
Once that was done all was fine and Bugzilla is now working but I really shouldn’t have had to jump through so many hoops to get it working!