Computer Immune Systems

LL writes "We might soon be seeing commercial delivery of autoimmune security systems. Rather than the surface bit pattern detections of antivirus checkers, these system attempt to provoke virii in a secure area (IBM) or match network packets against signature tags (Forrest). The interesting plug is that the author suggests that large programs such as operating systems should be made in such a way that no two copies are exactly alike. Now guess what favourite beast has this trait?"

Not unique

If you're talking about linux, why should it have this trait? Most people still use a stock kernel from their distribution. That's a lot of people using the Redhat binary kernel, say, which will be identical for every person using it.

Sure, you can play with the config or use patches or whatever, but a lot of the code will come out the same. It's not like the compiler puts some kind of unique fingerprint on the kernel you build.

axolotl

immune system analogy flawed

There are some broad analogies between biology, ecology, and computer systems. For example, "monocultures" are susceptible to "viruses". And many "viruses" can be detected effectively by tracking the appearance of "fragments" (of code or proteins) and correlating that with computer system damage. But, biology or not, those are ideas that any good engineer should come up with anyway.

Perhaps the biggest point of departure is that biological systems are evolutionary, while computer systems are designed by humans, with knowledge of the possible countermeasures. That means that many immune system strategies just won't translate.

But even more important is perhaps the observation that most biological systems (even plants and most animals) don't even have immune systems. They rely on other mechanisms for their defense, mechanisms that many engineers would probably consider "good engineering": make it hard for the viruses to get in, destroy viruses that do get in, minimize the effects of infection if it does occur, stop the spread of infection with various barriers, and have lots of redundancy. The evolutionary pressures for some animals to develop immune systems probably simply don't exist for computer systems.

So, if you want to push the biology analogy, it may well be better to do without an immune system and to simply design good, strong systems.

Way off-topic, but...

I know it's OT, but I thought it was cool..

A few years ago, Wired (before they lost their edge) ran a pseudo-retrospect issue from the future, in which they reviewed the turn of the millenium from a few decades ahead. It was a prety neat diversion. Anyhoo...

One of the main articles dealt with 'The Plague', a super-flu/AIDS/Ebola mutation that threatened to wipe out humanity. (It's striking how biologically apropos the computer virus analogy is, and how well it tracks with real life problems, solutions and latest computer development) The article was written in retrospect, like the whole issue, and in the form of interview with one of the top researchers involved in stopping the disease.

The truly neat thing about the story, and what keeps me remembering it, was that the disease was cracked not by medically traditional means but by a mathematician who found a way of attacking the geometric form of the virus. I don't know how unconventional this approach is in virology, but the cross-polination of medicine and math really struck me.

I'm a very strong believer in gestalt thinking, and in the fact that laws of nature from one field map remarkably well onto seemingly unrelated fields. Take Newton's Laws of Motion, abstract a bit and apply to sociology. Action-reaction. The Law of Entropy seems to hold true when placed in the context of politics. :) Somehow it all ties in to Asimov's Psychohistory too.

This is why the article resonated with me, and why the topic of evolving virii triggered me to go OT about memetic cross-breeding.

Evolving viruses.

I expect any time now to hear that someone has introduced a virus that evolves in the sense that a genetic algorithm evolves a solution to a problem. The internet is large enough an e-ecosystem to support millions of copies of a virus, so even if the survival rate of the variants produced by breeding and mutation was very low, there might be enough survivors each generation to evolve into a truly dangerous virus.

--
It's October 6th. Where's W2K? Over the horizon again, eh?

Computer AIDS?

I was going to moderate the first 3 posts down another notch but I actually had a thought on this topic. I guess somebody else will have to wield the moderation on this one.

So the idea is to increase security in a number of ways including (but not limited to) having each copy of the OS be unique, and having the AV package put the subject in a box and taunt it. (For those of you who haven't seen it, now's a good time to watch that Monty Python "Holy Grail" movie.)

So how strong are the odds that such methods could inadvertently result in some sort of computer auto-immune disorder? Could our anti-virals manage to interpret the kernel as a virulent entity to be removed? Or, are we all just too smart (or lucky) for that to happen?
"Una piccola canzone, un piccolo ballo, poco seltzer giù i vostri pantaloni."

Nothing is foolproof

If the point you are trying to make is that nothing is foolproof, I imagine the antivirus researchers would agree with you. I would be very surprised if they thought they could put together a system that would solve the virus problem once and for all. If they did, they probably wouldn't be using biological immune systems as a prototype, since biological immune systems are far from foolproof. After all, we still get sick from time to time. Sometimes we even die of disease; in fact, without outside intervention in the form of modern medicines and treatments we would die of disease a lot more often than we probably care to think about.

However, like our bodily immune systems, these systems could serve as a first line of defense. Their advantage lies not so much in that they are universal proof against infection (they aren't), but in that against "routine" infections they shut the virus down before it has the opportunity to do any real damage, far faster than would be possible if human intervention were required. Inevitably, some infections will slip through (just as with biological immune systems), and when that happens you need outside intervention; i.e., the computer equivalent of a trip to the doctor's office.

-r

Re:Evolving viruses.

Polymorphic and 'mating' viruses have been around for quite a while. Polymorphic viruses adding random code to their source to 'shape-shift' and attempt to avoid any signature identification (basically sprinkling noops randomly around).

Some viruses are actually pairs of viruses, which, when they find each other (both infect the same file or piece of memory, etc.), will join and/or manifest some new behavior (start their payload).

Very interesting stuff actually. It's too bad that malicious virus writers have tainted the whole topic. Self-replicating, autonomous programs are very interesting.

Jazilla.org - the Java Mozilla [sourceforge.net]

Re:Virus solution - better security models

To be honest, the Unix security model is almost as weak as the Windows security model in this aspect.

What you've said is largely irrelevant. Here's something I once wrote on this matter. You can change the Perl references to Bash, to accord with your own statement. I wish I'd saved the links to Abigail's virus. Check DejaNews.

--tom

_______________________________

No, it's really far more complex than that.

You are correct that it is no mean trick to write a program that can damage the system it runs on, largely irrespective of what kind of system we're talking about. And so long as you can hoodwink some unwitting user into executing that program on their system, that program can, of course, cause damages commensurate with the privileges and capabilities of that user.

What you've failed to consider is how the dramatic cultural differences between Unix and the much-maligned consumerist toys serve to affect the issue to our benefit and their detriment.

Probably the most important of these cultural differences is that Unix has historically been a source-only world. Programs are distributed in the form of source code, code which shall be configured, built, and ultimately installed on the target machine. Programs solely accessible in machine language form fall immediately under a taint of mistrust.

Think back to the last time you read a notice from someone whom you've never heard of before that was asking you to go fetch some random binary program from some random place on the net and then to run that program under full sysadmin privileges? I can already see the incredulous Unix sysadmin reading that and bursting out in uncontrollable guffaws. Because the de facto standard for program interchange in Unix is as source code, a Unix programmer will be far less likely to fall for your ploy than would your average Prisoner of Bill, who has been lulled into gullibility by a binary-only culture.

But for the sake of the argument, let's say that you've found a way to effect this trick. Suppose you're an employee of some reasonably respected company that happens to produce a binary-only distribution of their commercial software, and you decide to sneak something wicked into the binary image. You manage to replace the standard, clean copy on your company's ftp or http server, or even floppies or CDs, with your own naughty version. People are accustomed to downloading from your company, or using your company's floppies, so they do as they've always done, run the installation as the superuser, and you thereby have your way with their system.

If this scenario were to play out, just how dangerous--how destructive--could it really prove? Whom could you harm, and who would be immune to your ploy? The answer is that you could only hurt those folks running the exact platform for which your binary had been compiled, and everybody else is unassailable. By platform, I mean the whole feature vector that includes processor chip (eg Sparc vs Intel), operating system (e.g. SGI vs BSD), shared libraries (e.g. libc vs glibc), and site-specific configuration (e.g. shadowed vs non-shadowed password files.

Let's not get too full of ourselves and pretend that the Unix culture's predilection for source-only program distribution derives only, or even mainly, from altruism. We have no choice in this matter. If you're on Unix, you don't have the source, then you can't run the program on all your diverse systems. And if Unix programmers do not provide source, they cannot hope to have their program as widely used as it would otherwise be.

Consumer-targetted systems from Microsoft or Apple are two instances are a static monoculture, as vulnerable to mayhap as a field of cloned sweet corn. It only takes one genetically engineered virus to bring down the whole field. Unix is different.

In his acclaimed essay, In The Beginning, Neal Stephenson writes:

It is this sort of acculturation that gives Unix hackers their confidence in the system, and the attitude of calm, unshakable, annoying superiority captured in the Dilbert cartoon. Windows 95 and MacOS are products, contrived by engineers in the service of specific companies. Unix, by contrast, is not so much a product as it is a painstakingly compiled oral history of the hacker subculture. It is our Gilgamesh epic.

What made old epics like Gilgamesh so powerful and so long-lived was that they were living bodies of narrative that many people knew by heart, and told over and over again--making their own personal embellishments whenever it struck their fancy. The bad embellishments were shouted down, the good ones picked up by others, polished, improved, and, over time, incorporated into the story. Likewise, Unix is known, loved, and understood by so many hackers that it can be re-created from scratch whenever someone needs it. This is very difficult to understand for people who are accustomed to thinking of OSes as things that absolutely have to be bought.

There is no one thing called Unix. Instead, Unix comprises a diverse set of subtly (and often not so subtly) variant platforms. A nefarious binary laced with exquisitely designed evil bullets hidden inside it can hurt only a few of us. When Apple and Microsoft laugh at our diversity, be sure to remind them that is it their lack of the same that contributes to their incredible vulnerability--and to our strength. Hybrid vigor ultimately wins out over a monoculture, for the latter is too in-bred and fragile to prove long viable.

Let me now return to your particular suggestion, that of a malignant Perl program activated by a Makefile rule at installation time. Because you're talking source code, and because Perl tries rather hard to attain a high level cross-platform intercompatibility, this form of subterfuge would appear exempt from the inherent protections stemming from diversity in variant Unix platforms. So, could your trick be done? How much of a problem could this really be? What might happen?

The answer is that of course, it could be done. And in point of fact, a demonstration model is already available, courtesy of Abigail. Guess what? There's no reason to run around like a chicken with its head cut off: the sky isn't falling. This sort of approach stands little chance of making a big splash, because you aren't going to insinuate it into a place that can affect a lot of people. Sure, you might catch a few folks, but just how long to you think this kind of thing will go unnoticed? Remember, it's in source code. That means anybody who wonders what happened can just look at it. There's a very low barrier to entry. And even if the naughtiness removes itself from your copy once its dirty deeds are done, that naughtiness is still sitting there in plain view for easy inspection back wherever you got your copy from.

Is there a way around this? Well, yes, if you're as clever as Ken Thompson. Fortunately, you aren't, and neither are the crackers. If they were, they'd doubtless receive more Turing Awards for their vaunted efforts. :-)

The only way you're going to get good propagation is if your nastiness into a copy that a lot of people will download and install. There's a very fine reason why so many archives contain a checksum of the image. It's to help with this problem. Security of course depends on several matters, including the strength of the algorithm and the integrity of the authenticating agent. But better that than nothing.

Let's talk about propagation some more. I assume that the goal is to have a notable impact, which means you need to spread your bad code as widely as possible. A hacked up install script, even if all goes to your liking, just doesn't have a very high rate of reproduction. First of all, how often do how many people install this software? Secondly, how do you plan to trick them into doing so? It's not really much of a challenge to get one person to this, especially if they trust. If that's your goal, maybe you'll succeed. But the risk of being traced and apprehended is high.

So how come this stuff can spread like wildfire amongst the OS-challenged? Can't whatever mechanism that's used there be used to get at the rest of us, too?

Over the last few years, a frighteningly frequent conduit of contagion for viral infection on toy systems has been the implicit, automatic execution of code with little or not manual intervention on the part of the box's owner. DOWN THIS PATH LIES MADNESS!. That this can ever, ever happen is as a plain a symptom of complete and total cretinization in the toybox world as you are ever going to see. It's stupid, it's crazy, and it's dangerous. Any programmer who even suggests it needs to go back to flipping hamburgers. Any user who asks for this feature needs to be quietly taken into the back room by the doleful men in long trenchcoats, where he will be told in no uncertain terms that his request is not only in the best interest of no one but criminals, but that he also now has a permanent record even for asking about it.

No, I don't care that a customer asked for it. Customers are idiots, just like any other user. So what if they pay you? They're still idiots, and it's your professional responsibility to act responsibly, to refuse to go along with their madnesses. The customer is not always right. In fact, they're very often wrong. A physician or a lawyer doesn't do whatever the customer requests, and neither do you. They, meaning the customers or users, simply don't have the background and training; they don't have the experience of seeing why automatic execution from untrustable source is the work of the Devil.

It's not as though we in Unix have never seen this issue before. In fact, we've seen it time and time again. And guess what? We recognized the problem and we addressed it. And we don't cater to that kind of lunacy anymore.

Here are a few concrete examples.

Remember when vi would--or at least, could--automatically execute macro commands embedded in a file in a specific way? That was a dubious feature called modelines. On my OpenBSD systems, if I type :set modeline, the program comes back and says set: the modeline option may never be turned on.

Another example of learning from our mistakes is the issue of shell archives. Instead of automatically running the sharfile through /bin/sh, there are specially made unshar programs that will do the common things, safely, and nothing else.

When CGI was first getting big, owners of toy systems would blindly install compilers and interpreters in such a way that these would easily execute arbitrary content coming in off the wire. Despite my pleas, both Netscape and Microsoft were actually advocating this! After a year of warning admins not to do this, and sending mail to the companies who were saying to just go ahead, nothing changed. So I released latro [perl.com]. Then and only then did various companies retract their suggestions, even though they'd been aware of the nature of the problem for a long, long time. Sure, you could be equally stupid on Unix, but for some reason, we weren't. History counts.

Implicit execution of untrusted material is simply stupid beyond words. And for some reason, the toybox people keep falling for the same chump moves, from MIME attachments to word processor and spreadsheet macros to embedded active scripting controls. I don't know quite why they just keep doing this crap. My hunch, and it's only a hunch, is that this is happening because Microsoft and their moronic minions simply cannot for the all the tea in China ever manage to think outside of their quaint but completely fictional little single-user universe. Maybe they don't hire people who come from a background in multiuser and/or networked computing systems. Maybe they don't hire people with real experience at all, just script-kiddies trying to make a buck legitimately but with no true understanding. Maybe the software makers simply can't say no to a customer request, no matter how suicidal they know that request to be. I don't know.

Whatever the cause, decades of history are completely and repeatedly ignored. They keep making the same mistakes, and they don't fix the underlying causes. Sure, there are things that are hard. Denial of service attacks are hard. People who know exactly all the ramifications of IP who go sending maliciously hand-crafted packets aren't much fun either.

But these highly technical ploys aren't why most folks on their toyboxes are being screwed up, down, left, right, and sideways. They're being screwed because of very simple matters. They don't have the notion of a protected execution mode. They don't have file permissions or memory protections. They automatically execute content willy-nilly, often with complete access to the whole machine. They expect a program to show up in binary not source form. They don't compare robust checksums from a strongly authenticated sources. They live in an infinitely vulnerable monoculture. They expect things to just magically happen for them without a thought or a care, and guess what? Their wishes are duly granted, much to their eventual dismay.

It is possible that mass-market factors may someday end up plaguing Unix systems in ways not so far removed from the stupidities that the toy boxes are riddled with. We just have to tell them no, and to condemn in the strongest and loudest possible terms any backsliding into insecurities that if we ever had, long ago banished. Looking at the Winix phenomenon, in which a dozen different vendors put together and ship their own Linux operating systems, all specifically constructed to be user-obsequious and Unix-hostile all in order to appease the lowered expectations of a hundred million Windows idiots, who, despite their numbes, really can still be wrong. The stupidity of the masses must never be underestimated.

Re:Artificial this, artifical that

I would first like to say that the above poster is spot on with his comments, and I found his comments facinating. I would, however, like to pose a question to fellow/.ers concerning the terminology around "Artificial".

IMNSHO, This term is very over used. Any time a system goes live on a network, it's deemed to be somehow "alive" by putting an Artificial in front of it. A good example of this was when IBMs deep blue beat the a grand master at chess (Kasparoff(sp?), it was hyped as a "giant leap forward for Artificial inteligence".

There's nothing artificial about it. It was the result of many of the greatest programs and chess master toiling for years to pull the project off.

Its more acurate name would be Automated Intelegance.

And this 'Artificial Immune System' is also just and automated series of self updating decisions. Taking the human out of the loop doesn't make it artificial, it just makes it more cost effective.
_________________________

Details on Forrest's research

Stephanie has some interesting intrusion detection methods. Rather than looking at signatures of data presented in an attack, her approach analyzes sequences of system calls used and compares those sequences against known "correct" behavior (for that particular program). It's strongly based on the genetic notion of self. Surprisingly good results with few false positives. But don't take my word for it - go to her site and read the white papers!

Sexual Reproduction of Computer Virii

Currently, in the scientific sense, computer viruses reproduce asexually. There is one parent involved and it produces an exact copy. But just like in science this is weak as there is no variation.

Theoretically it should be possible to create viruses that reproduce sexually. There are two parents involved and the offspring shares traits of both parents. Have data structures similar to chromosomes that hold traits of the virus such as where it is stored, what it does, how it reproduces, its lifetime...

The viruses would then go around looking for other viruses of the same basic type (species), mix together the chromosomes and create varied offspring. You could even have designated virus breeding grounds.

In the programming side of this, someone would create the basic structure (species) of a virus and a way to insert traits. Virus writers would then come around and specify the traits they want, and send it out (either to a "friend" or to a possible designated virus breeding ground).

This would create a new type of virus. One that will eventually become so varied that any in that species can not really be removed easily.