Antivirus FAQ
From CastleCopsWiki
| Caution | The article below is currently in beta and has not been reviewed for factual errors. |
Q: How does an antivirus work?
A: The basic approach of antiviruses relies on analyzing known malware, extracting characteristic portions of the code and creating signatures for them. This database of signatures will then be used to detect malware by scanning suspect files. This is a overly simplified view of course. Antiviruses also use heuristics which are guesses or rules of thumbs that enable them to either spot variants of known malware (based on similarity) or (less commonly) completely new malware.
Q: What will my antivirus protect me from? Does it just protect me from viruses? Will it detect trojans,worms as well? What about adware, spyware, keyloggers or rootkits?
A: Despite the name, antiviruses have traditionally taken on the task of handling not just viruses but also worms and to a lesser extent trojans. The name antivirus merely reflects the common layperson use of the term virus as a catchall for all types of malicious software. On the other hand, some people recommend the use of specialized anti-trojans to supplement antiviruses because they fear that antiviruses are more focused towards handling worms that infect millions and are slower on handling trojans which are not self replicating. The number of quality anti-trojans are however in decline and many have began to focus on spyware and ad-ware instead. (TDS-3 was discontinued, Ewido has being renamed to AVG anti-spyware).
The situation with respect to adware and spyware is however a bit more complicated. Many antivirus companies were relatively slow to respond to the rise of this new class of threats, and as such people began using specialised anti-adware or anti-spyware products instead to cover this area. Upon realization of this need, antivirus companies had to decision facing them. One possible decision was to start protecting users against this new class of threats by enlarging the boundary of responsibility. This was actually what many antivirus companies did such as KAV,Bitdefender did. Other companies however declared that protection against adware was a completely different area of responsibility and produced separate antispyware products or components that were distinct from the main antivirus product. This was what Trends, McAfee , Norton etc did.
So the question of whether your antivirus protects you from adware & spyware (which btw does not yet have a settled definition) depends on which antivirus you are using. As a first approximation if the product you are using does not have a separate antispyware product, your antivirus probably does that too. There are exceptions however like the free version of Antivir which does not handle adware but the full version does. Another surer method is to look at the definition updates and see if it handles common adware and spyware like look2me. It probably makes sense to use some antispyware on demand though as backup even if your antivirus handles adware.
With respect to keyloggers, most antiviruses handle illegal keyloggers dropped by worms as a matter of course. However many do not target legal keyloggers that you can buy off the shelf. It has being speculated that these keyloggers are often used to keep track of employees and employers would prefer if antiviruses do not detect them. A specialised anti-keylogger might be a good idea depending on your needs.
After adware, Rootkits have become the next craze. As of writing, many antiviruses (Antivir, Sophos, KAV, Fprot, AVG etc) have started testing or adding anti rootkit capabilities either as part of the main program or as standalone programs. None of these companies have started charging for antirootkit capability though, so it is likely that your antivirus will (or will soon be able to) handle rootkits. Notice that even without generic antirookit capability, it is still possible for antiviruses to handle specific rootkits because most of them are not perfect and can be spoted by some tell tale signs without using antirootkit technology. Until antiviruses are confirmed to have mature generic antirootkit capabilities, it probably doesn't hurt to use some free antirootkit scanners on demand like Sysinternal's rootkit revealer or Icesword/Darkspy etc.
Q: Why do antiviruses fail ?
A: Antivirus fail for many reasons. Chief among them is that they do not have a signature for a new malware encountered and is not caught by the heuristic either. Even if they do have a signature existing malware can be modified to evade antiviruses using various methods -hex editing, using packers (packed malware) unsupported by the antivirus (also see server-side polymorphism), polymorphic (encryption) , metamorphic (replacing code structures with equalavant code) or combination of methods. As such particular antivirus might have a signature for a given malware, but fail to detect it if it is packed. Lastly the antivirus might have a bug in its detection routine under certain situations.
Q: Are antivirus dying? I read that whitelisting is the future.
Antivirus technology is essentially based on blacklisting technology, where known "baddies" are listed. The problems with this are pretty obvious : one generally has to list every known instance of malware and there are literally thousands created every day (heuristics help but generate too many False positives), and in view of the rapidly increasing numbers of malware, recently interest as shifted towards a strategy/technology known as whitelisting. Whitelisting turns blacklisting on its head, instead of listing all the bad guys, it lists all the good guys in a "whitelist", and denies all others.
Whitelisting of files/processes is not a new idea, and it has the advantage of conforming to the "default deny" principle in security, where anything not known to be good is blocked. There is no doubt that whitelisting has great promise as one additional layer of security, but it is unlikely that whitelisting alone will replace antiviruses.
Consider the following problems
- There are at least as many (probably the good files outnumber the bad by orders of magnitude ) legitimate files as malware files out there. Even taking the windows system components, there are dozen versions of explorer etc due to updates. Even the biggest whitelist, lists only a small percentage of all safe files out there. Often you will be faced with the problem of finding a presumably innocent file but one that is not known to the whitelist. What will you do?
- Even with a complete whitelist, any whitelist distributor would have to figure out a way to somehow maintain and quickly distribute the list (which could be millions of hashed entries) to average users. Imagine the chaos if the whitelist wasn't quickly updated to reflect a new windows system component added by MS and denied it!
- Who is to maintain and distribute the whitelist? Even leveraging of digitally signed files released by big companies such as Microsoft that digitally signed their files (hence avoiding the need to keep a centralized big database of all hash entries, you just need to check against the public key and PKI infrastructure) is not a complete solution, because not every digitally signed file can be trusted. You can probably trust microsoft signed files (but see later) but what about ABC.Com? Being able to establish the identity of the owner of code is helpful, but it is still difficult to know who to trust. And this is assuming everyone digitally signed their code and signs up with a CA.
- Even if whitelists were the complete solution, one would still need antiviruses. How else would the maintainer of the list decide if a certain file was safe enough to be whitelisted? Or do you expect people to dissemble every file by hand to check if it was safe?
- Whitelisting also has a problem when formerly whitelisted safe files become dangerous. Consider exploits that affect a certain file format. Image files like Jpg files are generally safe. However periodically, exploits are found in image processing programs that suddenly make Jpg a threat. Scanners can easily handle this by scanning for a certain exploit in the image file but Whitelisting would be helpless against that. Can you imagine trying to whitelist every image that doesn't have that exploit? Scanners don't have the problem.
- On a lesser note, other problems with whitelisting files and processes also appear when you consider scripts and microsoft office files. Should you whitelist microsoft office files? Millions of these files are created every day, if so, what about the macros?
In short whitelisting is a very useful technology/layer, and can be recommended as an additional layer but it is unlikely to replace antivirus, particularly if you are not very knowledgeable and value ease of use.
Q: What is real time or on access protection/scanners?
A: These terms (as opposed to on demand scanning) refers to the ability of the antivirus to scan files on the fly as they are either created/read (opened)/modified or executed (or some combination of) without any human intervention. Real time protection of different antiviruses varies, some will scan files the moment they hit your hard-disk while others will scan them only before you execute them and many allow you to configure it as you like. You can see this difference when you are surfing the net, and some harmful file is deposited in your browser cache. Some antiviruses will scan this file once it is created and instantly alert, while others will stay inert unless it is executed. Some people prefer the former, but arguably, the later gives you equal amount of protection since a file that isn't run is harmless. Also the later approach is likely to be less computationally taxing on the computer.
Because real time protection or on access scanning can slow down the computer, many antiviruses differ in other ways as well. One is in the treatment of archives (etc zip,rar,7z) and/or realtime packers. Again some antiviruses real time protection will try to automatically scan through these files when they hit your hard disk and scan the files within, while others will not and some may up to a certain file size. The same argument applies, a harmful file sitting in a zip file will not harm the user. For any harm to occur, the zip file must first be opened, and when that happens the file is exposed and the real time protection will pick it up. In this case though, real time scanning of all archives (and packers) can slow the machine a lot particularly if the archive is large. While you can generally only pack a file once , files can be created several layers deep with archives, so called "nested archives". So many antiviruses actually allow you to set an option to stop scanning archives above a certain size, or to stop after a certain number of layers (to prevent so called logic bombs where the AV is tricked into consuming large amount of resources trying to scan a file that has several layers of archives)
Lastly, many antiviruses allow you to set real time protection only to scan certain executable extensions, or to exclude certain directories all of which can speed things up.
Q: Can on access scanners detect more than on demand scans?
'A: In general no. In fact, some options are used only when on demand because it would be too taxing to do it on access! On occasion, the dropper might be missed by the scanner (both on access and on demand), but the dropped malware might be still detected by the on access scanner, while the on demand scanner would not. A bigger exception occurs with some antiviruses like FSecure's DeepGuard, and KAV's PDM which use behavioral analysis methods that monitor and analyze processes as they run and alert on malicious behavior. This of course is not available in on demand scans. Memory scans on process start will detect what on demand scans (without memory scans) won't.
Q: What is a memory scanner? Are they important?
As the term suggests memory scanner scans the memory! They are very useful, particularly against packers which unpack their exe directly into memory. The problem here is that while all AVs claim to have memory scanners, there are different grades of memory scanners from "fake" memory scanners to "real memory scanners".
The lowest grade "memory scanners" aren't really memory scanners at all, they just enumerates the list of loaded modules (exe or dll) and scans the corresponding static binary files with the usual file scanner. Totally useless because those files will be protected by packers. In between are those that really scan the process memory, those that scan process and modules (dll). Then there are "complete memory scanners" that scan the entire memory and can detect code-injecting trojans.
However it is not realistic for complete memory scanners to work on access (most do complete memory scans on demand e.g AVG Antispyware), as the scanner cannot scan the memory of a process after every instruction performed. Some compromise by automatically doing a memory scan when a new process starts so the on access scanner might detect the sample seconds after it is executed while a on demand scan of the file won't. Main problem is the scanner may scan the memory before the malware is unpacked (bringing no improvement compared with a file scanner) or after it has been (partly) executed (and may therefore have already performed some malicious activities).
Q: What are heuristics? How are they tested?
A: Antiviruses also use heuristics which are guesses or rules of thumbs that enable them to either spot variants of known malware (based on similarity) or (less commonly and less reliably) completely new malware.
Passive/static heuristics rely on spotting code structures that represent suspicious behavior, this is often not easy to do so some also use Active/dynamic heuristics actually emulate parts of the code in a safe environment to see what it does (roughly speaking). Emulation can also be used to handle packers.
Another class of heuristics involves packer/Cryptor based detection. Because packers give Antivirus scanners a big problem (see next section), many antiviruses (AntiVir in particular) have decided to flag all executables that are packed with certain packers (plus perhaps some other crude file characteristic). Many of these packers are rarely used by legitimate software, or are generated by packers that have being cracked (the programmer is using a cracked version of the packer software). This results in False positives of course, since practically any file packed using a certain a packer will be flagged as possibly malicious. Otherwise clean cracks are often also flagged because they tend to be packed using non-mainstream packers or cracked ones.
Note there are generic signatures/detections that can detect closely related malware families or malware that has being modified from the origin but usually does not detect brand new malware created from the scratch. This may or may not be considered heuristics.
Generally, when we talk about antivirus heuristics, we are not talking about runtime heuristics or behavioral analysis type features, which analyze processes as they are run and spot malware based on their exhibited behavior. See HIPS FAQ . These are more risky than traditional antivirus heuristics, because the process is already run (or at least partly executed) on the real machine, and the scanner has to be quick to step in and/or reverse the malicious behavior.
Heuristics are by nature speculative and may result in false positives. They are tested by retrospective tests and less commonly by testing them against newly created malware (this is frowned upon by the community though).
Q: What is the difference between packers and archives and how do antiviruses handle them?
A: File archives like zip,rar,7z are familar to most users. Files in the archive have to be opened first and reside on your hard-disk before you can execute them. Run time Packers or run time compressed files such as UPX are different though. As the term "run time" suggests, such files are uncompressed only when they are run, and reside in system memory without hitting the hard-disk. Antiviruses differ in their ability to scan through both archives and packers (run time compressors), however only the latter is important. Why ?
Assume that malware.exe is a malware that the antivirus has a signature for. It is zipped into malware.zip. In this case, whether the real time protection of the antivirus, is capable of (or is set to) scanning through to the zip file or not isn't important. Because malware.exe has to be exposed on the hard-disk before it can be executed and the moment it is on the hard-disk it will be spotted.
Now assume that malware.exe is treated with a packer like UPX to yield a new executable malwarecompressed.exe . This time the ability of the antivirus to scan through the upxed file is critical. This is because malware.exe is never exposed on the hard-disk , when you run the new exe malwarecompressed.exe it doesn't deposit malware.exe on the hard-disk before running malware.exe. Instead malware.exe is directly piped into the system memory! This totally bypasses your real time protection, unless it is capable of handling UPX.
Most antiviruses today certainly handle UPX, but the problem though is that there are hundreds of packers out there (and some are modified versions or even brand new packers created from the scratch to evade antivirus unpacking routines), and handling each one requires a lot of work (so called static unpackers), particularly some like Armadillo are well protected with encryption that makes it difficult to unpack. Also it might be possible to fool the antivirus by modifying the packing stub (part of the packed files that indicates how to unpack the rest of the file). Another solution to the problem involves handling packers generically using emulation. This involves setting part of the code run to unpack itself. This method is not fully reliable however for various reasons (anti-emulation tricks where loops are put in for example).
Lacking these 2 methods, the antivirus has to have a separate signature for each malware.exe that is packed in a different way!
A good memory scanner also helps, because you scan the memory itself and not the hard-disk and any runtime compressed file will be exposed in the memory. The disadvantage of this is that, because the malware is already in memory, any detection might be too late and some packers can still remain encrypted in memory.
Because of the difficulty of handling packed files, some antivirus like Antivir have began to mark all files that are packed by obscure packers (or packers known to be used widely by malware writers such as cracked versions of legit packers) as suspicious or malicious regardless of the content in them. This saves a lot of time and errs on the side of caution but generates false positives. Some other antiviruses still do unpacking but the existence of a unusual packer pushes up the file's suspicion rating and many trigger a heuristic detection if other conditions are met.
Note: The question of testing how good unpacking support each antivirus has is very difficult.
Q: What is a web shield/http scanner ? Are they needed if you have a realtime monitor.
"When you have a HTTP scanner, all malware that comes to your computer through HTTP traffic will be scanned, analysed (and blocked) *before* it is handed over to the browser. So if you are surfing on a web page with exploits on it which are directly executed in memory of the browser (such as ANI, WMF, JPG etc.), this is the only way to stop them before they activate - except using a HIPS. An on-access guard will eventually catch these files as well - AFTER the browser has parsed, displayed and executed them The on-access scanner only sees the browser cache files, not the traffic before."
However web shields may slow down surfing however.
http://www.wilderssecurity.com/showthread.php?p=1073294#post1073294
Q: What is a email scanner? Are they important?
Email scanners are similar to HTTP scanner except they scan the email as it is going in, via POP3, IMAP etc. The same considerations for email scanners apply as for HTTP scanners.
Q: If one antivirus has more signatures than another, does it mean that it has better detection abilities?
In general, the number of signatures in each antivirus is not comparable. Some antiviruses use weak signatures like CRC checksums and can only detect one sample per signatures. Others might use heuristic and generic signatures that can detect thousands of samples with one signature. In addition, antiviruses have different unpacking abilities, some might add extra signatures for different packed versions of the same malware, while others might not need this because they are able to unpack it.
So two antiviruses might have very different number of signatures listed, but still might detect roughly the same number of malware samples.
In this Sunbelt blog entry, they compare antivirus detection results with database size and calculate an efficiency ratio of Size/number of file detected, but they admit even this is not really comparable, because besides the above mentioned problems, different antiviruses databases might be compressed differently, some might have more disinfection routines all of which will affect size of the database.
Q: What are in-the-wild malware/virus?
There are many many malware created daily. However, only a small subset of them are released and eventually spreads and gains a foothold in the real world, infecting real users (not on test machines!). Such malware that do succeed are said to be "in-the-wild".
Formerly the term malware "In-the-wild" also refers to the list of malware maintained by WildList Organization. In general a malware is added to this in-the-wild list if it is independently reported by a number of independent virus reporters.
Q: What are zoo malware ?
Zoo malware refer to malware that is not in the wild.
Q: What are retrospective tests ?
Retrospectively tests freeze signature updates for a certain period, and then test against new malware discovered in the period. The main purpose of such tests, is to test how well antiviruses can detect new/unknown malware samples that are presumably not in their signatures. In other words, this is a test of heuristics.
Retrospective tests are done by testers such as Avcomparatives where, signature updates are frozen for 3 months. Retrospective tests allows testers to test how well antiviruses can detect new/unknown malware without resorting to creating new malware (or modifying existing malware), a practice that is frowned upon by most experts.
Q: What are the different types of certification available for antiviruses ?
See here
Q: Are all antivirus tests equally reliable ? Which ones are most reliable?
No. While it is easy to go to some vx site , download a archive of malware and start testing with different antivirus scanners, such tests are at best informal tests. Doing proper testing is a very difficult task, requiring specific skills and demands different skills than that demanded of being a system administrator for instance.
While there are many pitfalls to proper testing, the main one is the difficulty in having a proper test-bed. Obviously for the test to be as valid as possible you want a huge test-bed. However, it is hard to ensure that everything you have is actually valid working malware.
Q: Can I run two antiviruses at the same time?
Yes and no.
Q: Do I need an antivirus if I already use <insert other security software>?
Maybe. Depends on your needs. Bold text
