3. The Scanner (IVSCAN).

Regardless that the author of InVircible often claims that
"scanners are dead and should be replaced by generic anti-virus
methods", his product does include a virus-specific scanner -
IVSCAN. Like most other programs from the package, IVSCAN performs
some other anti-virus functions - like self-checking, anti-stealth
techniques, and so on. However, these (and the problems in them)
are covered elsewhere in this paper, because they and their
problems are common for all programs in the package that employ
such techniques. Here we shall concentrate on the ability - or the
lack thereof - of IVSCAN to detect known viruses. After all, this
is the main function of a known-virus scanner.

Testing a virus-specific on-demand (i.e. not memory-resident)
scanner, while difficult enough, is the relatively easiest task when
testing an anti-virus product. It is therefore not surprising that
such tests abound. In the Virus Test Center at the University of
Hamburg we have developed a stable methodology of measuring the
detection rate of on-demand scanners. The only problems that arise
are those caused by scanners which are so poorly designed, that they
are either buggy or "untestable" - for instance, crash during the
test, or are unable to produce a proper report file and so on.
IVSCAN does fall in the latter category, but nevertheless, for the
purposes of this paper, we made an exclusive effort to test it - an
effort that we normally wouldn't make for a regularly tested
product.

The problems that make IVSCAN "untestable" are mostly caused by
its awkward user interface. Those are covered in Section 9. Here
we shall concentrate only on the results of the tests. We will
mention here that among the most annoying problems was the
inability of the scanner to run unattended from batch files. If it
detects a virus, it stops at the end of the scanning and asks the
user whether the report file has to be displayed. There is no way
to force the program to exit immediately after the scan has been
completed.

Also, we were unable to test the scanner for boot sector virus
detection. The reason is that first, the product refuses to create a
report file when scanning floppies and, second, that it requires the
user to press a key each time it has found an infected floppy (but not
when the floppy is not infected). The floppy disk simulator we are
currently using is unable to handle the situation when different kinds
of user input are needed between each two scanned floppies. We can only
express our condolences to the users of the scanner who would have to
scan hundreds, if not thousands of floppies after an outbreak of a
boot sector virus.

The tests were performed on our well organized virus collection
([VirLib]), containing 21,986 samples of 5,789 different file
viruses. The testing protocol is clearly explained in [Tests]. Of
those viruses in our collection, IVSCAN detected 343 or 6% of the
viruses and 2425 or 11% of the samples. Just as a means for
comparison, one of the top performers, AVP 2.2, detected 21,977 or
99.96% of the samples and 5,777 or 99.86% of the viruses. We are
forced to admit that IVSCAN is the worst known-virus scanner we
have ever seen. Even MSAV is better than it.

The quality of the detection of IVSCAN wasn't great either.
Ideally, a scanner that performs exact identification, should have
reported 5,789 different file virus names. Alas, IVSCAN reported
only 66 different file virus names - clearly an unsatisfactory
result. Again, as a matter of comparison, FindVirus 7.08.11
reported 5,156 different file virus names.

Of those viruses from which IVSCAN detected at least some of the
samples, 57 file viruses were detected unreliably (i.e., at least
one sample was detected and at least one sample was not detected).
Unreliable virus detection is a very dangerous bug in known-virus
scanners - because it means that some infected objects will remain
undiscovered and will later cause a re-infection. According to many
security analysts, each re-infection is just as costly to the
victim companies as the original infection. As a comparison,
FindVirus 7.08.11 had only 7 unreliable detections of file viruses.

Finally, IVSCAN had 2 cases of unreliable identification (i.e.,
different samples of one and the same virus being reported under
different names) of file viruses.

All those numbers clearly indicate that the exact identification (or
even nearly exact identification) in IVSCAN is miserable to
non-existent. Obviously, this is yet another scanner that uses the
archaic and obsolete nowadays technology of looking for simple scan
strings. The users should not rely on it to tell them which
particular virus has infected their machine.

We would like also to mention that the scanner sometimes causes
false positives. We have a program that demonstrates the video
effect of one of the Murphy viruses. IVSCAN reported it as being
infected by "Murphy 2" during the installation. Interestingly, this
did not prevent the installation program from invoking the
integrity checker to compute a checksum of the suspected file - as
if it were clean. Had it indeed been a real virus, and had the user
failed to notice the report from the scanner (something that
happens easily, because at scan time the report flashes quickly and
then the report is presented to the user only on specific request),
the checksummer would have "conserved" a real virus as a clean file
in its database.

Interestingly enough, the scanner also reported some Central Point
Software anti-virus programs as "immunized by CPAV" and later the
report said "6 infected or immunized files found". Obviously, the
author of the scanner treats one of the products of his competitors
in the same way as a virus - something that this competitor might
not like too much. Even more curious - once the installation had
created the databases of checksums, later runs of the integrity
checker stopped reporting the Murphy "infection", but continued to
report the CPAV "immunization". Clearly, the author of the product
considers the presence of a competing product as something more
dangerous than the supposed presence of a virus and as something
that is more important to be reported to the user.

The fact that IVSCAN does not use the CARO standard scheme for
naming the viruses it detects does not make the matter easier and
additionally confuses the user. Clearly, this scanner is of a
rather low quality, does not perform adequately the function it is
designed to perform, and shouldn't be relied upon. In fact, it is
our conclusion that IVSCAN is the worst part of the (bad in
general) product. There are scanners with a much better detection
rate (and a much lower rate of unreliable detection and
identification too) - AVP, FindVirus, F-PROT, TbScan, to just name
a few. Even if the users decide to use InVircible as the anti-virus
package of their preference (something that is clearly not
advisable, having in mind the security holes in the other parts of
the package that are explained elsewhere in this paper and the fact
that all programs from the package intentionally destroy data), we
strongly urge them to at least replace the scanner part of the
package (IVSCAN) with one of the better and more reliable ones.

For those who would like to check the above results themselves, we
have made the full report produced from this test available via
anonymous ftp. The full URL reference is

ftp://ftp.informatik.uni-hamburg.de/pub/virus/texts/tests/vtc/invb601d.zip

The archive contains both the rough reports from the scanner when
run on our virus collection and the preprocessed reports, which make
computing the detection rates an easier task. The preprocessed
reports contain three columns - the full path of the file
containing the virus samples, the standard CARO name of the virus
in that file, and the name reported by the scanner (or blank if the
scanner has not detected any virus in the file).

Finally, we have to point out that in some cases, even when run in
scan-only (no repair) mode, IVSCAN can cause damage. As it turns
out, the program automatically cuts out the string "MsDos", if it
finds it at the end of the executable files. It did so with two of
the files on our machine. Unfortunately, those files contained
self-checking programs, and the self-checking included the area
containing the "MsDos" string. As a result of running IVSCAN on the
directory containing those programs, the latter were modified and
refused to run. It is always dangerous to modify the user's data
without permission. It is much more so, if the user is even not
notified about what is happening. This behavior, again, forces us
to classify InVircible as a Trojan Horse.

Bookmark/Search this post with:

delicious | digg | reddit | newsvine | furl | google | yahoo | technorati