This document compares the performance characteristics, while running reports, of AviSys and the most expensively advertised competitor. These tests were performed by Perceptive Systems (AviSys) in our facilities. We performed the tests with great care to ensure fairness, and discussed some results with the competitor's personnel to be sure we were not doing something wrong.
The following was run on a 486DX2-66, 16MB RAM, 12ms hard drive, Windows for Workgroups 3.11, 4MB 32 bit Windows disk cache. (Performance fairly typical for the average birder's computer. With the 32 bit cache, this machine is faster running reports than many Pentium 60 machines and some Pentium 90 machines.) This is run with Birder's Diary Version 2.02.
Task |
AviSys |
Birder's Diary |
AviSys Advantage |
|---|---|---|---|
State List |
7 seconds | 2 min. 26 sec. | 21x faster |
ABA List |
7 seconds | 3 min. 36 seconds | 31x faster |
World List |
9 seconds | 5 min. 35 sec. | 37x faster |
World Summary |
21 seconds | 5 hours, 2 minutes | 857x faster |
To quote the competitor, "While [our software] will physically run with a 486 processor, it is a painful thing to watch!" We certainly agree. But let's not blame the computer -- look at the next benchmark. . .
The competitor recently increased the minimum requirements for his software to a Pentium with 16MB RAM. Then he raised it again to a Pentium 120 with 32MB of RAM! Therefore, we repeated the benchmarks on a Pentium 133 with 32MB RAM and a 10ms hard drive, running Windows 95. And we used his newest release, Version 2.5.
Task |
AviSys |
Birder's Diary |
AviSys Advantage |
State List |
2.5 seconds | 47 seconds | 19x faster |
ABA List |
2.5 seconds | 1 minute, 4 seconds | 26x faster |
World List |
3 seconds | 1 minute, 38 seconds | 33x faster |
World Summary |
8 seconds | 1 hour, 8 minutes | 510x faster |
Don't you prefer snappy software to software that leaves you sitting there tapping your fingers on the desk? Stare at your monitor for 1 minute, 4 seconds. . . .doing nothing. (Heck! Try it for just 47 seconds!) Now you understand!
We selected the most frequently used report types for the tests, based on our communications over the years with AviSys customers.
AviSys Version 4.5
Birder's Diary Version, 2.02, as of August, 1997 for the 486 benchmarks, and his very latest (11/26/98) Version 2.5 for the Pentium 133 benchmarks.
Competitor - 7,034 record sighting file.
AviSys - 7,262 record sighting file.
Both sighting files held identical data, except for an additional 228 records in AviSys. (7,000 records is about average for serious birders using AviSys for two years. Some AviSys users have over 70,000 records; many have more than 30,000.)
North America reports used the ABA list. "No Group" was selected. Sequence was taxonomic. (These selections are irrelevant in AviSys.)
In North America reports, a date range of 1/1/90 - 4/30/97 was used, and output was directed to Print Preview in both programs.
All reports were run twice, and the second run was used for the result. (This avoided any cache priming effect.) Then a repeat run was done to certify the result. (Using just one run would have been more to the detriment of the competitor's performance.)
Timing started at clicking the "Preview" (or equivalent) button in the last setup dialog and lasted until the first page was visible on screen. In AviSys, one report had two intervening option dialogs; those were immediately cleared and the time consumed was included in these results.The first three reports are three column lists of species names. The competitor calls them "Lists"; AviSys calls them "Species Seen" reports. Conversational language is used, below, to describe the reports:
"What species have I seen in California?" - Result 298 species
"What species have I seen in the ABA Area?" - Result 406 species
"What species have I seen in the entire World?" - Result 945 speciesThe final report, World Summary, is a formatted tabular report. It is a summary of species seen counts for each state, province and ABA reporting Area and Region.
Birder's Diary
ABA Annual Listing Report format.
Report deliberately stopped after 5 hours, 2 minutes (18,120 secs). It was still updating, and had progressed to "Pacific Ocean."
The competitor's support staff states their benchmark for this report, with a 5,700 record sighting file on a Pentium 120, runs 1 hour 30 minutes.
Here is an exact quotation from their Windows Help file on this subject:
"This report is by far the most complex in Birder's Diary. It will take three or four minutes to run if you have only a few sightings. It will take over an hour if you have thousands of sightings. It will take even longer if you have a slow computer."AviSys
Two AviSys reports: State Summary and World Summary, that additionally provide seen counts for all nations, show current checklist counts for all states and provinces, and show N.A. regional counts. That's far more information than the competitor's report. Total accumulated time for both reports used.
In answer to the competitor's complaint that his program runs much faster on the highest performance machines with a lot of RAM, we provide the following results from a test on a 300 MHZ Pentium II with 64MB RAM and a 9ms drive:
- AviSys runs in 6 seconds.
- The competitor runs in 41 minutes, 3 seconds.
- AviSys advantage: 410x faster.
- [The AviSys time can't get any faster than this because of the time required to respond to three dialogs, which used up 3 of the 6 seconds. Not counting "clicking" time, AviSys runs 820 times faster.]
Performance is absolutely vital in birding database software.
If data entry is not quick, easy and crisp, and if viewing data
and running reports are not quick and easy tasks, birders will
soon become disillusioned and will stop using the software or,
at best, will not get the benefit they paid for. (To put all this
in perspective, while at your computer, check your watch and then
stare at the screen for a full minute — if you can stand
it. Nothing else so approaches infinity.) We are birders, not
computer slaves. We want to get our data in, and get our reports
out with a minimum of grief, with little wasted time, and we want
to have fun doing it. We want results that enhance our birding
skills and enjoyment. And, yes, we want to fiddle around with
our data — quickly, though. We want a program to respond
to us — we don't want to wait for it.
Some software is quick, intuitive, easy and responsive. Other
software is literally sluggish, boring, and puzzling to use. Birders
need to make software decisions on those and other factors.
In case there is a question, the reports used here are not
hard-wired in AviSys. That fact is supported by the use of a random
date range. Every single record was read, and all results were
calculated.
It may seem unfair to include the State/Region Summary in this
comparison. After all, it is the competitor's slowest report (we
think). However, these reports were selected because they are
those most commonly used by AviSys users, and the State and World
Summary reports are used widely and regularly -- everybody wants
to see their state/province counts and nation/continent/Area counts
on one page. In fact, the summary reports are frequently the first
serious reports run by new users after they get a bunch of data
entered.
In addition to Print Preview, AviSys also has special high speed, scrollable, pageable, easy-to-read screen viewers for all reports and listings. Had those been used in the above tests, the AviSys reports that took 7 seconds would have run in 3 seconds. When AviSys users want to look at data on screen, as opposed to printing it on paper, they invariably use the viewers. The competitor's program does not have such viewers so we could not make comparisons.
The Birder's Diary was developed with Visual Basic, using the
Microsoft Access relational database engine (Jet). It is a RAD
(Rapid Application Development) product, using a typical commercial
database approach.
Unfortunately, birding database software is a brutal and deceptively
complex application that, when done with any degree of thoroughness,
quickly brings a commercial database system to its knees (see
the last report, above).
AviSys is a hand-coded program, using a combination of an extremely
reliable, proprietary database engine with indexed data files,
plus additional binary data files. All "relationships"
between files are handled in memory with special algorithms, and
all database calls are done at a low level (hands-on) by the program.
Basically, the terrible, crippling overhead of commercial database
systems is completely avoided in AviSys.
Of course, AviSys required a tremendous amount of programming,
most of which was done between 1988 and 1995 and has reliably
served legions of AviSys users since 1991. Thus, while AviSys
Version 4.5 has had a complete change of interface and many very
significant enhancements in 1996 through 1999, the most vital
and complex parts, the parts most responsible for speed, responsiveness,
and reliability, have stood the test of time.
The competitor has recently published his own benchmark results, using his own data. Those results suggest that somehow we cheated -- they claim much faster run times than we experienced for his software. However, the data set he used provides him with a great opportunity to turn in apparently faster results than our benchmarks indicate -- and he was not about to overlook that opportunity.
His Data: He uses a data set of "...6,500 sightings representing over 2,200 species."
The Impact: Since the benchmark reports (excepting the World Summary, which he conveniently doesn't use) are North America reports, and since he is obviously a world birder (over 2,200 species), it becomes apparent that the number of North America records used in his runs of the reports is significantly lower than the number of records we used in the benchmark.
Our Conclusion: The speed of his software is directly related to the number of records used in a report, and by looking at his results we can extrapolate (our educated guess) that he was using fewer than 2,000 North America records, while the data set we used contained 5,271 North America records out of 7,034 total. It is our impression that, by carefully selecting the data set, he can claim speeds nearly three times faster than we show for his software simply because he doesn't have to process as much data -- (and even then, he doesn't come close to AviSys' speed with the same reports)
(By the way, our N.A. vs World ratio of records, and the total record count, much more closely represent the average birding software user.)
Birder's Diary
29.5MB plus approximately 3.6MB of various DLL's and OCX's installed in your Windows directory by the competitor's setup program. Data file maintenance requires an additional 52MB free on drive C.
AviSys
4.8MB including a 1.2MB automatic backup file. No DLL's or OCX's. Data file maintenance requires 1MB free space on the drive where AviSys is installed.
Birder's Diary
No printed documentation. Windows Help Files, virtually unillustrated. Computer tutorial.
AviSys
128 page bound and fully illustrated user's guide, which also includes thorough coverage of birding record-keeping strategies and techniques. Fully illustrated Windows Help files.
Birder's Diary
Free support for first accumulated 30 minutes per customer. $50.00 per hour in 15 minute increments after the first 30 minutes. And they have just publicly announced that they will strongly enforce the $50.00 charge. (All information as received from the competitor. Verified with a second call to a different competitor representative.)
AviSys
Never a charge for support. Frankly, AviSys is so easy to use and trouble-free that support is simply not an issue here. We prefer a rock-solid product, with few support calls, to a product that generates so many calls that we would have to discourage them with a $50.00 charge.
[Listing][Attributes][Places][Benchmarks][Questions][New Feat][Docs][Quotes][Song]