In the software business, quality is often left behind in the rush to be latest and greatest. ICSA Labs, a unit of Verizon Business, provides third-party testing and certification of security products. Security products are no exception, according to a study released Monday by ICSA Labs.
The company examined 20 years of its testing data to create the "ICSA Labs Product Assurance Report". The report indicates that nearly 80 percent of security products fail to perform as intended when first tested, and generally require two or more cycles of testing before achieving certification. Also see Broken Windows Revisited: Why Insecure Software and Security Products Hurt the Global Economy ICSA found the most common reason why a product fails during initial testing is that it doesn't adequately perform as intended. ICSA studied data from their seven certification programs; anti-virus, network firewall, Web application firewall, network IPS, IPSec VPN, SSL VPNs and custom testing, which are customized testing programs designed for specific clients. Across the seven product categories, core product functionality accounted for 78 percent of initial test failures. The failure of a product to completely and accurately log data was the second most common shortfall.
Examples include an anti-virus product failing to prevent infection and firewalls not filtering malicious traffic, ICSA noted in a release on the findings. Incomplete or inaccurate logging of who did what and when accounted for 58 percent of initial failures. According to the report, logging is a particular challenge for firewalls. The report findings suggest that logging is often considered a nuisance and is undervalued. Almost every network firewall (97 percent) or Web application firewall (80 percent) tested has experienced at least one logging problem. Security testing issues range from vulnerabilities that compromise the confidentiality or integrity of the system to random behavior that affects product availability.
Rounding out the top three, said ICSA, is the finding that 44 percent of security products had inherent security problems. Other issues identified in the study include poor product documentation and patching. ICSA officials said only 4 percent of the products tested in their labs pass their rigorous certification process in the first round.