ITSEC Evaluation, what could be improved?


'ITSEC Evaluation, what could be improved?' is an interesting question. There is plenty of scope for significant reduction in evaluation costs, improvements in information to potential users and policing of vendor claims, but that assumes ITSEC survives long enough to justify changes.

There is also a very big question about the value of certification schemes.

What is often forgotten is that security is all about time and deterrence.  IT systems are not a special case. One company can live with an encryption system that takes 8 hours to break while another may have information that has to be protected for much longer - and its usually not a government secret that requires real strength of protection, but something like a drug formula or the design of a relatively mundane product with a huge time critical potential market. Another company may be so cash poor that they cant spend any more money on IT and still remain solvent, so they just have to pray that they are never attacked.

So really the only intelligent way forward is Dynamic Risk Management rather than security but then ITSEC and the other criteria do not provide adequate information to map onto a DRM matrix and you have to fall back onto CRAMM or some other methodology that allows some wild assumptions and assertions.

Reality of life is that an increasing number of companies will certify to ISO 9000 and BS 7799, mandate ITSEC E3 for IT products, register under Data Protection, and provide a very good living for consultants but a very poor level of Risk Management. When something goes horribly wrong no one gets fired because they ticked all the Standards boxes.

The life of ITSEC as a criteria is uncertain, but to some degree limited.  This year the UK ITSEC Scheme Secretariat, UKITSECSS, agreed a programme with the UK ITSEC Commercial Licensed Evaluation Facilities, CLEFs (whoever said that govies couldn’t come up with short sexy titles for things), whereby they would begin running parallel evaluations to Common Criteria on all products being submitted this year for ITSEC evaluation.

The first US CC evaluation facilities were set up earlier this year and facilities had already been running in Canada with, I think, the Milky Way firewall being the first product to complete CC evaluation and be certified. With these North American evaluation centres becoming available, and a high percentage of IT product originating in North America, its highly likely that the demand on ITSEC CLEFs will reduce dramatically.

At present, the main advantage of ITSEC is that the mutual recognition agreements have finally been signed between UK, Germany, and France and ALL evaluated products are listed (including those that are restricted to government use only) in the product listings issued by each of the three governments, irrespective of the location of the ITSEC CLEF or the national Scheme Secretariat issuing the certificate.

It is an interesting note that the working parties originally took just under six months to agree the criteria and the methods of evaluation, but its taken eight years for the first three mutual recognition agreements to be developed by the lawyers and agreed by the politicians. It is also interesting to note that it took us six months primarily because the criteria had to be down graded to keep the US happy. The first cut criteria genuinely reported Assurance, Integrity, and Availability as whole system measurements so that a user could see not only that a given level of assurance was provided, but also that the certified product would function effectively as an IT product. That was subverted somewhat by the desire to change the ‘F’ grading system to appear a close mapping to Orange Book and to delete the resilience/availability reporting. The other change was to drop the obligation on vendors to disclose their Target Of Evaluation documents.

After the US Senate Committee objected to simply adopting ITSEC (Not Invented Here Syndrome) in place of TCSEC, NIST and NSA got together to produce FC-FIPS which was very close to the original ITSEC criteria and included some Quality Assurance elements. Common Criteria is the traditional political compromise that attempts to combine what are perceived as the best features of TCSEC, ITSEC, and FC-FIPS but probably fails to achieve that because its usually the lowest common denominator that gets selected.

For those who are interested in history, and differing views on the subject, I believe that FC-FIPS is still available as a document and is still listed in the Federal Information Processing Standard document lists of the National Institute of Standards and Technology, NIST. I never bothered to get a copy of the general publication but Version 1 (Draft issue December 1992) was a cracking read and only a little over an inch thick for Volume 1.

Having been involved in TCSEC, UKLC, and ITSEC evaluations, I have some views on security evaluations from a vendor perspective and some further views from a user perspective.

One interesting fact of TCSEC evaluations was that the evaluation work and issue of certificate by NCSC was at no cost to the product vendor. The cost placed on the vendor, and therefore eventually on the user, was that of engineering support costs and the cost of employing a Vendor Security Analyst with an NCSC ticket who would present the product to the evaluators and creatively describe a modified commercial product as being a product developed from ground up to meet a precise US Fed specification - or to be flexible and economic with the truth (thats a consequence of the Orange Book depending heavily on Mil-Std 2167A which is fine for specifying army boots or aircraft carriers but not designed for IT kit where there is no initial tight govt specification to which the product is designed). That situation, together with the fact that products were only evaluated in the (US) ‘national interest’, led to the most effective products being strictly controlled and available only to a small range of US and some NATO government agencies. It also resulted in certified product costing considerably more than the commercial product from which it derived.

ITSEC was intended to change all that and, to a limited extent, it has succeeded. The basic concept was that most ITSEC certified products would be available to anyone who wanted to buy them and that this would create a huge market that would result in certified products costing little more than standard commercial equivalents. What’s actually happened is many vendors, particularly firewall vendors, are rushing to get a certificate and then selling product that is either different from the evaluated product or is deployed and configured in a different form so that it does not necessarily provide the same assurance as the product which has been evaluated. Sometimes that’s because no one would seriously consider using the TOE configurations because they need something that reliably transmits
or processes information.

As an extreme example, I could go to PC World and buy a bog standard Intel clone PC with the usual ‘quality’ selection of packaged software and present that for evaluation as a secure product. At least in theory, I could construct a TOE that claimed the product to be E3 or E4 assurance if a particular installation process is followed. This process requires all cables to be cropped as close to the case as is possible with side cutters, an 8 lb lump hammer is then applied enthusiastically until the PC has been reduced to a pile of fragments, and the results are then mixed with concrete and poured into a suitable container. If I provided a mathematical model, maybe using Z or VDM and NPL security clauses, I might even be able to produce an E6 TOE. Product cost increase would be very small apart from the evaluation costs which would be excessive at E3, crude extortion at E4, and astronomical at E6.

Therefore, an ITSEC Certificate is only really meaningful if you have a set of ITSEC documentation, a copy of the product TOE and the time and skill to decide if the product you are attempting to install actually matches all of that in the configuration which you intend to employ. The main value of the certificate is that most vendors have actually been forced to consider their product more carefully, and remove some of the bugs that they are normally happy to ship out to customers in standard product. The documentation is usually also a great improvement.For that privilege you will be paying a premium to cover the very high cost of evaluation and certification because the vendor gets billed by the CLEF for the evaluation, also has to carry evaluation  support costs for his own engineers, and the CLEF then passes on the costs they are charged by the Scheme Secretariat.

From a vendor perspective the CLEF charges can be excessive. I received a quote from one CLEF (probably since renamed Creative Accounting Inc) for an evaluation that included a four week visit to a Californian development lab to investigate their quality assurance methods (although this facility held a variety of US and UK govt tickets for producing very high spec products for aerospace defence application). The really interesting factor was that the CLEF folk seemed particularly interested in visiting when it was good surfing conditions. Of course if you ask a CLEF why they are obscenely expensive, they will blame the charges from their Scheme Secretariat, who of course claim that their charges are lower than they should be and its all down to the CLEF.

It is probably impossible to produce a really effective system of evaluation because the moment a computer is installed people add products to it, change configurations, and subvert any security that it had on day one. BS7799 isn’t an answer because its also a fudge to produce something that might be adopted generally. What could be done though is for ITSEC, or any other scheme, to have product inter operability tables.

The main problem here is that most popular applications wouldn’t ever get listed because they just don’t offer the integrity and assurance and the vendors probably would not agree to providing the key vendor with enough access to their products to effect full integration.

An example of the approach would be where a vendor of a trusted operating system had his product evaluated on a specific range of hardware and then added a selection of products to provide a range of functionality popular with users. Each item would be evaluated and then the integrated set of products would be evaluated to identify any covert channels or other subversion of  the assurance and integrity of the platform product. It would almost certainly result in a range of availability levels depending on the mix of applications and their configuration. Its also probable that there would be several Assurance and Integrity levels for the same reasons.
It wouldn’t provide total solutions but it could provide workstation and server kits that could be bought as evaluated system modules and a risk policy could make assertions and assumptions about the network links, based on the form of authentication and encryption employed.

ITSEC as it stands today could be used to evaluate in this manner but it would be horribly expensive and the first attempts would produce a steep learning curve for the CLEF selected to do the evaluation.

However, the main challenge would be in getting enough vendors together to work co-operatively. Its not impossible because there are examples where a level of co-operation has been achieved in the IT industry, notably in the binary compatibility agreements that have made it possible for Intel and Intel clone hardware vendors to produce platforms that MS operating systems and products such as SCO and Linux will co-exist and inter-operate, providing a composite platform on which applications from a host of other vendors can be mounted. There might be considerable scope for improvement, but that co-operation has been largely responsible for the explosive uptake of computer products during the last 20 years.

Ian Johnstone-Bryden

Back