"ISO 9000 doesn’t mean good quality, it just means you make the same mistakes every time!"

How many times have we heard people knock the ISO 9000 series of Quality Assurance standards with a comment like the one above? The implication of course is that an ISO certified company could just as likely manufacture bad product as good, and all that ISO 9000 certification achieves is a consistent approach. I am sure we have all heard something similar! There is some evidence to support the view but no explanation of why it has occurred, this article means to stimulate a discussion.

Being an assessor of both testing and calibration laboratories, ‘Metrology’ is close to my heart, it seems appropriate therefore, to start with the question: ‘How do we decide if a product is good or bad, during or following manufacture?’ The answer is simple and always comes back that ‘we measure it’! It follows then, that the accuracy of our measurement determines our success in ensuring that good product is passed and bad product is rejected. Or to put it a little more succinctly, ‘if we screw up the measurement we can ship screwed up product’. This applies to almost any product and at any stage of its manufacture or development. The converse is also true, we could of course, also be failing good product, and that can be very expensive!

We can consider the measuring process itself as a comparison against a reference standard or specified criteria. The accuracy of this comparison determines the pass/fail quality of the product. The calibration of our testing and measuring equipment is therefore fundamental, and is one very important part of any quality control process. ISO 9000 covers this well in clause 11.2, which in summary requires:

The equipment shall be used in a manner ensuring that the:

measurement uncertainty is known

accuracy and precision required is consistent with the specified capability

equipment is calibrated at prescribed intervals

calibration is against certified equipment with a known relationship to national / international standards

Further there shall be defined process for the equipment detailing the:

Control of all equipment

method of calibration and acceptance criteria of the equipment

identification and labelling of equipment

In addition, the results of calibrations must be reviewed and appropriate action taken if at any time the ‘before adjustment results’ are found unsatisfactory.

There are many ways to describe measurement, whether it is physical, electrical or indeed any method of evaluating against a known datum. It may be a questionnaire designed to measure feedback on how popular a new chocolate bar might be, or methods designed to identify the presence of tumours in the body. In both cases the measurement is an objective assessment against a reference standard or specified criteria. If we were off to the hospital, it would be good to think we could rely on the equipment!

For simplicity we can limit our discussion to the measurement of some electrical or physical quantity and consider some examples, remembering that the philosophy applies to all forms of measurement.

The ability to accurately measure the specified performance of a product, is a well-established requirement of the quality system. If our equipment is properly calibrated against known references for the parameters associated with our measurement we should be in good shape, passing good and failing bad.

Why then are we going wrong?

If we were manufacturing garden bird tables, it is likely that the specification would give a height, a table size, a base size etc. With what precision should we measure this height during production, do we need to measure it in microns, millimetres or centimetres, do we need anything with more accuracy than a tape measure? What tolerance should we give to the measurement? How should we define the calibration of the tape measure? Now these are all perfectly reasonable questions to ask, but lets face it provided nothing serious goes wrong with the tape measure we should have a fairly consistent set of finished bird tables. They should be fit for the purpose intended, whether they were designed for Blue tits, Blackbirds or Buzzards.

Could we take the same approach as we did for our bird table, if we were manufacturing machine tools, micrometer parts or watch components? The answer is possibly surprising, but it is still a resounding yes, the precision, accuracy and tolerance, need to be considered according to the requirements and in relation to the actual value being measured. If this is properly considered and the calibration is defined appropriately, then we can ensure that our product is not just of a consistent standard, but that it is also of the right standard.

If it so important to decide and define the calibration relative to the application concerned then how can it be acceptable for the vast majority of manufacturers to send their equipment out for calibration with just the words ‘please calibrate’ written on the purchase order?

How can the calibration laboratory know what precision is important, how accurate the equipment needs to be, and which parameters are important to the measurement concerned?

They may be providing an expert service but they are not mind readers.

I have heard many arguments about what happens when a calibration laboratory tries to find out the actual needs of a customer, most often it is claimed that the customers just want a certificate to show it has been calibrated. It is a requirement of an accredited laboratory to identify on the calibration certificate, any deviations to or exclusions from the specified calibration. This should relate to the equipment manufacturer handbook if no other specification is forthcoming. Many calibration laboratories refuse to accept this interpretation of the requirements and claim that their customers do not want attention drawn to the incomplete calibration as this may arouse their ISO 9000 assessor’s attention.

Does this mean that the question of suitability of calibration is only asked if the certificate declares and exclusion?

ISO 9000 also requires that when equipment is found to be out of calibration, appropriate measures be taken to quantify the impact on the produced goods. How can this be done when customers do not want a pre and post adjustment calibration, to find out the true extent of the measurement error?

How many products did we sell which did not meet the specified criteria?

There are many other horror stories associated with equipment calibration, it is time for the certification bodies to dig deeper into calibration requirements and more regularly investigate if :

The calibrations are defined in sufficient detail, to enable the correct precision to be used in measurement, with the necessary accuracy to ensure that the item being verified is correct within its allowable tolerances?

There are many reasons given for not insisting that ISO registered organisations apply the appropriate rules to equipment calibration. It is understandable that Certification bodies cannot find assessors with the necessary technical skills to ensure every calibration is properly defined for the vast range of company types who seek certification. But they could ensure that an appropriate system exists to define the calibration from the very earliest stage in a product development and certainly before any test equipment is procured.

It is a sorry state of affairs when the document specifically written by the International Standards Organisation for the control of metrological equipment, ISO 100012-1, is only used for guidance and not mandated. Surely the guidance should also be for the assessors, to review the approach taken and to ensure that a proper and appropriate system is in place!

All the excuses of difficulty, complexity and cost to industry to do it properly, simply do not stand up, the costs of not doing it must be significantly more. I appreciate the difficulty for a company with many thousands of items of test equipment in its inventory, but they were all purchased to perform a specific function, how did anyone decide whether a particular piece of equipment was fit for purpose? It must have been done once, when the equipment was originally specified, the ISO 9000 system should ensure that this evaluation is carried through into a defined calibration.

The certification bodies themselves are singularly failing in their duty to customers if they do not make sure that the quality system provides for, and ensures adequate and proper calibration.

Many of the items of equipment may be general purpose; where a calibration to the manufacturer handbook is usually all that is required. When a particular calibration does not contain all the clauses of the handbook, surely it is best if the calibration certificate properly identifies this. Some of the calibrations required by the handbook can be very specialised and could herald a significant increase in the cost of calibration if performed. Are they relevant to the measurement or the normal usage of the equipment? The equipment controller has merely to identify the equipment with a limited use label and highlight the aspects that have not been validated, then the user of the equipment is suitably informed and the situation is under control. Indeed by properly analysing the calibration needs of test equipment, significant savings can be made by specifying only the relevant parameters to be calibrated and which areas can be merely confidence checked.

Manufacturers are completely in the dark if they do not take the lead and define the calibrations they require.

If calibration were to be properly addressed throughout all ISO 9000 registered organisations and enforced appropriately by the certification bodies, then we would rapidly progress to a situation where ISO 9000 registration really did ensure a good quality product. Companies could then be reassured that they were not wasting their time in making erroneous measurements.

The reality is that only a few percent of all calibrations performed are accredited. The calibration laboratories are responding to customer demands to offer a variety of so called, ‘different levels of service’. The interesting point is that many of these alternative levels of calibration often do not meet the requirements of ISO 9000 itself and yet they are still common. As soon as the calibration requirements are in the slightest bit complex then usually the customers mistakenly rely on the calibration laboratory to use their crystal ball and predict the requirements. This unfortunately would seem to be the case in the majority of calibrations performed.

Could the widespread failure to properly address the calibration issue explain the variance seen in the quality of products, and lead to the not always positive reputation of ISO 9000?

With a little thought and the emergence of just a two-tier calibration system, we could resolve all these problems in a cost-effective way. We need a defined and appropriate calibration for all parameters that contribute to the uncertainty of our measurements either directly or indirectly. In addition we need a more simple confidence check to verify the measuring equipment is meeting its defined criteria. Both of these activities should be accredited to give independently verified confidence to the customers that all their equipment was performing as it should, and also mean the necessary emphasis was directed at the critical areas.

Testing for Electromagnetic Compatibility (EMC)

Electromagnetic Compatibility (EMC) testing has become an issue of wide ranging importance since the implementation of the EMC Directive in 1992. This European Directive basically requires that all electrical or electronic equipment sold in Europe will not cause undue interference to other apparatus (emissions) and will not be interfered with by other apparatus (immunity). Interference levels can be measured and characterised or simulated and applied in order to determine whether a particular apparatus meets or fails to meet its specified performance. Many tests have been devised to measure or simulate a variety of these signals and situations.

Manufacturers now have to consider EMC testing throughout the design and production of their product before applying CE marking to demonstrate that the mandatory requirements are met. The resulting increase in the number of EMC testing laboratories has meant a corresponding increase in the number of laboratories accredited for EMC testing. Although there are still a few EMC test laboratories who are not accredited, the additional confidence afforded to the manufacturer who uses an accredited testing laboratory for this complex work should not be under-estimated. The range of apparatus to be tested is so wide that the standards address them only in a simplistic way and therefore leave much room for interpretation. This interpretation can lead to large variations in results. All accredited EMC testing laboratories undergo an extensive assessment against the relevant testing standards to ensure that a consistent and repeatable approach is taken. The consensus achieved through the accreditation process has significantly reduced the level of interpretation to a more ‘black and white’ approach.

All accreditation is of course reliant on the inherent co-operation between the accreditation body and the laboratories. Wherever ‘grey areas’ persist, such as in the testing standards themselves, the quality of these relationships become even more important.

The growth of the EMC Test Laboratory Association (EMCTLA) is a success story whose roots are founded on these principles of co-operation and accreditation. Formed in 1991 by a few of the then accredited test laboratories, its 46 members now include almost all of the UKAS accredited EMC test laboratories in the UK and a few that are currently in the process of accreditation. It was formed to help address many of the issues of interpretation and to work toward a consistent approach to testing. It is interesting to note that almost all the members of the EMCTLA are in direct competition with each other and yet have managed to construct and manage a very healthy working organisation that has achieved widespread national and international recognition within the industry.

The EMCTLA is governed by a steering committee that is elected from the main committee, who meet about twice a year. The main work of the association is carried out within the working groups and the EMCTLA has four such groups to address areas of common interest. These cover: -

WG A, Military EMC testing

WG B, EMC Directive and Technical Guidance notes

WG C, Technical Memoranda

WG D, EMC for Radio and telecommunications

The working groups are very active and have produced a number of useful guidance and interpretative documents. Last year an EMCTLA / UKAS liaison committee was established which is run in conjunction with working group C. In addition to EMCTLA representatives, this group also comprises UKAS assessment managers and technical assessors. The EMCTLA is now responsible for administering the logistics of inter-laboratory comparisons and is active in seeking new methods and reference standards for developing further comparison exercises. (Results of these exercises are still of course sent direct to UKAS and remain confidential.)

EMCTLA members sit on UK National committees, are recognized by the DTI group responsible for the EMC Directive, and are represented in the European commission’s think tank for the EMC Directive. The EMCTLA has also proved to be a powerful international voice with links to The Associated Council of Independent Laboratories (ACIL) in the USA and has contacts with the VCCI in Japan and with many other bodies.

This article was printed in the UKAS Newsletter

As a technical assessor for UKAS since 1992, Phil has been involved in over 500 laboratory visits of EMC Testing and RF and microwave Calibration Laboratories.

Further information on the EMCTLA can be obtained from the secretary:

Dave Imeson

EMC Test Laboratories Association

PO Box 129

Romsey

Hampshire

SO51 6ZT

Tel: +44 (0)1794 323382

Fax: +44 (0)1794 324152

dave.imeson@btinternet.com