16/02/2024

The new ISO 5060 standard “Translation Services – Evaluation of Translation Output – General Guidance” was published in February. We take a look at the details and explain why clients and translation service providers who want to assess the quality of a delivered translation cannot ignore this standard.

There are more and more statutory regulations for products and their technical documentation, such as the Machinery Directive or the Medical Devices Regulation (MDR). These specifications are leading to more and more stringent requirements not only in technical editing, but also in translation processes. Ultimately, the aim is to minimise risks for product users, prevent liability claims against manufacturers and comply with legal requirements. At the same time, however, the risks are increasing as a result of the growing and often unprofessional use of machine translation and artificial intelligence.

Since 2020, the ISO committee for translation services has been working on the ISO 5060 standardisation project under the project management of Dr Christopher Kurz. The standard was written by industry experts for clients, translation service providers, translators and other interested parties who wish to evaluate their own work or work delivered to them. Experts working on the MQM framework were also on the committee, as was our Head of Quality Management, Eva-Maria Tillmann, as one of the DIN representatives. The German version DIN ISO 5060 is also close to finalisation.

Quality always needs an understandable benchmark and can only be considered in connection with the set requirements, regardless of the topic and context. It is then achieved when all requirements for a product or service have been met.
The standards ISO 17100 for human translation and ISO 18587 for the post-editing of machine translation and, last but not least, ISO 9001 oblige service providers to identify and document customer requirements and to design processes leading to the translation product as well as quality assurance measures and controls in such a way that, ideally, all requirements are met upon delivery to the customer.

Translations must therefore be considered as products that must meet requirements. A systematic comparison of source-language content and target-language content, taking into account the specifications of the translation project, is essential for their evaluation. The new ISO 5060 standard for the evaluation of translations is an ideal means of arriving at a meaningful and objective judgement on the quality of a delivered translation.

The standard can be used to evaluate human translation results as well as post-edited machine translations and even unedited machine translations. The evaluation can be carried out on the complete document or just an extract and after each step of the translation process – depending on what and who you want to evaluate. Clients can use it to evaluate the overall performance of their suppliers. Translation service providers, in turn, can systematically and objectively assess individual services provided by translators, revisers or post-editors or the results of individual machine translation systems. This is also possible retrospectively, e.g. in the event of a complaint about delivered translations.

For this systematic approach, it is essential that evaluations are performed according to fixed rules and that evaluators have the appropriate skills. So evaluations – regardless of who is evaluating – must always be reproducible and performed over a longer period of time according to the same criteria so that development is evident and identifiable.

Different text types or departments may have different requirements for translations, which is why the evaluation criteria can also be very diverse. However, the standard helps to create suitable evaluation systems based on specific requirements for all applications. Clients and service providers can also check their requirements identically. This is because if a requirement is not formulated clearly enough, where there is doubt it is not possible to decide clearly whether it has been met or whether there is a translation error.

Specifically, the standard helps users to create an error typology and clarifies error weighting, severity levels and critical errors. It guides you through the evaluation procedure, the creation of samples and an evaluation form and specifies requirements for the skills of evaluators. This makes the ISO 5060 standard an important tool for anyone who wants to assess and optimise quality.

Would you like to find out more about the standardised evaluation of translation quality or the new standard? If so, you are welcome to contact our Head of Quality Management at e.tillmann@oneword.de.