Peculiarities of translation assessment

The present paper addresses a few major issues concerning student translation evaluation and considers the following questions:

1) Why is it so difficult to assess translations?

2) How can translation competences be evaluated?

3) What exactly should we look for while assessing classroom translations?

The question of language competence assessments is extensively studied in TEFL literature and there is hardly any new method in assessment of Basic English skills to consider. On the contrary, the question of students’ translation ability assessment is poorly researched and presents substantial difficulties for teachers.

When we assess speaking we look for accurate and appropriate use of grammar, vocabulary and pronunciation, appropriate cohesive means to link ideas within utterances, and reach content. Assessing reading skills we pay attention to the student’s ability to understand the story, extract its main idea, find the information which is not clearly stated in the text but can be inferred form it. Written work assessment deals with evaluating the level of mastering tools of the language: organization and structure, presence of topic sentences and supporting ideas and details, word choice, precise syntax, errorless spelling and punctuation.

Everything gets a lot more complicated when we turn to translation assessment. What should be taken into consideration? How important are strong spelling and grammar skills if the target text fails to convey the ideas expressed in a source text? Or should an error in spelling be counted if it does not prevent from understanding the main idea of the text, which was communicated quite clearly.

In this light it is now appropriate to discuss what translation competences are. Waddington defines 'translation competence' as "a combination of linguistic competence and the ability to translate". Thus we can distinguish three basic elements comprising translation competence. The first one can be defined as the "comprehension of the source text, meaning that the primary skill needed to translate is to well comprehend the source text.” The second component then is the ability to produce a text in the target language since competent translators must be excellent users of their mother tongue. This ability should not be limited to what they have acquired subconsciously from their environment. Translators must have the specialists’ knowledge of the language to which they are translating. The third element in translation competence structure is "editing competence". It requires a detailed analysis and comparative-contrastive study of both source text and target text. This comparison leads to the translator's awareness of possible mismatches in his rendering. He then will be able to revise his product in the target language (6).

The problem with assessment of a translation competence is that you can evaluate it only indirectly through studying translation performance. Literature review reveals two models for competence evaluation – German and Anglophone.

German model is of a more analytical nature and is based on micro-textual analysis of texts. Evaluation of translation according to this model is done based on scrupulous comparison of the source text and target text and bares some quantitative features. The comparison is realized at all levels including a morpheme level to guarantee the exact matching of source language items and their target language equivalents. One of the shortcomings of this method is ignoring cultural elements.

Anglophone method involves less analysis and is performed at a macro textual level and culture is considered a crucial element which impacts the translation process and product. In fact, the comparison of the source text and target text in Anglophone tradition is indirect. As a result, there is less objectivity in testing systems.

In translation classes depending on the class objective teachers may resort to different methodology which, in turn, leads to different types of testing. In translation assessment teachers use the same type of tests that are generally given to regular TEFL students: placement tests, diagnostic tests, progress tests and achievement tests.

Speaking of placement tests it should be noted that they can be used not just as means to determine the initial competence of translation students, but also as an instrument in reorganization and revision of a curriculum. According to the results students show foreign languages departments may have to decide to implement corrective or intensive courses.

Diagnostic tests are meant to detect student problems in order to get rid of them unless it is too late in the semester. Such kinds of texts are supposed to encourage students to correct their weak points.

Progress tests are the most popular and frequently set tests in a university classroom. The objective of a progress test is to determine if the students have mastered material the material that has already been taught. In theory, if the teaching has been sufficient, if the syllabus is organized efficiently, if the test is well written and of course, if the students have been attentive, marks on a progress test should be high. If the marks are not all above 75 out of 100, then the instructor will have to determine why and alter the weekly course distribution (2). Such kind of tests is to be done on a weekly or biweekly basis depending on a number of classes taught per week and includes different translation dictations, vocabulary quizzes, back translations and others.

Achievement tests are designed to determine whether the students have mastered the scope of knowledge and skills specified in the syllabus. Achievement tests should include tasks that cover all the material studied during the semester. Their results should be studied very closely in order to evaluate not just the level of students’ translation competence but also the strengths and weaknesses of the program which has been offered to students.

The first three types of tests mentioned above belong to the category of formative assessment. Formative assessment takes place during the semester and is designed to prompt instructors on how to adjust their teaching. Feedback from formative assessment must be communicated to students as soon as possible. Students react more positively to formative assessment if the results are analyzed by the instructor and the teaching style or class content is altered according to their needs.

Achievement tests are referred to as summative assessment which contrasts with formative assessment first of all by its purpose. The purpose of summative assessment is to attribute value, and for that reason it is oftentimes more quantitative than the qualitative formative assessment (3). Summative assessment occurs at the midpoint and/or end of instruction so as to determine the extent to which syllabus objectives have been met.

Both types of assessment are necessary and complementary. However, if summative evaluation shows that the majority of the class is not at the level the instructor had targeted, then it has come too late and the formative assessment was also not sufficiently well planned (3).

Both teachers and students can learn from tests, both can benefit a lot. Teachers can elicit a lot of immediate feedback from their students to adjust their ways of teaching based on the students' needs. Students themselves can learn from their results. They can change or adjust their ways of learning.

Let us consider some of the ways to organize translation competence assessment. According to Larson (4) there exist five ways to evaluate classroom translation:

- Comprehension checks;

- Naturalness and readability testing;

- Consistency checks;

- Comparison with the source text;

- Back – translation into the source language.

Good comprehension testing is the key to a good translation. The purpose of this test is to see whether the text for translation is understood correctly. One of the ways to conduct this type of test is to ask students give a summary of the material read. Another option is to ask different types of questions, each with different purpose: to give information about the discourse style, details, main points of meaning. This test is intended to assess students’ comprehension competence.

The purpose of naturalness tests is to see if the form of the translation is natural and the style is appropriate. Performing this type of assessment the teacher should first read through the whole section to check the flow of translation and overall meaning of the text. Then the text has to be checked for accuracy and compared with the source text for omissions, additions, or any changes of meaning. Then the target text can be checked for readability. It is done by asking somebody unfamiliar with the text to read it. As he does so, the teacher will notice any places where the reader hesitates, stops and re-reads the sentence. These pauses and hesitations signal problems in translation. The best way to do such assessment is to ask end users of the translation to read it since what is readable for one type of audience is not readable for another.

After the translation is performed it is very important to check it for consistency. This check concerns both the technical details of text presentation and vocabulary use. This is especially useful when long technical, political or religious texts are translated.

In the final review, the formatting of the text and of any supplementary material like footnotes, glossary, and index or table of contents, should also be checked for formatting style.

Comparison with the source text is done to check for equivalence of information content. This type of verification should be done by students before they turn in their translations and later is done by the teacher who checks the target text against the source text in order to identify problems.

The last way to check a translation is by having other students make a back – translation of the translated text into the source language. In case this type of assessment is used it is important that back-translation is made by students who haven’t seen the source text.

Translation assessment can be done at several levels: by the translator (self-assessment), by other students (peer assessment) and by the instructor.

The type of translation the instructor should choose depends on the type of translation performed. Basically there are three options to choose from: to assess general impression, to count errors and to fill in an analytical grid.

The first two methods though sometimes used by the instructors are not recommended for being ineffective and subjective. An experienced instructor is able to differentiate between a 75/100 paper and a 90/100 paper, but a general impression mark as not convincing for the student since it does not provide any reasoning for the grade. Error count method does not seem effective due to the fact that it fails to take into consideration the seriousness of mistakes made.

The most appropriate type of translation assessment is by using an analytical grid. The grids developed for assessing speaking and writing can be used for this purposes. The grid provides the instructor with the opportunity to set clear assessment criteria based on simple arithmetic.

Here is an example of such grid:

Working with this grid the instructor evaluates translator competences: fluency/flow is evaluated through performing naturalness and readability testing, terminology evaluation includes consistency testing and comparison with the source text, general content is assessed by comparing the impact the target text makes on a reader.

Another way to assess translation suggested by Farahzad (1) is to evaluate two main features in each unit of translation: accuracy and appropriateness. Accuracy can be checked at a sentence / clause level while appropriateness is checked at a text level. Accuracy conveys the information of the source text in a precise manner; appropriateness implies fluency and native-like style of the target text. Thus unnatural translations which convey the message correctly receive low grades and inaccurate translations receive no grades at all no matter how natural and fluent they may sound.

The third way to assess a target text is to do it holistically, especially with long texts. The instructor may come up with his own evaluation criteria or use the following scheme:

1) Accuracy - 20 percent;

2) Appropriateness - 20 percent;

3) Naturalness - 20 percent;

4) Cohesion - 20 percent;

5) Style of discourse/choice of words - 20 percent.

The next way to evaluate the text is an objectified scoring, where sentences and clauses are translation units. Thus each verb in a sentence of a source text gets a score, and each verb in a clause gets a half-score. After accuracy and appropiacy is evaluated, the text may be checked for cohesion and style (e.g. transitional, appropriate use of pronouns, linkages, etc.). If the source text is neutral the instructor may allot a smaller number of points to the translation that in cases where preservation of style is important.

Yet there is one more scheme to evaluate assessment based on nature of the errors made:

- Language errors;

- Translation errors.

Language errors are inappropriate renderings which affect expression in the target language; these are divided into five categories: spelling, grammar, lexical items, text and style.

Translation errors are errors of two types:

- Inappropriate renderings which affect the understanding of the source text; these are divided into eight categories: countersense, faux sense, nonsense, addition, omission, unresolved extra linguistic references, loss of meaning, and inappropriate linguistic variation (register, style, dialect, etc.);

- Inadequate renderings which affect the transmission of either the main function or secondary functions of the source text (1).

Instructors may also ask their students for perform assessment themselves. Asking students to assess their own progress is one way of initiating them to see their work objectively. Below is an example of a translation student self-assessment paper that can be given to the students at the beginning of the semester or course.

An instructor may add/change evaluative statements for the particular course and for different level of students. Some students may show surprise at the mark they receive. A self-evaluation sheet filled out directly after an assignment may provide the student with helpful clues to their weaknesses.

Students prove to be effective evaluators and revisers of each other’s work. They are even more effective when they help decide on the criteria for the assignment undertaken. For example, students can agree that spelling errors which do not prevent from understanding would not be considered as serious, but errors that change the message to be conveyed would be. They may not be asked to put a mark to the work, but they can find areas in the translation that are not clear or which they themselves translated differently. In fact peer assessment is an extremely useful learning experience (2).

One final point on the topic is that no matter what sort of assessment instructors may choose it is helpful to monitor students’ mistakes by using the following chart:

Under "Mistakes" students write the word, phrase or sentence which was understood as incorrect in their translation.

Under "Possible Correction" they try to produce an "error free" version.

The source of the answer for students' correction is entered under the column "Source" as: 'Myself'; 'Peer'; 'Dictionary'; 'Teacher'.

The column "Type of Mistake", filled in by the students, can become a good exercise to help students recognize what types of mistake they are making and consequently eliminate them (5).

REFERENCES

1. Farahzad, F. (1992), Testing Achievement in Translation Classes. Amsterdam/ Philadelphia. John Benjamins Publishing Co.

2. Heaton, J.B. (1990), Classroom testing. Longman, New York.

3. Heaton, J.B., (1990), Writing English Language Tests. Longman, New York.

4. Larson, Mildred L. (1984). Meaning-based translation. University press of America.

5. Sainz, Julia M. (1992), Student-Centered Corrections of Translations. John Benjamins Publishing Co., Amsterdam/ Philadelphia