Sentence-layer annotation represents the most coarse-grained annotation in this corpus. We adhere to definitions of objectivity and subjectivity introduced in (Wiebe et al., 2005). Additionally, we followed guidelines drawn from (Balahur & Steinberger, 2009). Their clarifications proved to be quite effective, raising inter-annotator agreement in a sentence-layer polarity annotation task from about 50% to >80%. All sentences were annotated in two dimensions.

The first dimension covers the factual nature of the sentence, i.e. whether it provides objective information or if it is intended to express an opinion, belief or subjective argument. Therefore, it is either objective or subjective. The second dimension covers the semantic orientation of the sentence, i.e. its polarity. Thus, it is either positive, negative or neutral.

In the second layer, we model the contextually interpreted sentiments on the levels of words and NP/PP phrases. That is, the annotation decisions are based on the meaning of the words in the context of the sentence.

Word sentiment markers: The sentiments on the level of individual words are expressed by single character markers added at the end of the words.

A word might be positive (+), negative(-), neutral(empty), a shifter (~), an intensifier (^), or a diminisher (%).

If a word ends with a hyphen (e.g., "auf beziehungs-_ bzw. partnerschaftliche Probleme-", an underscore is added to the word in order to prevent missinterpretations of the hyphen as a negative marker.

Currently, only words that are part of an NP/PP are marked with sentiment markers. Annotated words are nouns, adjectives, negation particles, prepositions, adverbs.

The world level annotation was done by 3 persons individually. The individual results were harmonized into a single reference annotation.

Phrase level markers:

Each phrase is marked up textually by brackets, e.g. "[auf beziehungs-_ bzw. partnerschaftliche Probleme-]". The type of a phrase (NP/PP) is not written to the brackets. We follow largely the annotation model of TIGER for structuring embedded NPs and PPs.

Currently, the following limitations with regard to TIGER exist: (1) Adjectival phrases are not marked up (2) Relative or infinitival sentences are not included in NPs/PPs if they appear at the end of a phrase or if the are discontiguous. We do not only annotate the phrases which immediately contain words that are marked up as polar. Any dependent subphrase (NP/PP) is integrated into all its dominating NPs/PPs, e.g. "[Die tieferen Ursachen [der Faszination+]]". Dependent subphrases without any polar words are also included, however, there is no internal bracketing for them, e.g. "[hohe+ Ansprüche an Qualität und Lage]"

At the level of phrases, we distinguish the following markers: positive (+), negative (-), neutral(0), bipolar (#). The category 'bipolar' is used mainly for coordinations where negative and positive sentiments of something are kept in balance by the writer. This is quite common for a lot of binomial constructions as "Krieg und Frieden".