The dataset: online restaurant reviews in French

Our study is based on a dataset of online restaurant reviews in French. We have chosen Michelin-starred restaurants and non-starred restaurants, respectively 63 restaurants in Paris, from review sites. The total dataset, or corpus of reviews, consists of 1,000 reviews written in French from 126 restaurants. To produce annotated data, we first segmented 1,012 reviews into sentences. Then three annotators manually annotated each sentence into one of four evaluation types (which will be described in detail future posts):

(1) the reviewer’s opinion on the restaurant (positive, negative, or mixed opinion);

(2) the reviewer’s input/feedback to potential customers and restaurant owners (suggestion, advice, or warning)

(3) whether the reviewer wants to return to the restaurant (intention); and

(4) the reviewer’s neutral statement about the experience (description).

Using Fleiss’s Kappa measure, we obtained 0.90, which is considered ‘almost perfect’, according to the Landis and Koch (1977) scale. As a result, we obtained 2,943 annotated sentences, yet the class distribution is strongly unbalanced. Our approaches for solving such class imbalance problems will also be described in future posts.

Landis, J. Richard. and Gary G. Koch. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159-174.

Cite this article as: Hyun Jung KANG, “The dataset: online restaurant reviews in French”, in The Language of X, 26/02/2019,

Beyond the Polarity in Evaluating Experience

As internet technology develops, it has changed the way people search information and share user-generated content, as well as the way people communicate with each other. Hence, people are now engaged in a new form of word of mouth, Electronic Word of Mouth (eWOM). An increasing number of people are consulting online reviews in their decision-making processes. Google named this moment of searching, Zero Moment of Truth (ZMOT). According to the company (2011), 88% of consumers gather information from others’ experiences before the actual moment of purchase. Since today searching information is accessible from anywhere—on any device and at any given time, ZMOT is part of everyday life.

The growth of eWOM, as well as the availability of huge data coincide with those of studies in opinion mining (or also called as sentiment analysis). Majority studies in the field have focused on extracting positive and negative opinions expressed in online reviews and the target of these opinions. However, my research goes beyond the coarse-grained positive vs. negative opposition and propose a corpus-based model that detects evaluative language at a finer-grained level. With this model, we classify sentences into one of four evaluation types: opinion, suggestion, intention and description.

Moreover, previous works assume that positive and negative classes are evenly distributed. However, real time applications show that evaluation categories are highly imbalanced (Gopalakrishnan & Ramaswamy, 2014). This is confirmed in my data: observations per evaluation type are unevenly distributed (positive opinions account for 68% of all evaluations). To deal with the uneven distribution of evaluations, I use resampling and algorithmic approaches. The methodology for the classification problem will be detailed in the following posts.

Throughout the following posts, I will also consider questions such as: How do people express and evaluate their experience? What are some of the common linguistic features used in online reviews? What are some discourse features used in positive versus negative reviews? How the valence can be modified from one pole to the other? What are the topics talked about? How do evaluation change in time?

By introducing my research, I hope to provide a more comprehensive picture of the full spectrum of evaluations in opinion mining and sentiment analysis.

Dichter, Ernest. (1966). How word-of-mouth advertising works. Harvard Business Review, 44(6), 147-166.

Gopalakrishnan, Vinodhini., and Chandrasekaran Ramaswamy. (2014). Sentiment Learning from Imbalanced Dataset: An Ensemble Based Method, International Journal of Artificial Intelligence, vol. 12, no. 2, pp. 75-87.

Google and Shopper Sciences. (2011). “The Zero Moment of Truth Macro Study”, in Think With Google, 09/10/2018,

Cite this article as: Hyun Jung KANG, “Beyond the Polarity in Evaluating Experience”, in The Language of X, 18/02/2019,

Hello World!

In 2012, my friend Julian, who brought me into this world of computational linguistics, used to say that I should have a blog on linguistics. I thought that it was a great idea, but I had nothing to write about. The topic, “linguistics”, was too broad, and I didn’t know where to begin. Time flew, and I had always kept this idea in mind. But the truth is that I was obsessed with being perfect. I thought that I should write something important and that it would be better if I create my blog using web programming languages. Eventually, the perfectionism kept me away from having my blog.

In January 2019, I attended a seminar on research writing in English, organized by a professor of my lab. One of the things I appreciated in this seminar is that he showed his blog and explained to us how simple it is to create a blog on Hypotheses. He also enumerated advantages by having our research blog. I don’t know what struck me, but I was convinced to put it into action. Ta-da! The very next day, I requested to Hypotheses to create a blog, and here I am, writing my first post on my blog.

What Are My Goals?

First, I would like to share my research using various materials (e.g., images, slides, videos, etc.), which is something that I can’t do in journal papers. Secondly, I need to practice developing my ideas before formalizing them into a more traditional academic piece. Finally, I hope to get feedback from others so that I can improve my research.

Why “Language of X”?

When I was working on my master’s thesis on online restaurant reviews, in 2014, I came across Dan Jurafsky’s book, “The Language of Food”. Whereas Dan Jurafsky is mostly known for developing the first automatic system for semantic role labeling (SRL), I got to know him by this book. The book was fun to read; it combined history, linguistics, and sociology that surround our food. I found particularly interesting the author’s linguistic analysis on restaurant menus and online reviews. Being inspired by the book, I would like to explore what linguistics can tell us about food and other subjects. For this reason, I named the blog “Language of X”, leaving some room for other topics.