Computers perform remarkably in formerly difficult tasks. This article reports the
preliminary results on the prediction of two characteristics of judgments of the European Court of Justice, which require the knowledge of concepts and doctrines
of European Union law and judicial decision-making: The legal importance (doctrinal outcome) and leeway to the national courts and legislators (deference). The
analysis relies on 1704 manually labelled judgments and trains a set of classifiers
based on word embedding, LSTM, and convolutional neural networks. While all
classifiers exceed simple baselines, the overall performance is weak. This suggests
first, that the models learn meaningful representations of the judgments. Second,
machine learning encounters significant challenges in the legal domain. These arise
doe to the small training data, significant class imbalance, and the characteristics of
the variables requiring external knowledge.
The article also outlines directions for future research.