Statistical Machine Translation (Winter 2018/2019)
|Course web page:||https://user.phil.hhu.de/~waszczuk/teaching/hhu-smt-wi18/|
|(This web page, which will be updated throughout the course.)|
|Office hours:||by appointment|
In this course, we will introduce the basic methods of statistical machine translation (SMT), such as word based and phrase based models (in the end, we will also consider more sophisticated methods). A main issue in SMT is not only the models themselves, but rather the estimation of their parameters, hence there is a strong focus on some methods of machine learning.
Format of Course
Every week, there will be one theoretical session where we introduce the main concepts, methods and techniques, and one practical session where we implement them in a subsequently growing program.
We will program in Java, hence some background is required (but there will be a short introduction).
Passing the Course
- BN: You will need to complete the theoretical and the programming exercises, which we will be working on during the practical sessions. You may also need some additional time at home to finalize and polish your solutions.
AP: Just as BN, plus there will be
a final written examinationa project. The project topics to choose from will be announced at the end of January.
Introduction and OverviewSlides
Probability theory (part I)Lecture slides, lab session slides
Probability theory (part II)Homework exercises and the accompanying code
Catch-up sessions doodle: https://doodle.com/poll/pfv3g6ea7v6xnqwePlease fill in the doodle by Wednesday, October 24!
Bayes’ theorem, parameter estimation (maximum a posteriori, maximum likelihood), and n-gram models (lecture slides)
Note: extra session from 16:30 to 18:00
IBM model I(lecture slides)
IBM model I (continued)
Higher IBM models (2 & 3)(lecture slides, updated with answers)
Higher IBM models (continued)Notes on efficient EM
IBM 4 and 5 (slides)Phrase-based translation (slides, phrase extraction algorithm)
Phrase-based models (continued)Viterbi for IBM-1, writeMostProbableAlignments2File, writeTransProbTable2File
Theoretical session: decoding, i.e., how to efficiently determine the best translation for a given sentence using a combination of the phrase-based model and the bigram language model (slides, complementary material)
Practical session: continue working on phrase extraction and phrase translation probability estimationUPDATE 20/01/2019: added complementary material about the Levenshtein alignment
Current trends in SMT
Theoretical session: selected approaches in neural MT (slides)Homework exercises (implementation of a simple decoding algorithm) and the accompanying code
Catch-up & projectThere will be no lecture, the remaining time will be dedicated to (i) finishing the last practical/theoretical exercises (if needed), and (ii) project presentation and discussions (for those who are interested to do the project and get AP).
Here is a potential project topic, which should give you some idea about how the SMT project can look like (and what are the deliverables).
There are other possible topics and you are encouraged to propose your own project topic.
This course draws heavily from the Statistical Machine Translation book by Philipp Koehn.