This is a numerical optimisation and not a machine learning project. The emphasis here is
on optimisation methods and their convergence not on classication performance. Still,
it is important to pose your problem in the best possible way and solve it with appropriate
tools. Always provide reference to sources of any claims, theorems, methods etc. Mathematical
rigour on par with the lecture and clear and succinct writeup (3k words total hard limit,
please include the word count) is expected.
All methods need to be implemented by you (no use of optimisation packages is allowed)
except for codes that were implemented by you or made available by us in the course of tutorials.
For setting up the classication problem you can use dedicated packages, but do not use the
built in optimisation routines.
Include your implementations of the optimisation methods and an excerpt from testing
script showing how they are called at the end of your report. Restrict the latter to the minimum
necessary to validate that you used your own implementations. The codes will not count towards
the word limit.
Optimisation problem underlying SVM classification.
Support vector machines (SVMs) are a well established and rigorously funded technique for
solution of classication problems in machine learning.
Choose a classication problem and the most appropriate SVM formulation for it: pri-
mal/dual, linear/nonlinear, choice of loss etc (justify appropriateness of all choices for
your classication problem).
Shortly introduce and explain the chosen formulation (no need for comprehensive deriva-
tion just key steps to demonstrate understanding).
Identify the type of the resulting optimisation problem and any challenges it poses.
Marks: 10 baseline + up to 5 bonus (more complicated problem: nonlinear, non-standard loss
Solve the obtained optimisation problem with two appropriate methods.
Application of the method and underlying theory:
Note: Theory will only be given marks if there is a serious attempt on numerical solution.
Propose two appropriate methods for solution of the optimisation problem (justify your
choices e.g. applicability, convergence, eciency etc).
Brie y recap known methods or introduce methods if they were not part of the syllabus
and apply them to your optimisation problem.
Discuss global convergence in the context of your problem (are there any results that are
applicable? provide references in case of either positive or negative statement).
Discuss theoretical local convergence rates predicted for your problem.
You can paraphrase theorems from the lecture or other respectable sources (books, journals,
rusted lecture notes etc). Always state the result and explain why it applies to your problem.
you cannot exactly match the problem with the theory, discuss what is the departure from
he theory in your case and what elect on convergence do you expect and why.