L2 norm multiple kernel learning and its application to biomedical data fusion

Shi Yu1, Tillmann Falck2, Anneleen Daemen1, Leon-Charles Tranchevent1, Johan A.K. Suykens2, Bart De Moor1, Yves Moreau1

1Bioinformatics Group, Department of Electrical Engineering, Katholieke Universiteit Leuven, Kasteelpark Arenberg 10, Heverlee B-3001, Belgium

2Systems, Models and Control Group, Department of Electrical Engineering, Katholieke Universiteit Leuven, Kasteelpark Arenberg 10, Heverlee B-3001, Belgium

Table 2 in the paper: Summary of algorithms implemented in the paper
Algrithm Nr. Formulation Nr. Name Optimization Download Notes
1 1-A 1-SVM L&infin MKL SOCP one_class_Kinput_SOCP_Linf.m
1 1-B 1-SVM L&infin MKL QCQP qclp_oneclass_Linf_regularized.m For unreqularized 1-SVM 1-SVM L&infin MKL, set the lowerbound lb parameter to 0.
2 2-A 1-SVM L&infin (0.5) MKL SOCP one_class_Kinput_SOCP_Linf.m
2 2-B 1-SVM L&infin (0.5) MKL QCQP qclp_oneclass_Linf_regularized.m
3 3-A 1-SVM L1 MKL SOCP one_class_Kinput_SOCP_Linf.m Suppose the averagely combined kernel is Ka, use [{Ka}] as the input of Kmix
3 3-B 1-SVM L1 MKL QCQP qclp_oneclass_Linf_regularized.m Suppose the averagely combined kernel is Ka, use [{Ka}] as the input of Kmix and set the lb parameter to 0.
4 4-A 1-SVM L2 MKL SOCP one_class_Kinput_SOCP_L2.m
5 5-B SVM L&infin MKL QCQP qoqc_classicalSVM_Linf_multiclass
5 5-C SVM L&infin MKL SIP sip_classicalSVM_Linf_multiclass
6 6-B SVM L&infin (0.5) MKL QCQP qoqc_classicalSVM_multiclass_regularized.m
7 7-A SVM L1 MKL SOCP
7 7-B SVM L1 MKL QCQP sip_classicalSVM_Linf_multiclass Suppose the averagely combined kernel is Ka, use [{Ka}] as the input of Kmix
8 8-A SVM L2 MKL SOCP socp_SVM_sedumi_L2_multiclass.m
9 9-B Weighted SVM L&infin MKL QCQP qoqc_weighted_classicalSVM_binary.m
10 10-B Weighted SVM L&infin (0.5) MKL QCQP qoqc_weighted_classicalSVM_binary_regularized.m
11 11-B Weighted SVM L1 MKL QCQP qoqc_weighted_classicalSVM_binary.m Suppose the averagely combined kernel is Ka, use [{Ka}] as the input of Kmix
12 12-A Weighted SVM L2 MKL SOCP socp_weighted_classicalSVM_sedumi_L2_binary.m
13 13-B LSSVM L&infin MKL QCQP qoqc_LSSVM_Linf_multiclass_jointlambda The regularization parameter λ is jointly estimated in the code.
13 13-C LSSVM L&infin MKL SIP sip_LSSVM_Linf_MKL_multiclass_jointlambda.m The regularization parameter λ is jointly estimated in the code.
14 14-B LSSVM L&infin (0.5) MKL QCQP qoqc_LSSVM_multiclass_Linf_regularized.m
15 15-D LSSVM L1 MKL QCQP linsolve_LSSVM_multiclass_alpha.m
16 16-B LSSVM L2 MKL SOCP socp_LSSVM_MKL_cvx_L2_multiclass_jointlambda.m The regularization parameter λ is jointly estimated in the code.
16 16-C LSSVM L2 MKL SIP sip_lssvm_L2_MKL_multiclass_jointlambda2.m The regularization parameter λ is jointly estimated in the code.
17 17-B Weighted LSSVM L&infin MKL QCQP qoqc_weighted_LSSVM_Linf_multiclass_jointlambda.m The regularization parameter λ is jointly estimated in the code.
18 18-B Weighted LSSVM L&infin (0.5) MKL QCQP qoqc_weighted_LSSVM_Linf_multiclass_regularized.m
19 19-D Weighted LSSVM L1 MKL linear linsolve_weighted_LSSVM_multiclass_alpha.m the kernel input parameter is the averagely combined kernel matrix Ka
20 20-A Weighted LSSVM L2 MKL SOCP socp_weighted_LSSVM_MKL_cvx_L2_binary_jointlambda.m The regularization parameter λ is jointly estimated in the code.

To run most of the codes, you need to install Sedumi and MOSEK.

The Excel file of the complete experiment presented in Table 3: Classification of patients in rectal cancer clinical decision using microarray and proteomics data sets.
Download Notes
experiment3.xls The Excel file contains 8 sheets correspond to 8 MKL classifiers. The values are error of AUC. The numbers of selected genes and proteins range from 11 to 36. The area with yellow shadow correspond to Table 6 presented in the manuscript.

Supplementary Table 1: Ln-norm and Lm-norm MKL algorithms implemented in the Supplementary materials of the paper
Algrithm Nr. Name Optimization Download Notes
1 1-SVM Ln-norm MKL SOCP cvx one_class_Kinput_cvx_Ln_norm In the SOCP cvx formulation, the dual problem is solved. So the constraint in the MKL problem is based on the Ln-norm.
2 Vapnik's SVM Ln-norm MKL SOCP cvx classicalSVM_multiclass_cvx_Ln_norm.m In the SOCP cvx formulation, the dual problem is solved. So the constraint in the MKL problem is based on the Ln-norm.
3 Vapnik's SVM Lm-norm MKL SIP cvx sip_classicalSVM_multiclass_cvx_Lm_norm.m In the SIP cvx formulation, the kernel coefficients are regularized by the Lm-norm.
4 Weighted Vapnik's SVM Lm-norm MKL SOCP cvx weighted_classicalSVM_multiclass_cvx_Ln_norm.m In the SOCP cvx formulation, the dual problem is solved. So the constraint in the MKL problem is based on the Ln-norm.
5 LSSVM Ln-norm MKL SOCP cvx LSSVM_MKL_multiclass_jointlambda_cvx_Ln_norm.m In the SOCP cvx formulation, the dual problem is solved. So the constraint in the MKL problem is based on the Ln-norm. This algorithm only solves binary classification problem.
6 LSSVM Lm-norm MKL SIP cvx LSSVM_MKL_multiclass_jointlambda_sip_Lm_norm.m In the SIP cvx formulation, the kernel coefficients are regularized by the Lm-norm.
7 Weighted LSSVM Ln-norm MKL SOCP cvx Weighted_LSSVM_MKL_multiclass_jointlambda_cvx_Ln_norm.m In the SOCP cvx formulation, the dual problem is solved. So the constraint in the MKL problem is based on the Ln-norm. This algorithm only solves binary classification problem.

For any problem, please send an email to Shi Yu shi.yu@esat.kuleuven.be

click for free hit counter
Download a free hit counter here.