﻿ 基于特权信息的SVM模型研究及应用

# 基于特权信息的SVM模型研究及应用Research and Application of SVM Model Based on Privileged Information

Abstract: When using Support Vector Machine (SVM) to training classification model, we may encounter ad-ditional information in training samples. Because it will seriously affect classification accuracy of test samples, it really can’t be ignored. In this paper, a SVM model based on privilege information is proposed, which contains many cases of privilege information and can solve many kinds of dis-tribution problems effectively. This paper first described the basic principles of SVM classification, and then it introduced the concept of privilege information. Next this paper proposed a SVM model based on privilege information and gave its special cases. Then, the application of SVM model based on privilege information was introduced. Finally, the existing problems and the development direction of this research were summarized.

1. 引言

2. 支持向量机(SVM)分类原理

SVM对数据的学习是把分类问题转化为一个有约束的二次规划问题进行求解，得到最优解以构造最优决策分类超平面来实现分类 [6] 。以二分类样本集的学习问题为例，已知样本集为 $\left\{\left({x}_{i},{y}_{i}\right),i=1,2,\cdots ,l\right\}$ ，其中 ${x}_{i}\in {R}^{l}$ 表示输入变量， ${y}_{i}\in \left\{-1,1\right\}$ 表示输出变量， $l$ 为样本个数。则得到SVM模型为

$\begin{array}{l}\underset{\omega ,b,\xi }{\mathrm{min}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\frac{1}{2}{‖w‖}^{2}+C\underset{i=1}{\overset{l}{\sum }}{\xi }_{i}\\ \text{ }\text{s}\text{.t}\text{.}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{y}_{i}\left(\left(w\cdot \varphi \left({x}_{i}\right)\right)+b\right)\ge 1-{\xi }_{i}\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{\xi }_{i}\ge 0,i=1,\cdots ,l\end{array}$

$\begin{array}{l}\underset{\alpha }{\mathrm{max}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\underset{i=1}{\overset{l}{\sum }}{\alpha }_{i}-\frac{1}{2}\underset{i=1}{\overset{l}{\sum }}\underset{j=1}{\overset{l}{\sum }}{\alpha }_{i}{\alpha }_{j}{y}_{i}{y}_{j}K\left({x}_{i},{x}_{j}\right)\\ \text{\hspace{0.17em}}\text{s}\text{.t}\text{.}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\underset{i=1}{\overset{l}{\sum }}{y}_{i}{\alpha }_{i}=0\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }0\le {\alpha }_{i}\le C,i=1,\cdots ,l\end{array}$

${b}^{\ast }={y}_{j}-\underset{i=1}{\overset{l}{\sum }}{y}_{i}{\alpha }_{i}^{\ast }K\left({x}_{i},{x}_{j}\right),\forall j\in \left\{j|{\alpha }_{j}^{\ast }>0\right\}$

$f\left(x\right)=\mathrm{sgn}\left\{\underset{i=1}{\overset{l}{\sum }}{\alpha }_{i}^{\ast }{y}_{i}K\left({x}_{i},x\right)+{b}^{\ast }\right\}$

3. 具有特权信息的SVM模型

3.1. 基于特权信息的监督学习

3.2. 具有特权信息的SVM模型及其特例

3.2.1. 具有特权信息的SVM模型

(1) 其中 $n\left(n\in {N}^{+}\right)$ 个样本无特权信息，其余 $\left(l-n\right)$ 个样本具有特权信息；

(2) 具有特权信息的 $\left(l-n\right)$ 个样本来自于 $t\left(t\in {N}^{+}\right)$ 个不同的特权空间。

$\begin{array}{l}\underset{w,{w}^{1},\cdots ,{w}^{t},b,{b}^{1},\cdots ,{b}^{t},\xi ,{\xi }^{1},\cdots ,{\xi }^{t}}{\mathrm{min}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\frac{1}{2}\left[\left(w\cdot w\right)+\gamma \left(\left({w}^{1}\cdot {w}^{1}\right)+\cdot \cdot \cdot +\left({w}^{j}\cdot {w}^{j}\right)+\cdot \cdot \cdot +\left({w}^{t}\cdot {w}^{t}\right)\right)\right]+C\underset{i=1}{\overset{n}{\sum }}{\xi }_{i}\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }+{C}_{1}\underset{n+1}{\overset{m}{\sum }}\left[\left({w}^{1}\cdot {z}_{i}^{1}\right)+{b}^{1}\right]+{\theta }_{1}{C}_{1}\underset{i=n+1}{\overset{m}{\sum }}{\xi }_{i}^{1}+\cdot \cdot \cdot +{C}_{j}\underset{A}{\overset{B}{\sum }}\left[\left({w}^{j}\cdot {z}_{i}^{j}\right)+{b}^{j}\right]\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }+{\theta }_{j}{C}_{j}\underset{i=A}{\overset{B}{\sum }}{\xi }_{i}^{j}+\cdot \cdot \cdot +{C}_{k}\underset{d}{\overset{l}{\sum }}\left[\left({w}^{t}\cdot {z}_{i}^{t}\right)+{b}^{t}\right]+{\theta }_{k}{C}_{k}\underset{i=d}{\overset{l}{\sum }}{\xi }_{i}^{t}\end{array}$

$\begin{array}{l}\text{ }\text{ }\text{s}\text{.t}\text{.}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{y}_{i}\left[\left(w\cdot {z}_{i}\right)+b\right]\ge 1-{\xi }_{i},i=1,\cdot \cdot \cdot ,n\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{y}_{i}\left[\left(w\cdot {z}_{i}\right)+b\right]\ge 1-\left[\left({w}^{j}\cdot {z}_{i}^{j}\right)+{b}^{j}+{\xi }_{i}^{j}\right],j=1,2,\cdot \cdot \cdot ,t\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }{\xi }_{i}\ge 0,i=1,\cdot \cdot \cdot ,n\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }{\xi }_{i}^{1},{\xi }_{i}^{2},\cdot \cdot \cdot ,{\xi }_{i}^{t}\ge 0\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\left({w}^{j}\cdot {z}_{i}^{j}\right)+{b}^{j}\ge 0,j=1,2,\cdot \cdot \cdot ,t\end{array}$

$\begin{array}{l}\underset{\alpha ,\beta }{\mathrm{max}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\underset{i=1}{\overset{l}{\sum }}{\alpha }_{i}-\frac{1}{2}\underset{i,k=1}{\overset{l}{\sum }}{\alpha }_{i}{\alpha }_{k}{y}_{i}{y}_{k}K\left({x}_{i},{x}_{k}\right)\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }-\frac{1}{2\gamma }\underset{i,k=n+1}{\overset{m}{\sum }}\left({\alpha }_{i}+{\beta }_{i}-{C}_{1}\right)\left({\alpha }_{k}+{\beta }_{k}-{C}_{1}\right){K}^{1}\left({x}_{i}^{1},{x}_{k}^{1}\right)\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }-\cdot \cdot \cdot -\frac{1}{2\gamma }\underset{i,k=A}{\overset{B}{\sum }}\left({\alpha }_{i}+{\beta }_{i}-{C}_{j}\right)\left({\alpha }_{k}+{\beta }_{k}-{C}_{j}\right){K}^{j}\left({x}_{i}^{j},{x}_{k}^{j}\right)\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\cdot -\cdot \cdot -\frac{1}{2\gamma }\underset{i,k=d}{\overset{l}{\sum }}\left({\alpha }_{i}+{\beta }_{i}-{C}_{t}\right)\left({\alpha }_{k}+{\beta }_{k}-{C}_{t}\right){K}^{t}\left({x}_{k}^{t},{x}_{k}^{t}\right)\end{array}$

$\begin{array}{l}\text{s}\text{.t}\text{.}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\underset{i=1}{\overset{l}{\sum }}{\alpha }_{i}{y}_{i}=0\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\underset{i=1}{\overset{n}{\sum }}\left({\alpha }_{i}+{\beta }_{i}-C\right)=0\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\underset{i=n+1}{\overset{l}{\sum }}\left({\alpha }_{j}+{\beta }_{j}-{C}_{j}\right)=0,j=1,\cdot \cdot \cdot ,t;\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }0\le {\alpha }_{i}\le {\theta }_{j}{C}_{j},i=1,\cdot \cdot \cdot ,l;j=1,\cdot \cdot \cdot ,t\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{\hspace{0.17em}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{\alpha }_{i}\ge 0,{\beta }_{i}\ge 0\end{array}$

$f\left(x\right)=\left(w\cdot z\right)+b=\underset{i=1}{\overset{l}{\sum }}{\alpha }_{i}{y}_{i}K\left({x}_{i},x\right)$

3.2.2. 具有特权信息的SVM模型特例

1) 全部样本存在特权信息的SVM模型

$\begin{array}{l}\underset{\omega ,{\omega }^{\ast },b,{b}^{\ast }}{\mathrm{min}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\frac{1}{2}\left[\left(w\cdot w\right)+\gamma \left({w}^{\ast }\cdot {w}^{\ast }\right)\right]+C\underset{i=1}{\overset{l}{\sum }}\left[\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\right]\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{s}\text{.t}.\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{y}_{i}\left[\left(w\cdot {z}_{i}\right)+b\right]\ge 1-\left[\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\right],i=1,\cdots ,l\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\ge 0,i=1,\cdots ,l\end{array}$

2) 全部样本存在特权信息且松弛变量改动的SVM模型

${\xi }_{i}=\left[\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\right]+{\xi }_{i}^{\ast },i=1,\cdots ,l$

$\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\ge 0,{\xi }_{i}^{\ast }\ge 0,i=1,\cdots ,l$

$\begin{array}{l}\underset{\omega ,{\omega }^{\ast },b,{b}^{\ast },{\xi }^{\ast }}{\mathrm{min}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\frac{1}{2}\left[\left(w\cdot w\right)+\gamma \left({w}^{\ast }\cdot {w}^{\ast }\right)\right]+C\underset{i=1}{\overset{l}{\sum }}\left[\left({w}^{\ast }\cdot {z}^{\ast }\right)+{b}^{\ast }\right]+\theta C\underset{i=1}{\overset{l}{\sum }}{\xi }_{i}^{\ast }\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{s}\text{.t}\text{.}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{y}_{i}\left[\left(w\cdot {z}_{i}\right)+b\right]\ge 1-\left[\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\right]-{\xi }_{i}^{\ast }\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\ge 0\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{\xi }_{i}^{\ast }\ge 0\end{array}$

3) 只有部分训练样本存在特权信息的SVM模型

$\begin{array}{l}\underset{\omega ,{\omega }^{\ast },b,{b}^{\ast }}{\mathrm{min}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\frac{1}{2}\left[\left(w\cdot w\right)+\gamma \left({w}^{\ast }\cdot {w}^{\ast }\right)\right]+C\underset{i=1}{\overset{n}{\sum }}{\xi }_{i}+{C}^{\ast }\underset{i=n+1}{\overset{l}{\sum }}\left[\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\right]\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{s}\text{.t}\text{.}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{y}_{i}\left[\left(w\cdot {z}_{i}\right)+b\right]\ge 1-{\xi }_{i},i=1,\cdots ,n\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{y}_{i}\left[\left(w\cdot {z}_{i}\right)+b\right]\ge 1-\left[\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\right],i=n+1,\cdots ,l\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{\xi }_{i}\ge 0,i=1,\cdots ,n\text{ }\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\ge 0,i=n+1,\cdots ,l\text{ }\text{ }\text{ }\end{array}$

4) 来自多空间特权信息的SVM模型

$\begin{array}{l}\underset{\omega ,{\omega }^{\ast },{\omega }^{\ast \ast }，b,{b}^{\ast },{b}^{\ast \ast }}{\mathrm{min}}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\frac{1}{2}\left[\left(w\cdot w\right)+\gamma \left(\left({w}^{\ast }\cdot {w}^{\ast }\right)+\left({w}^{\ast \ast }\cdot {w}^{\ast \ast }\right)\right)\right]\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }+C\underset{i=1}{\overset{n}{\sum }}\left[\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\right]+{C}^{\ast }\underset{i=n+1}{\overset{l}{\sum }}\left[\left({w}^{\ast \ast }\cdot {z}_{i}^{\ast \ast }\right)+{b}^{\ast \ast }\right]\end{array}$

$\begin{array}{l}\text{s}\text{.t}\text{.}\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{y}_{i}\left[\left(w\cdot {z}_{i}\right)+b\right]\ge 1-\left[\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\right],i=1,\cdots ,n\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }{y}_{i}\left[\left(w\cdot {z}_{i}\right)+b\right]\ge 1-\left[\left({w}^{\ast \ast }\cdot {z}_{i}^{\ast \ast }\right)+{b}^{\ast \ast }\right],i=n+1,\cdots ,l\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\left({w}^{\ast }\cdot {z}_{i}^{\ast }\right)+{b}^{\ast }\ge 0,i=1,\cdots ,n\text{ }\\ \text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\text{ }\left({w}^{\ast \ast }\cdot {z}_{i}^{\ast \ast }\right)+{b}^{\ast \ast }\ge 0,i=n+1,\cdots ,l\end{array}$

4. 具有特权信息的SVM模型应用

1) 具有特权信息的SVM模型在恶意软件检测中的应用

Burnaev提供了改进分类问题的方法即允许合并特权信息。从实验结果可以看出，在某些情况下，特权信息可以显著提高异常检测的准确性。在特权信息对于相关问题的结构没有用的情况下，特权信息不会对分类功能产生重大影响 [10] 。

2) 基于特权信息的SVM模型在高级学习范式中的应用

3) 基于特权信息的SVM模型与经验风险最小化算法的综合应用

5. 总结与讨论

(1) SVM算法的核心是核函数及其参数，它们的正确选取对SVM的预测及泛化性能影响很大 [16] 。对于具体问题，基于特权信息的SVM模型究竟选择哪种核函数并找到最优的参数对求解问题至关重要。因此，如何快速准确地选择核函数及对应的参数是亟待解决的问题。

(2) 在大规模及实时性要求较高的系统中，基于特权信息的SVM算法受制于求解问题的收敛速度和系统规模的复杂程度。尤其要处理大规模数据时，基于特权信息的SVM算法需要解决样本规模和速度间的矛盾，提高训练的效率和精度。

(3) 如何有效地将二分类有效地扩展到多分类问题上，基于特权信息的多分类SVM模型的优化设计也是今后研究的内容。

(4) 针对特定问题如何实现基于特权信息的SVM模型与其他算法的融合，从而顺利地解决问题也是今后需要研究的方向。

[1] Vapnik, V.N. (1998) Statistical Learning Theory. John Wiley & Sons, Hoboken.

[2] 熊平. 数据挖掘算法与Clementine实践[M]. 北京: 清华大学出版社, 2011.

[3] Bishop, C.M. (2006) Pattern Recognition and Machine Learning. Springer, Berlin.

[4] 朱明. 数据挖掘[M]. 合肥: 中国科学技术大学出版社, 2002.

[5] Burnaev, E. and Smolyakov, D. (2016) One-Class SVM with Privileged Information and Its Application to Malware Detection. ArXiv, Los Alamos.

[6] Gine, E. and Koltchinskii, V. (2006) Concentration Inequalities and Asymptotic Results for Ratio Type Empirical Processes. Annals of Probability, 34, 1143-1216.
https://doi.org/10.1214/009117906000000070

[7] 邓乃扬, 田英杰. 数据挖掘的新方法—支持向量机[M]. 北京: 科学出版社, 2004.

[8] Manevitz, L.M. and Yousef, M. (2001) One-Class SVMs for Document Classification. Journal of Machine Learning Research, 2, 139-154.

[9] Vapnik, V. and Vashist, A. (2009) A New Learning Paradigm: Learning Using Privileged Information. Neural Networks, 22, 544-557.
https://doi.org/10.1016/j.neunet.2009.06.042

[10] Boser, G. (1992) A Training Algorithm for Optimal Margin Classifiers. Proceedings of the Fifth Annual Workshop on Computational Learning Theory, Pittsburgh, 27-29 July 1992, 144-152.
https://doi.org/10.1145/130385.130401

[11] Cristianini, N., Shawe-Taylor, J. 支持向量机导论[M]. 李国正, 王猛, 曾华军, 译. 北京: 电子工业出版社, 2004.

[12] Pechyony, D. and Vapnik, V. On the Theory of Learning with Privileged Information. John Wiley & Sons, Hoboken.

[13] Boucheron, S., Bousquet, O. and Lugosi, G. (2005) Theory of Classification: A Survey of Some Recent Advances. ESAIM: Probability and Statistics, 9, 329-375.
https://doi.org/10.1051/ps:2005018

[14] Cortes, C. and Vapnik, V. (1995) Support-Vector Networks. Machine Learning, 20, 273-297.
https://doi.org/10.1007/BF00994018

[15] Schölkopf, B., Smola, A.J., Williamson, R.C. and Bartlett, P.L. (2000) New Support Vector Algorithms. Neural Computation, 12, 1207-1245.
https://doi.org/10.1162/089976600300015565

[16] Micchelli, C. and Pontil, M. (2005) Learning the Kernel Function via Regularization. Journal of Machine Learning Research, 6, 1099-1125.

Top