pystruct.learners.FrankWolfeSSVM¶

class pystruct.learners.FrankWolfeSSVM(model, max_iter=1000, C=1.0, verbose=0, n_jobs=1, show_loss_every=0, logger=None, batch_mode=False, line_search=True, check_dual_every=10, tol=0.001, do_averaging=True, sample_method='perm', random_state=None)[source]¶

Structured SVM solver using Block-coordinate Frank-Wolfe.

This implementation is somewhat experimental. Use with care.

Parameters:

Parameters:	model : StructuredModel Object containing the model structure. Has to implement loss, inference and loss_augmented_inference. max_iter : int, default=1000 Maximum number of passes over dataset to find constraints. C : float, default=1 Regularization parameter. Corresponds to 1 / (lambda * n_samples). verbose : int Verbosity. n_jobs : int, default=1 Number of parallel processes. Currently only n_jobs=1 is supported. show_loss_every : int, default=0 How often the training set loss should be computed. Zero corresponds to never. tol : float, default=1e-3 Convergence tolerance on the duality gap. logger : logger object, default=None Pystruct logger for storing the model or extracting additional information. batch_mode : boolean, default=False Whether to use batch updates. Will slow down learning enormously. line_search : boolean, default=True Whether to compute the optimum step size in each step. The line-search is done in closed form and cheap. There is usually no reason to turn this off. check_dual_every : int, default=10 How often the stopping criterion should be checked. Computing the stopping criterion is as costly as doing one pass over the dataset, so check_dual_every=1 will make learning twice as slow. do_averaging : bool, default=True Whether to use weight averaging as described in the reference paper. Currently this is only supported in the block-coordinate version. random_state : int, RandomState instance or None, optional (default=None) If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by np.random.
Attributes:	w : nd-array, shape=(model.size_joint_feature,) The learned weights of the SVM. ``loss_curve_`` : list of float List of loss values if show_loss_every > 0. ``objective_curve_`` : list of float Cutting plane objective after each pass through the dataset. ``primal_objective_curve_`` : list of float Primal objective after each pass through the dataset. ``timestamps_`` : list of int Total training time stored before each iteration.

model : StructuredModel

Object containing the model structure. Has to implement loss, inference and loss_augmented_inference.

max_iter : int, default=1000

Maximum number of passes over dataset to find constraints.

C : float, default=1

Regularization parameter. Corresponds to 1 / (lambda * n_samples).

verbose : int

Verbosity.

n_jobs : int, default=1

Number of parallel processes. Currently only n_jobs=1 is supported.

show_loss_every : int, default=0

How often the training set loss should be computed. Zero corresponds to never.

tol : float, default=1e-3

Convergence tolerance on the duality gap.

logger : logger object, default=None

Pystruct logger for storing the model or extracting additional information.

batch_mode : boolean, default=False

Whether to use batch updates. Will slow down learning enormously.

line_search : boolean, default=True

Whether to compute the optimum step size in each step. The line-search is done in closed form and cheap. There is usually no reason to turn this off.

check_dual_every : int, default=10

How often the stopping criterion should be checked. Computing the stopping criterion is as costly as doing one pass over the dataset, so check_dual_every=1 will make learning twice as slow.

do_averaging : bool, default=True

Whether to use weight averaging as described in the reference paper. Currently this is only supported in the block-coordinate version.

random_state : int, RandomState instance or None, optional (default=None)

If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by np.random.

Attributes:

w : nd-array, shape=(model.size_joint_feature,)

The learned weights of the SVM.

``loss_curve_`` : list of float

List of loss values if show_loss_every > 0.

``objective_curve_`` : list of float

Cutting plane objective after each pass through the dataset.

``primal_objective_curve_`` : list of float

Primal objective after each pass through the dataset.

``timestamps_`` : list of int

Total training time stored before each iteration.

References

Lacoste-Julien, Jaggi, Schmidt, Pletscher:

Block-Coordinage Frank-Wolfe Optimization for Structural SVMs,xi JMLR 2013

With batch_mode=False, this implements the online (block-coordinate) version of the algorithm (BCFW) BCFW is an attractive alternative to subgradient methods, as no learning rate is needed and a duality gap guarantee is given.

Methods

`fit`(X, Y[, constraints, initialize])	Learn parameters using (block-coordinate) Frank-Wolfe learning.
`get_params`([deep])	Get parameters for this estimator.
`predict`(X)	Predict output on examples in X.
`score`(X, Y)	Compute score as 1 - loss over whole data set.
`set_params`(**params)	Set the parameters of this estimator.

__init__(model, max_iter=1000, C=1.0, verbose=0, n_jobs=1, show_loss_every=0, logger=None, batch_mode=False, line_search=True, check_dual_every=10, tol=0.001, do_averaging=True, sample_method='perm', random_state=None)[source]¶

fit(X, Y, constraints=None, initialize=True)[source]¶

Learn parameters using (block-coordinate) Frank-Wolfe learning.

Parameters:

Parameters:	X : iterable Traing instances. Contains the structured input objects. No requirement on the particular form of entries of X is made. Y : iterable Training labels. Contains the strctured labels for inputs in X. Needs to have the same length as X. contraints : ignored initialize : boolean, default=True Whether to initialize the model for the data. Leave this true except if you really know what you are doing.

X : iterable

Traing instances. Contains the structured input objects. No requirement on the particular form of entries of X is made.

Y : iterable

Training labels. Contains the strctured labels for inputs in X. Needs to have the same length as X.

contraints : ignored

initialize : boolean, default=True

Whether to initialize the model for the data. Leave this true except if you really know what you are doing.

get_params(deep=True)¶

Get parameters for this estimator.

Parameters:

Parameters:	deep: boolean, optional : If True, will return the parameters for this estimator and contained subobjects that are estimators.
Returns:	params : mapping of string to any Parameter names mapped to their values.

deep: boolean, optional :

If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params : mapping of string to any

Parameter names mapped to their values.

predict(X)¶

Predict output on examples in X.

Parameters:

Parameters:	X : iterable Traing instances. Contains the structured input objects.
Returns:	Y_pred : list List of inference results for X using the learned parameters.

X : iterable

Traing instances. Contains the structured input objects.

Returns:

Y_pred : list

List of inference results for X using the learned parameters.

score(X, Y)¶

Compute score as 1 - loss over whole data set.

Returns the average accuracy (in terms of model.loss) over X and Y.

Parameters:

Parameters:	X : iterable Evaluation data. Y : iterable True labels.
Returns:	score : float Average of 1 - loss over training examples.

X : iterable

Evaluation data.

Y : iterable

True labels.

Returns:

score : float

Average of 1 - loss over training examples.

set_params(**params)¶

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as pipelines). The former have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Returns:	self :