Distributed optimization on the base of AMPL modeling language and everest platform

Distributed Optimization on the Base of AMPL Modeling Language and Everest Platform doi 10 1016/j procs 2016 11 037 Peer review under responsibility of organizing committee of the scientific committee[.]

Peer-review under responsibility of organizing committee of the scientific committee of the

5th International Young Scientist Conference on Computational Science

© 2016 The Authors Published by Elsevier B.V

Procedia Computer Science 101 , 2016 , Pages 313 – 322

YSC 2016 5th International Young Scientist Conference on Computational Science

Distributed optimization on the base of AMPL modeling

language and Everest platform Sergey Smirnov, Vladimir Voloshinov, and Oleg Sukhoroslov

Institute for Information Transmission Problems of the Russian Academy of Sciences,

Moscow, Russia sasmir@gmail.com, vladimir.voloshinov@gmail.com, sukhoroslov@iitp.ru

Abstract

Optimization modeling in science and industry requires the use of state-of-the-art software and high-performance computing resources A common problem faced by researchers is how

to integrate related software and leverage available computing resources in a distributed envi-ronment The paper presents an approach for solving this problem based on unifying remote access to optimization software via RESTful web services and using AMPL (A Mathematical Programming Language) to describe scenarios of computing with optimization models This approach is implemented in AMPLX toolkit that enables modifying any AMPL script to solve problems by a pool of distributed solvers The toolkit is based on Everest platform that is used

to expose optimization tools as web services and run these tools across distributed resources The proposed approach and AMPLX toolkit have been verified by a number of decomposition algorithms including branch-and-bound algorithm for a special nonlinear optimization problem

Keywords: distributed computing, web services, REST, optimization, algebraic modeling languages

Application of optimization models in science and business requires usage of state-of-the-art software and high-performance computing resources For a decades research community has focused on standalone tools such as solvers and algebraic modeling languages (AML) inter-preters designed for a single computing server Despite the considerable progress in the area of distributed computing since 2000’s, integration of optimization tools into a distributed problem solving environment remains an important challenge and is far from completion The existing solutions are either too complex to be used by an unskilled researcher, or too limiting in ability

to seamlessly incorporate all available computing resources

In this paper we present an approach to deploy optimization systems in a heterogeneous computing infrastructure addressing these problems It is based on providing remote access

to optimization tools via unified RESTful web services [1], flexible mapping of services to distributed computing resources and leveraging AML as a convenient domain-specific language

313

We use AMPL (A Mathematical Programming Language) [2], one of the most popular AMLs familiar to optimization experts

The proposed AMPLX toolkit is an open source extension of AMPL enabling to execute any optimization modeling algorithm that is presented as an AMPL script in a distributed environment The use of AMPLX consists in modifying of AMPL script in accordance with simple rules in order to exchange data with remote solvers via the regular AMPL interpreter If the AMPL script includes a phase of solving a number of independent optimization subproblems they will be solved in parallel by a pool of remote solvers

AMPLX relies on Everest platform [3, 4] to expose optimization tools as RESTful web services and flexibly deploy and run these tools across distributed resources, such as standalone servers, computing clusters and grids The computational performance of AMPLX environment can be modified by adding or removing resources transparently to users In contrast to other AML interpreters, such as GAMS [5] or Xpress-Mosel [6, 7] we do not know any other attempt

to run AMPL in a distributed environment except rather restrictive NEOS-Kestrel utility [8] The paper is structured as follows Section 2 provides an overview of optimization software Section 3 discusses related work Section 4 briefly overviews Everest platform and its relevant features Section 5 presents basics notions of the AMPLX toolkit Section 6 describes example

of AMPLX usage Section 7 concludes and discusses future work

Optimization modeling is an important tool for scientific research for more than 50 years In many cases the solution of optimization problems is not a main goal of researches but is a part of a more complex data processing workflow Therefore researchers have no incentive to spend much time doing low-level programming or tuning of numerical optimization methods In contrast, they need an easy to use, unified approach to solve such problems by reusing existing optimization software, e.g solvers

The heterogeneity of optimization software and available computing infrastructure is one

of the reasons complicating the wide practical usage of optimization models in a distributed computing mode, especially at a pilot phase of research There is a lack of tools that allow researchers to efficiently use existing software in distributed computing infrastructures while keeping the established optimization modeling practice Here we briefly overview common features expected from state-of-the-art solvers and other optimization software

Firstly, it must accept canonical, mathematical programming (MP), formulation of opti-mization problems [9] As to semidefinite, optimal control and other advanced formulations, we

do not touch them here since they often may be reduced to MP-form So, further we consider the following optimization problem (with continuous and integer variables):

f o (p, x) → min

x , x∈R N , x= (xc, xz) , xc∈R Nc, xz∈R Nz

f i (p, x) ≤0 (i∈I) , gj (p, x) =0 (j ∈J) , x∈M(p). (1)

Above: f o (p, x) is an objective function to be minimized with respect to a finite number

of constraints, inequality and equality, represented respectively by two collections of functions

indexed by sets I and J (for inequality and equality constraints) f i (p, x)(i ∈I) and g j (p, x)(j ∈J).

We knowingly distinguish variables (x) and parameters (p) of a problem Parameters p remain unchanged during search of decision variables x This separation of variables and parameters

is very useful in practice As a rule, the preparation of optimization models to pass them to a solver takes a lot of time Moreover, very often researchers should compare optimal solutions for

a number of different sets of parameters (the so called “parameter sweeping”, “what-if” and case studies scenarios) The optimization software should support various classes of optimization

problems (1) based on types of f o , f i and g j functions (on x), e.g linear programming (LP),

quadratic programming (QP), “general” non-linear programming (NLP) and so on

Another typical requirement for optimization software is to support state-of-the-art numeri-cal optimization methods and their “concomitant characters” The established algorithmic and theoretical practice is based on first and second derivatives of problems functions, dual variables

(α i for inequality constraints, β j for equalities) and Lagrange function

∇ x f o (p, x) , ∇ x f i (p, x) , ∇ x g j (p, x) , ∇2

xx f o (p, x) , ∇2

xx f i (p, x) , ∇2

xx g j (p, x) ,

L (α, β, p, x) = f . o (p, x) +

i∈I

α i f i (p, x) + 

j∈J

β j g j (p, x) , ∇ x L ( , x) , ∇2

xx L ( , x) (2)

Note that the above formulations are used even for discrete problems with integer variables E.g the well-known branch-and-bound algorithm for mixed-integer problems uses relaxed sub-problems when some of integer variables are considered as continuous

The correct programming of (1) and (2) for passing to a solver via its low-level API may

be very hard and time consuming, especially for complex optimization models The problem complicates more when solvers are used in a distributed computing environment Fortunately, the state-of-the-art solvers together with algebraic model languages and systems (AML/AMS) [10] provide a convenient workaround to do that The most well-known examples of AMLs are GAMS (General Algebraic Modeling System), AMPL (A Modeling Language for Mathematical Programming), Xpress-Mosel, FICO Xpress Optimization Suite and Zimpl

Except some syntactic differences all these systems have the following common features:

- symbolic notation of optimization problems including indexed sets of parameters, variables, functions, constraints, formulas for objective and constraints functions etc.;

- separation of symbolic notation and data values that define a particular instance of a problem (including numerical values of parameters, multi-dimensional sets of indices, etc.);

- automatic differentiation (for NLP) for the first and second derivatives;

- support of “duality” as an access not only to the values of decision (primal) variables, but also to Lagrange multipliers (dual variables) corresponding to an optimal solution;

- presence of an AML interpreter to run high-level algorithms in the form of AML scripts that perform solution of one or multiple (probably dynamically generated) MP-problems;

- seamless integration with solvers via open API (even for commercial AMLs)

The general scheme of AML usage presented in Figure1 consists of the following steps The user prepares the symbolic description of the problem as a text file (*.mod) and specifies the values of symbolic sets and parameters in a data file (*.dat) Both files are processed by an AML interpreter to create the so called stub file (*.nl in AMPL) representing the problem in

a form readable by AML-compatible solvers An AML-compatible solver returns the problem solution as a file (*.sol) which contains primal/dual variables and may be loaded back by an AML interpreter (e.g for generation of another problems and/or outputs) All these steps and additional processing of input data and solutions may be described in a special AML script which may be executed by an appropriate AML interpreter

In case of Xpress-Mosel [6] the above general scheme is slightly modified: model, data and Mosel script are compiled into a binary executable (*.bim file) to be run by Mosel interpreter The use of AML is crucial at preliminary phases of any R&D project when structure of optimization model, data and general computing scheme are not established and may be chosen before diving into low level programming, e.g because of performance issues

Figure 1: The general scheme of AML usage

The porting of optimization software into distributed computing environments has been ad-dressed by several projects in the last decade In this section we provide a brief overview of these attempts and compare them with AMPLX, presented in Section 5

NEOS Server, https://neos-server.org/, is a free web-based service for solving numerical optimization problems via remote access to almost 60 solvers for various types of problems NEOS provides an XML-RPC based Kestrel utility [8] to send problems to the hosted solvers Kestrel may be used from any correct AMPL script as a blocking remote call that does not allow to efficiently handle a number of independent problems in parallel AMPLX avoids this shortcoming by asynchronous processing of independent problems by a pool of available solvers Moreover, NEOS user is tied to the dedicated infrastructure and can not add new computing resources, solvers, etc In contrast, an administrator of the AMPLX system can add/remove computing resources dedicated for solvers at any time transparently for potential users COIN-OS project [11] proposes a number of specifications for data representation and net-work exchange enabling to deploy distributed system of optimization web services The so called OSxL standards are used to describe optimization problems, their solutions, solver op-tions, optimization services registry etc These standards are based on XML, SOAP and WSDL specifications underlying the so called Big Web Services technologies These technologies be-came not so widely used as they were decade ago due to their complexity and proliferation of RESTful web services The OSxL standards also provide a few capabilities to describe complex (nondeterministic, iterative etc.) algorithms based on optimization models By reusing the established AML, AMPLX surpasses COIN-OS in usability: if user knows AMPL he must not learn more than 15 OSxL dialects to describe optimization models and computing scenarios GAMS Grid [5] is a GAMS extension for distributed computing It targets pools of ma-chines controlled by a single resource managaer such as Condor or Sun Grid Engine Owing to Everest capabilities, AMPLX supports the use of ad-hoc computing infrastructures consisting

of multiple standalone servers, clusters controlled by different batch systems and even resources

of grid infrastructures Moreover, as it can be seen from the NEOS Server access statistics1, GAMS language loses to AMPL on the breadth of usage in the recent years

Pyomo [12] is an open source software package that supports a diverse set of optimization capabilities for formulating, solving, and analyzing optimization models Pyomo provides an

1 http://www.neos-server.org/neos/report.html

open source alternative for AMPL by enabling symbolic description of optimization problems and computing scenarios on Python language involving AMPL-compatible solvers and AMPL data files Pyomo also provides distributed computing capabilities based on Pyro, a Python library for building distributed applications The main disadvantage of Pyomo is much (often

10 times) worse performance when generating AMPL stubs compared to the native AMPL in-terpreter, particularly for large (multi-dimension) mathematical programming problems Some important AMPL features, e.g import of user defined C/C++ functions into models, are cur-rently at a beta release phase Nevertheless, Pyomo can compete with AMPLX if researchers are willing to save some funds by avoiding purchasing of a licensed AMPL interpreter and are ready to migrate from AMPL to the Python language

Mosel-Xpress [6, 13, 7] is a commercial system included in the Xpress Optimization Suite environment developed by FICO company It is the most advanced among similar toolkits for distributed optimization modeling In 2010 FICO presented a new, substantially upgraded version of Mosel language and interpreter that, among other features, provided native support for distributed computing Users can program distributed computing algorithms based on the event driven programming paradigm directly in the high-level Mosel language In contrast to other interpreted AMLs, Mosel-Xpress has to be compiled into intermediate byte-code running

by a special virtual machine The system supports two protocols for exchanging data between parallel computing activities: SHMEM (shared memory) and MEMPIPE (inter processor) Both protocols are low-level and require a homogeneous computing infrastructure such as a server or a computing cluster FICO provides Mosel-Xpress in a form of a private cloud service deployed on the internal computing infrastructure In contrast to Mosel-Xpress, AMPLX is open source and supports a wider range of ad-hoc computing infrastructures

AMPLX is based on Everest [3, 4], a web-based platform for building computational web services This section contains a brief overview of this platform

Everest provides users with tools to quickly publish and share computing applications as services The platform also manages execution of applications on external computing resources attached by users In contrast to traditional distributed computing platforms, Everest imple-ments the Platform as a Service (PaaS) model by providing its functionality via remote web and programming interfaces A single instance of the platform can be accessed by many users in order to create, run and share applications with each other The platform is publicly available online to all interested users [4]

Applications are the core entities in Everest that represent reusable computational units

that follow a well-defined model An application has a number of inputs that constitute a valid request to the application and a number of outputs that constitute a result of computation corresponding to some request Upon each request Everest creates a new job consisting of one

or more computational tasks generated by the application according to the job inputs The tasks

are executed by the platform on computing resources specified by a user The dependencies between tasks are currently managed internally by applications The results of completed tasks are passed back to the application and are used to produce job outputs or new tasks if needed The job is completed when there are no incomplete tasks are left The described model is generic enough to support a wide range of computing applications

Users can publish applications via provided generic application template that makes it possible to avoid programming The template supports running arbitrary applications with command-line interface and produces a single task corresponding to a single command run

Recent developments made it possible to dynamically add new tasks or invoke other applications from a running application via the Everest API This enabled users to create and publish complex many-task applications such as workflows or AMPLX toolkit presented in this paper Everest also provides a built-in generic application for running parameter sweep experiments

An application is automatically published as a RESTful web service [1] This enables pro-grammatic access to applications, integration with third-party tools and composition of appli-cations into workflows The platform’s web user interface also generates a web form for running

an application via web browser An application owner can manage the list of users that are allowed to run the application

Instead of using internal computing infrastructure, Everest performs execution of application tasks on external resources Users can attach computing resources to the platform and flexibly bind these resources to applications The platform implements integration with standalone

machines and clusters by using a developed program called agent The agent runs on the

resource and acts as a mediator between it and Everest enabling the platform to submit and manage computations on the resource The platform also supports integration with other types

of resources, such as the European Grid Infrastructure

Everest manages execution of tasks on remote resources and performs routine actions related

to staging of task input files, submitting a task, monitoring a task state and downloading task results The platform also monitors the state of computing resources and uses this information during job scheduling It is possible to run an application on multiple resources In this case the platform performs dynamic scheduling of application tasks across the specified resources From this point of view Everest can be seen as a multitenant metascheduling service

The goal of AMPLX development is to run any AMPL script by the standard interpreter communicating with remote solvers installed in distributed computing environment Namelly: 1) all MP problems either predefined or dynamically composed during running of the script will be sent to remote solvers; 2) set of independent subproblems will be solved in parallel by

a pool of computing resources whose configuration might be changed transparently for users Besides software components AMPLX defines a set of rules for modifying an AMPL script in accordance with AMPLX requirements

Figure 2 illustrates AMPLX architecture and principles of its functioning (regardless of com-puting workflow) AMPLX script is executed by the standard AMPL interpreter AMPLX in-cludes a number of small files amplx_*.amp (a kind of macroses) containing AMPL operators for: setting a few AMPLX parameters ( amplx_*); writing AMPL stubs for a set of optimization problems; interacting with dedicated Everest application solve-ampl-stub via Everest Python API This application acts as a front-end for a pool of solvers (SCIP, COIN-CBC, IPOPT etc.) covering basic types of mathematical programming problems (LP/MILP/NLP/MINLP) These solvers are installed on heterogeneous resources (standalone servers, clusters, clouds etc.) at-tached to the solve-ampl-stub by the Everest agent

Figure 3 provides an example of AMPLX use On top of the figure excerpts of some AMPL script are presented At the beginning solver (IPOPT here) is chosen, then after some data preparation there is a for-loop where a set of independent subproblems are being prepared and solved in turn Below the same excerpts modified in accordance with AMPLX rules are shown First of all, an initialization AMPLX macros amplx_globals.amp is called This macros declares a set of parameters that will be used by AMPLX For example, amplx_solver and amplx_solver_opts_file are used to set solver type and solver’s options file passed

Figure 2: AMPLX architecture

Figure 3: Example of AMPL to AMPLX conversion

to the solve-ampl-stub by Python subsystem of AMPLX Next, the original loop is replaced with three parts: 1) loop for creating stub files for the set of subproblems (without solv-ing) by using amplx_probName parameter, set of subproblem names amplx_probSet and amplx_write_problem_stub.amp macros; 2) passing of created stubs to the solve-ampl-stub application for parallel solving (results of optimization will be returned from solvers and written

to the local folder as a set of *.sol files); 3) processing of solutions in the same way as has been done in the original loop

The source code of AMPLX toolkit is available at http://gitlab.com/ssmir/amplx The system may be used either directly at the host with installed AMPL or remotely via run-amplx Everest application, https://everest.distcomp.org The demo AMPLX scripts for Danzig-Wolfe decomposition can be found at http://distcomp.ru/~vladimirv/restopt/amplx/dw/

Figure 4: X-Ray diffraction (on nuclei and

electronic shells) Experiment and modeling

(solid line)

Figure 5: Neutron diffraction (on nuclei only) Experiment and modeling (solid line)

The toolkit have been already used to solve several real optimization problems In this section

concern one of such problems related to Carbonaceous Structure Identification as Inverse

Opti-mization Problem An algorithm has been developed for structure identification of amorphous

carbonaceous nanomaterials with a joint x-ray and neutron diffraction data analysis, using the data on the chemical composition of the sample from other diagnostics [14, 15] Optimization criteria are to minimize a gap between experimental and model data for X-Ray and neutron diffractometries (we use three independent criteria):

j=1 zXr

j + (1−K)n

k=1

zNe

k  −−−−−−−−−−−−−→

(zXr,zNe,x,y,a,A,c,B) min (criterion L1) ;

j=1



zXr

j

2 + (1−K) n

k=1



zNe

k

2

−−−−−−−−−−−−−→

(zXr,zNe,x,y,a,A,c,B) min (criterion L2) ;

K max

j=1:m zXr

j + (1−K) max

k=1:n zNe

k  −−−−−−−−−−−−−→

(zXr,zNe,x,y,a,A,c,B) min (criterion L ∞) ;−

(3)

with the following common constraints:

zXr

j = SXrexp(q i)−N

i=1

x i ·SXr

i (q j)− a·SXr

C (q j)− A·SXr

impur(q j ) (j = 1 : m),

zNe

k = SexpNe(q k −N

i=1 y i·SNe

i (q k − c·SNe

C (q k − B·SNe

impur(q k ) (k = 1 : n), N



i=1

x i + a = A, N

i=1

y i + c = B, x i

A = y B i (i=1 : N ), A a =B c (x i , y i , a, A, c, B)  0

(4)

Let’s comment (3, 4) without going into details (see [14] for them) Input data are as follows:

SXr

exp(q i ) and SexpNe(q k) are the experimental X-ray and neutron scattering intensities, respectively

(see Figures 4, 5); j and k are indices of discrete values of scattering vector modulus q for X-ray and neutron data; m and n are the total numbers of scattering vectors for X-X-ray and neutron data; N is the total number of nanostructures selected for optimization; S iXr(q j) and

SNe

i (q k ) are X-ray and neutron model scattering intensities curves calculated for the i-th carbon

Figure 6: Successive solution of L1, L2, L ∞ Figure 7: Concurrent solution of L1, L2, L ∞

nanostructure; parameters S CXr(q j ), S CNe(q k ), SimpurXr (q j ) and SimpurNe (q k) are similar ones relating

to the amorphous carbon medium (C) and to all the impurities in the samples

Presence of last equalities in (4) (bilinear by variables) makes all three problems (L1, L2, L ∞) non-convex Therefore a direct application of nonlinear local solver, e.g IPOPT, may return solution that is not a global optimum It was the reason of developing a special branch-and-bound algorithm to find an approximate solution of three problems with predefined accuracy (relative gap to unknown optimal objective value) This algorithm has been implemented as an AMPLX script involving the use of NLP and LP solvers to solve auxiliary bound subproblems with relaxed to linear last equalities in (4) The algorithm has been published in https: //everest.distcomp.org/apps as the optBnB_Nlp-2-amplx

Performance of distributed computing scenarios depends mainly on that how subtasks are dispatched over available processors Everest Python API enables to analyze quality of that dispatching We provided AMPLX with a kind of trace visualization tool (based on Everest Python API and http://jkff.info/software/timeplotters/ This tool helps to evaluate effectiveness of a scenario and to conceive possible improvement Returning to the problem

of this section, Figure 6 shows an execution trace of AMPLX run corresponding to the case when all three optimization problems are solved in turn The execution of approximately 60 subtasks took 25 minutes, while no more than 8 solvers of 16 available ones worked in parallel Figure 7 shows a similar trace for a run corresponding to a simultaneous execution of three

algorithm sessions for L1, L2, L ∞ respectively In this case, due to a more efficient use of available resources, all three jobs took about 10 minutes with 13 solvers run in parallel

We have presented AMPLX, an open source extension of AMPL enabling the execution of any optimization modeling algorithm presented as an AMPL script in a distributed environment

It became a promising tool that is already using for development of decomposition, many-task, algorithms for global and discrete optimization problems For example, it is used for intensive testing of local elimination algorithm for discrete optimiztion problems with sparse quasi-block constraints structure [16] Every execution of the algorithm for problems with 100 variables and from 50000 to 100000 constraints leads to solving of about 1000 MILP subproblems

As to AMPLX services deployed on Everest platform, we plan to significantly increase the capacity of computing infrastructure associated with solve-ampl-stub application Future

work will also include the use of recent Everest developments to implement an exchange of information between solvers running within an AMPLX session

We are also investigating the development of a fully open source solution similar to AMPLX

on the base on Pyomo [12] Though Pyomo generates AMPL stubs slower than the native AMPL interpreter, it allows to parallelize the process of stub generation which is rather time consuming for large optimization problems

Acknowledgments

This work is supported by the Russian Science Foundation (project No 16-11-10352)

References

[1] Leonard Richardson and Sam Ruby RESTful web services ” O’Reilly Media, Inc.”, 2008 [2] Robert Fourer, David M Gay, and Brian W Kernighan AMPL: A mathematical programming language Thomson/Brooks/Cole, 2003.

[3] O Sukhoroslov, S Volkov, and A Afanasiev A web-based platform for publication and distributed

execution of computing applications In Parallel and Distributed Computing (ISPDC), 2015 14th International Symposium on, pages 175–184, June 2015.

[4] Everest [online] http://everest.distcomp.org/

[5] Michael R Bussieck, Michael C Ferris, and Alexander Meeraus Grid-enabled optimization with

gams INFORMS Journal on Computing, 21(3):349–362, 2009.

[6] Xpress-mosel user guide [online] http://www.fico.com/en/wp-content/secure_upload/ Xpress-Mosel-User-Guide.pdf

[7] Y Colombani and S Heipcke Multiple models and parallel solving with mosel xpress optimization suite white paper Technical report, FICO, 2014 http://www.fico.com/en/latest-thinking/ white-papers/multiple-models-and-parallel-solving-with-mosel

[8] Elizabeth D Dolan, Robert Fourer, Jean-Pierre Goux, Todd S Munson, and Jason Sarich Kestrel:

An interface from optimization modeling systems to the neos server INFORMS Journal on Com-puting, 20(4):525–538, 2008.

[9] C.A Floudas and P M Pardalos (Eds.) Encyclopedia of Optimization 2-nd Edition Springer &

Business Media, 2009

[10] J Kallrath (Ed.) Algebraic Modeling Systems: Modeling and Solving Real World Optimization Problems, volume 104 Applied Optimization, Springer Science & Business Media, 2012.

[11] Robert Fourer, Jun Ma, and Kipp Martin Optimization services: A framework for distributed

optimization Operations Research, 58(6):1624–1636, 2010.

[12] W.E Hart, C Laird, J.-P Watson, and D.L Woodruff Pyomo–optimization modeling in python,

volume 67 Springer, 2012

[13] Susanne Heipcke Xpress–mosel:multi–solver, multi–problem, multi–model, multi–node modeling

and problem solving In Algebraic Modeling Systems, volume 104, pages 77–110 Springer, 2012.

[14] V.S Neverov, V.V Voloshinov, A.B Kukushkin, and A.S Tarasov Algorithm of amorphous carbonaceous nanomaterial structure identification with a joint x-ray and neutron diffraction data analysis, 2014 http://arxiv.org/abs/1301.3418

[15] A Kukushkin, V Neverov, N Marusov, I Semenov, B Kolbasov, V Voloshinov, A Afanasiev,

A Tarasov, et al Few-nanometer-wide carbon toroids in the hydrocarbon films deposited in

tokamak t-10 Chemical Physics Letters, 506(4):265–268, 2011.

[16] Oleg Shcherbina Graph-based local elimination algorithms in discrete optimization In Founda-tions of Computational Intelligence Volume 3, pages 235–266 Springer, 2009.