Proposal for pfunc Function Interface [DONE]¶
Note
This proposal was implemented some time around summer 2009, and merged into the trunk around new years 2010.
Following discussion on theano-dev (titled TheanoObject), the following changes are proposed to make function-construction calls more readable and intuitive, and to make it easier to share values between functions.
The strategy is to
- introduce a new kind of
Variable
(SharedVariable
) that has a container associated with it, and can allow multiple functions to share a value. - introduce a class called
Param
to serve a role similar to that ofIn
, - introduce a friendlier version of function (tentative name
pfunc
),
The following code gives a very quick idea of what is being proposed:
..code-block:: python
a = lscalar() b = shared(1) #NEW: create a shared variable
f1 = pfunc([a], a+b) f2 = pfunc([Param(a, default=44)], a + b, updates={b: b + 1})
b.value # -> 1
f1(3) # -> 4 f2(3) # -> 4 (but update b.value with += 1) b.value # -> 2
f1(3) # -> 5
b.value = 0 f1(3) # -> 3
Param and pfunc¶
The examples above give the general flavour of what pfunc and Param are for. Their signatures are below. Corner cases and exotic examples can be found in the tests.
def pfunc(params, outputs, mode=None, givens=None, updates=None)
"""Function-constructor for graphs with shared variables.
:type params: list of either Variable or Param instances.
:param params: function parameters, these are not allowed to be shared
variables
:type outputs: list of Variables or Out instances
:param outputs: expressions to compute
:param mode: compilation mode
:type updates: iterable over pairs (shared_variable, new_expression). List, tuple or dict.
:param updates: update the values for SharedVariable inputs according to these expressions
:rtype: theano.compile.Function
:returns: a callable object that will compute the outputs (given the inputs)
and update the implicit function arguments according to the `updates`.
"""
...
class Param(object):
def __init__(self, variable, default=None, mutable=False, strict=False):
"""
:param variable: A node in an expression graph to set with each function call.
:param default: The default value to use at call-time (can also be a Container where
the function will find a value at call-time.)
:param name: A string to identify this parameter from function kwargs.
:param mutable: True -> function is allowed to modify this argument.
:param strict: False -> function arguments may be copied or cast to match the
type required by the parameter `variable`. True -> function arguments must exactly match the type
required by `variable`.
:param implicit: see help(theano.io.In)
"""
Note that if some update value is not a variable, it will be cast into
a SharedVariable
using the shared
function. This ensures it is
properly taken into account to build the Theano function underlying the
pfunc
. A consequence of this is that if this update value is mutable
(e.g. a Numpy array), it may be modified after the function is created.
NNet Example¶
Of course there are lots of ways to write the following code, but this is one simple one.
import numpy, theano
from pfunc import pfunc
from sharedvalue import shared
from theano import tensor
from theano.tensor.nnet import sigmoid
class NNet(object):
def __init__(self,
input = tensor.dvector('input'),
target = tensor.dvector('target'),
n_input=1, n_hidden=1, n_output=1, lr=1e-3, **kw):
super(NNet, self).__init__(**kw)
self.input = input
self.target = target
self.lr = shared(lr, 'learning_rate')
self.w1 = shared(numpy.zeros((n_hidden, n_input)), 'w1')
self.w2 = shared(numpy.zeros((n_output, n_hidden)), 'w2')
self.hidden = sigmoid(tensor.dot(self.w1, self.input))
self.output = tensor.dot(self.w2, self.hidden)
self.cost = tensor.sum((self.output - self.target)**2)
self.sgd_updates = {
self.w1: self.w1 - self.lr * tensor.grad(self.cost, self.w1),
self.w2: self.w2 - self.lr * tensor.grad(self.cost, self.w2)}
self.sgd_step = pfunc(
params = [self.input, self.target],
outputs = [self.output, self.cost],
updates = self.sgd_updates)
self.compute_output = pfunc([self.input], self.output)
self.output_from_hidden = pfunc([self.hidden], self.output)