Provide programmatic access to param names and values

[fritzo]

@ragulpr pointed out a need for programmatic access to params.

Maybe we can implement this as a .param_values property that returns an OrderedDict from names to current values. Note that .params currently over-specifies possible parameters and does not serve this purpose.

class Normal(ExponentialFamily):
    @lazy_property
    def param_values(self):
        return OrderedDict([
            ('loc', self.loc),
            ('scale', self.scale),
        ]) 

This is still subtle to use since some params are not tensors, e.g. Multinomial takes an int n.

[tbrx]

Do we have any particular standardization for how attributes of Distributions are named, relative to the .params name?

Just thinking that for many (most?) distributions, then they keys in .params would actually be able to be used directly to get the values needed for this dict. It would be nice if a default implementation could cover most distributions, and then we would only need to explicitly implement .canonical_params for those distributions which have overspecified params dicts.

[fritzo]

There is no such standard and I believe there cannot be. This is difficult because some distributions take multiple parameterizations (e.g. probs vs logits for Categorical, covariance_matrix vs scale_tril for MultivariateNormal). Not only are these params lazily constructed, but they should all be specified in the .params dict (some are currently missing and this is a bug). Thus a naive implementation of .param_values that simply inspected .params would overparameterize some distributions, e.g. specifying both logits and probs for Categorical.

[tbrx]

Ah, sorry, wasn't clear — I meant actually for the values themselves. For example, for the normal distribution, the .params has keys 'loc' and 'scale', and the values themselves are at self.loc and self.scale.

So in theory in the example you gave above we could just loop over the .params.keys(), and then p[key] = getattr(self, key).

This doesn't work for either Categorical or MultivariateNormal, as you say. But I think it could work for a lot of them if our naming of params and attributes were consistent, and we would just have to implement canonical_params specifically for those few outliers.

[ragulpr]

I've been working in ways like this:

class Pareto(PositiveDistribution):
    r"""
    """
    params = {'scale': constraints.positive, 'shape': constraints.positive}
    support = constraints.positive
    has_rsample = True

    def __init__(self, scale, shape, is_discrete=False):
        self.scale, self.shape = broadcast_all(scale, shape)
        self.params_tuple = (self.scale, self.shape)
        batch_shape = torch.Size() if isinstance(scale, Number) else self.scale.size()
        super(Pareto, self).__init__(batch_shape, is_discrete=is_discrete)

And then you can initialize it as

params_tuple = (torch.ones(1),torch.ones(1)) 
dist = Pareto(*params_tuple,**distkwargs)

I rarely find myself accessing the parameters by name dist.scale as I'm working with many distributions with different names. I think this is an important abstraction. Also, the more we can sideline having to name parameters the safer we are as this is a religious topic 😄. What is the need for a dict vs a tuple?

• I'm not sure about the level of thread-safety of this approach.
• I'm not sure about the performance-cost of looping as proposed by @tbrx