Multiple inputs with different dtype error

[trajepl]

When using multiple inputs with different types, torchsummary generates random inputs with same type torch.FloatTensor

You can delete the assignment of dtype , then pass it as a parameter to get differnt random inputs with various types:

    ## del
    # if device == "cuda" and torch.cuda.is_available():
    #     dtype = torch.cuda.FloatTensor
    # else:
    #     dtype = torch.FloatTensor

    # multiple inputs to the network
    if isinstance(input_size, tuple):
        input_size = [input_size]

    # batch_size of 2 for batchnorm
    # modified
    x = [torch.rand(*in_size).type(dtype) for in_size, dtype in input_size]
    # print(type(x[0]))

Then use to get the summary of your model:
summary(model, [((size1), dtype1), ((size2), dtype2)])

[Naireen]

What use case would this be for? the current design is to ensure that the user doesn't accidentally pass in certain dtypes as torch.FloatTensor, and then other types as torch.cuda.FloatTensor, as that wouldn't work. However, depending on the network architecture, you can't even pass in torch.FloatTensor and torch.ByteTensor for multi-inputs, due to the expected input types (eg for conv layers).

[trajepl]

So I show the code of my model as following:

class WNGat(nn.Module):
    def __init__(
            self,
            in_channels: int,
            hidden_channels: int,
            out_channels: int,
            heads: int = 1,
            concat: bool = True,
            negative_slope: float = 0.2,
            dropout: float = 0.,
            use_checkpoint: bool = False,
            **kwargs) -> None:

        super(WNGat, self).__init__()
        self.conv1 = GATConv(in_channels, hidden_channels,
                             heads, dropout=dropout)
        self.conv2 = GATConv(hidden_channels*heads,
                             hidden_channels, heads, dropout=dropout)
        self.conv3 = GATConv(hidden_channels*heads,
                             out_channels, heads, dropout=dropout)
        self.use_checkpoint = use_checkpoint

    def forward(self, inp, edge_index, size=None):
        if self.use_checkpoint:
            inp = torch.autograd.Variable(inp, requires_grad=True)
            h1 = F.elu(checkpoint(self.conv1, inp, edge_index))
            h2 = F.elu(checkpoint(self.conv2, h1, edge_index))
            oup = F.elu(checkpoint(self.conv3, h2, edge_index))
        else:
            h1 = F.elu(self.conv1(inp, edge_index))
            # h2 = F.elu(self.conv2(h1, edge_index))
            oup = F.elu(self.conv3(h1, edge_index))
        return oup

Simply, the input of 'GATConv' is a graph where contains node featuressize: (num_nodes, num_node_features) and edges size:(2, num_nodes). For example, there is a graph contains three nodes(0, 1, 2) and each node owns 128-dim features vector:

edges:
    0->1
    1->2
    2->0
    2->2

so the parameters for our model are:

inp: size: [3, 128]  dtype: torch.FloatTensor
edge_index: size: [2, 4]  dtype: torch.Long Tensor  data: [[0, 1, 2, 2],[1, 2, 0, 2]]

In this case, the code would not work as the dtype of inp and edge_index are changed to FloatTensor.

In order to make the code to fit my model, I change some lines in torchsummary.py locally:

## 1 - line 49
 # if device == "cuda" and torch.cuda.is_available():
    #     dtype = torch.cuda.FloatTensor
    # else:
    #     dtype = torch.FloatTensor

    # multiple inputs to the network
    if isinstance(input_size, tuple):
        input_size = [input_size]

    # batch_size of 2 for batchnorm
    x = [torch.rand(*in_size).type(dtype) for in_size, dtype in input_size]
    # print(type(x[0]))


## 2 - line 100
    if input_size and len(input_size[0]) > 1 and type(input_size[0][0]) is tuple:
        total_prod = 1
        for input_item in input_size:
            total_prod *= np.prod(input_item[0])
        total_input_size = abs(np.prod(total_prod) * batch_size * 4. / (1024 ** 2.))    
    else:
        total_input_size = abs(np.prod(input_size) * batch_size * 4. / (1024 ** 2.))

Finally, I call the function summary as following to get the right results:

model = model.to(device)
summary(model, [((data.num_nodes, data.num_node_features),
                 torch.FloatTensor), ((2, data.num_edges), torch.LongTensor)])

----------------------------------------------------------------
        Layer (type)               Output Shape         Param #
================================================================
           GATConv-1                  [-1, 512]         734,208
           GATConv-2                  [-1, 256]         131,328
================================================================
Total params: 865,536
Trainable params: 865,536
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 80174.01
Forward/backward pass size (MB): 0.01
Params size (MB): 3.30
Estimated Total Size (MB): 80177.31
----------------------------------------------------------------

[trajepl]

I find a pull request which maybe work in my case.

[Naireen]

Passing in the either dtypes or device type explicitly seems like a popular request, so it's something I would like to add, as this will also address other issues such as being able to run on a specified gpu, eg #92 . However, based on your use case, that PR actually isn't sufficient, as it only allows you to pass in one dtype, and then forces all inputs to that dtype. In your example, you have both torch.FloatTensor, and torch.LongTensor. I have a branch named issue_102 up, let me know if that addresses your issue! I was unable to run your code sample, as its incomplete since GATConv is undefined.

[trajepl]

Thanks. It worked.