pytorch中的named_parameters()和parameters()
作者：Hanawh
pytorch named_parameters()和parameters()
nn.Module
nn.Module里面关于参数有两个很重要的属性named_parameters()和parameters()，前者给出网络层的名字和参数的迭代器，而后者仅仅是参数的迭代器。
import torchvision.models as models
model = models.resnet18()
for param in model.named_parameters():
print(param[0])
'''
conv1.weight
bn1.weight
bn1.bias
layer1.0.conv1.weight
layer1.0.bn1.weight
layer1.0.bn1.bias
layer1.0.conv2.weight
layer1.0.bn2.weight
layer1.0.bn2.bias
layer1.1.conv1.weight
layer1.1.bn1.weight
layer1.1.bn1.bias
layer1.1.conv2.weight
layer1.1.bn2.weight
layer1.1.bn2.bias
layer2.0.conv1.weight
layer2.0.bn1.weight
layer2.0.bn1.bias
layer2.0.conv2.weight
layer2.0.bn2.weight
layer2.0.bn2.bias
layer2.0.downsample.0.weight
layer2.0.downsample.1.weight
layer2.0.downsample.1.bias
layer2.1.conv1.weight
layer2.1.bn1.weight
layer2.1.bn1.bias
layer2.1.conv2.weight
layer2.1.bn2.weight
layer2.1.bn2.bias
layer3.0.conv1.weight
layer3.0.bn1.weight
layer3.0.bn1.bias
layer3.0.conv2.weight
layer3.0.bn2.weight
layer3.0.bn2.bias
layer3.0.downsample.0.weight
layer3.0.downsample.1.weight
layer3.0.downsample.1.bias
layer3.1.conv1.weight
layer3.1.bn1.weight
layer3.1.bn1.bias
layer3.1.conv2.weight
layer3.1.bn2.weight
layer3.1.bn2.bias
layer4.0.conv1.weight
layer4.0.bn1.weight
layer4.0.bn1.bias
layer4.0.conv2.weight
layer4.0.bn2.weight
layer4.0.bn2.bias
layer4.0.downsample.0.weight
layer4.0.downsample.1.weight
layer4.0.downsample.1.bias
layer4.1.conv1.weight
layer4.1.bn1.weight
layer4.1.bn1.bias
layer4.1.conv2.weight
layer4.1.bn2.weight
layer4.1.bn2.bias
fc.weight
fc.bias
'''模型参数：named_parameters()、parameters()、state_dict()区别
torch中存在3个功能极其类似的方法，它们分别是model.parameters()、model.named_parameters()、model.state_dict()，
下面就具体来说说这三个函数的差异：
一、model.parameters()和model.named_parameters()差别
- named_parameters()返回的list中，每个元组（与list相似，只是数据不可修改）打包了2个内容，分别是
layer-name和layer-param（网络层的名字和参数的迭代器）； - parameters()只有后者layer-param（参数的迭代器）
1、model.named_parameters()里的网络层名字
import torchvision.models as models
model = models.resnet18()
for param_tuple in model.named_parameters():
name, param = param_tuple
print("name = ", name)
print("-" * 100)打印结果：
name = conv1.weight
----------------------------------------------------------------------------------------------------
name = bn1.weight
----------------------------------------------------------------------------------------------------
name = bn1.bias
----------------------------------------------------------------------------------------------------
name = layer1.0.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer1.0.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer1.0.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer1.0.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer1.0.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer1.0.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer1.1.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer1.1.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer1.1.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer1.1.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer1.1.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer1.1.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer2.0.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer2.0.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer2.0.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer2.0.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer2.0.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer2.0.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer2.0.downsample.0.weight
----------------------------------------------------------------------------------------------------
name = layer2.0.downsample.1.weight
----------------------------------------------------------------------------------------------------
name = layer2.0.downsample.1.bias
----------------------------------------------------------------------------------------------------
name = layer2.1.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer2.1.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer2.1.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer2.1.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer2.1.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer2.1.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer3.0.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer3.0.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer3.0.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer3.0.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer3.0.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer3.0.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer3.0.downsample.0.weight
----------------------------------------------------------------------------------------------------
name = layer3.0.downsample.1.weight
----------------------------------------------------------------------------------------------------
name = layer3.0.downsample.1.bias
----------------------------------------------------------------------------------------------------
name = layer3.1.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer3.1.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer3.1.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer3.1.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer3.1.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer3.1.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer4.0.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer4.0.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer4.0.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer4.0.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer4.0.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer4.0.bn2.bias
----------------------------------------------------------------------------------------------------
name = layer4.0.downsample.0.weight
----------------------------------------------------------------------------------------------------
name = layer4.0.downsample.1.weight
----------------------------------------------------------------------------------------------------
name = layer4.0.downsample.1.bias
----------------------------------------------------------------------------------------------------
name = layer4.1.conv1.weight
----------------------------------------------------------------------------------------------------
name = layer4.1.bn1.weight
----------------------------------------------------------------------------------------------------
name = layer4.1.bn1.bias
----------------------------------------------------------------------------------------------------
name = layer4.1.conv2.weight
----------------------------------------------------------------------------------------------------
name = layer4.1.bn2.weight
----------------------------------------------------------------------------------------------------
name = layer4.1.bn2.bias
----------------------------------------------------------------------------------------------------
name = fc.weight
----------------------------------------------------------------------------------------------------
name = fc.bias
----------------------------------------------------------------------------------------------------Process finished with exit code 0
2、model.named_parameters()里的网络层名字、参数
import torchvision.models as models
model = models.resnet18()
for param_tuple in model.named_parameters():
name, param = param_tuple
print("name = ", name)
print("-" * 100)
print("param_tuple = ", param_tuple)
print("*" * 200)打印结果：
C:\Program_Files_AI\Anaconda3531\python.exe C:/Users/Admin/OneDrive/WorkSpace_AI/0-基于知识库的智能问答系统-华控智加/01-意图识别/test.py
name = conv1.weight
----------------------------------------------------------------------------------------------------
param_tuple = ('conv1.weight', Parameter containing:
tensor([[[[-1.4115e-05, 2.9187e-02, 2.9325e-03, ..., -4.2247e-02,
1.7490e-02, -4.5253e-02],
[-2.4594e-02, -3.0836e-02, 3.8604e-02, ..., 3.5473e-02,
-4.7046e-03, -2.9440e-02],
[ 2.4811e-02, 1.2679e-02, 1.0070e-02, ..., -8.3476e-03,
1.7960e-02, -1.7406e-02],
...,
[-1.3021e-02, 2.9023e-02, -6.1800e-02, ..., -5.2802e-02,
-4.7817e-02, -2.2377e-02],
[-3.8513e-03, -1.0603e-02, -3.9712e-02, ..., 5.1941e-03,
8.2868e-03, -8.3469e-03],
[ 3.8993e-03, 3.2017e-02, -3.6292e-02, ..., -2.0210e-02,
-4.0358e-02, 1.7709e-02]],[[-1.0894e-03, 1.5720e-02, 7.0129e-03, ..., -1.2024e-02,
1.8644e-02, 1.7892e-02],
[-2.3866e-02, 9.1136e-03, 3.5243e-02, ..., -1.6756e-02,
1.4441e-03, 4.7943e-02],
[-2.0514e-03, 4.3022e-02, 2.6358e-02, ..., -2.3662e-02,
-7.8241e-04, 1.0167e-02],
...[[-4.6689e-02, -1.1407e-03, 1.8674e-02, ..., 1.2649e-03,
-2.9532e-02, 6.4535e-04],
[ 1.4171e-03, -1.9274e-02, -8.6811e-03, ..., 2.4428e-02,
6.9516e-03, 4.3715e-02],
[ 1.9982e-02, 1.3124e-02, 9.1508e-03, ..., 2.5405e-02,
-1.3132e-02, 4.0835e-02],
...,
[-3.4174e-03, 1.8623e-02, -1.4386e-02, ..., 1.0627e-03,
-5.1297e-04, 2.2055e-02],
[ 2.7333e-02, 2.4858e-02, -5.4305e-02, ..., -1.2139e-02,
1.7735e-03, -3.4184e-03],
[ 1.1412e-03, 1.5794e-02, -2.0699e-02, ..., -1.7846e-02,
3.7425e-02, -1.6059e-02]]],
...,
[[[-2.7389e-02, -3.8327e-02, -2.9043e-02, ..., -7.6396e-03,
-1.6519e-02, 3.9659e-02],
[ 2.8740e-03, -1.0621e-02, -9.2430e-03, ..., 2.2581e-02,
5.1526e-03, -2.0006e-02],
[ 1.3575e-02, 1.5290e-02, -1.7260e-02, ..., 6.3830e-03,
-1.9759e-02, 1.5501e-02],
...,
[ 1.6091e-02, 2.4038e-02, 2.4507e-02, ..., -4.5613e-02,
-3.6233e-02, 2.1632e-02],
[-1.1573e-02, -3.6514e-02, 4.1576e-02, ..., 1.8090e-02,
-2.3350e-02, -8.7074e-03],
[-1.5837e-02, -3.1353e-02, 1.8726e-02, ..., 9.3698e-03,
3.0781e-02, 1.0976e-02]],[[-2.7063e-02, 8.7158e-03, 2.7193e-03, ..., -1.6670e-03,
-4.3033e-03, 7.2011e-04],
[ 2.7870e-03, 1.4264e-02, -5.0581e-02, ..., 2.5463e-02,
7.6864e-03, -4.9655e-02],
[ 2.6030e-03, 2.5918e-02, 2.9615e-02, ..., 3.0676e-02,
-2.7723e-02, -7.3628e-03],
...,
[ 2.5969e-02, -1.4247e-02, 1.2516e-02, ..., 5.9602e-03,
-3.2843e-02, 3.5822e-02],
[ 1.2845e-02, -2.0035e-02, 9.9398e-04, ..., -3.1800e-02,
5.7984e-03, 2.8756e-02],
[ 2.3458e-02, 3.8193e-02, -2.3754e-03, ..., -1.3867e-02,
8.0831e-03, -3.2438e-02]],
...[[-9.9291e-03, -5.6023e-03, -1.7064e-02, ..., 8.8544e-03,
-5.8145e-03, 2.3248e-02],
[ 1.2148e-02, -1.0730e-02, -1.2682e-02, ..., 9.4389e-03,
1.2149e-02, 3.8613e-03],
[ 3.5913e-02, -5.2048e-04, -8.7133e-02, ..., -2.0969e-03,
-5.4117e-03, 5.4637e-05],
...,
[ 4.0351e-03, -1.3189e-02, 3.1229e-02, ..., 3.2340e-02,
-2.8351e-02, 1.0634e-02],
[ 2.6041e-02, -3.0633e-04, -1.2732e-02, ..., 2.9417e-02,
-7.3859e-03, 1.7207e-02],
[ 6.9960e-04, 3.8486e-03, 1.0397e-02, ..., 1.4535e-03,
-3.6449e-02, 3.4848e-02]]]], requires_grad=True))
********************************************************************************************************************************************************************************************************
name = bn1.weight
----------------------------------------------------------------------------------------------------
param_tuple = ('bn1.weight', Parameter containing:
tensor([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
1., 1., 1., 1., 1., 1., 1., 1., 1., 1.], requires_grad=True))
********************************************************************************************************************************************************************************************************
name = bn1.bias
----------------------------------------------------------------------------------------------------
param_tuple = ('bn1.bias', Parameter containing:
tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
requires_grad=True))
****************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************
name = layer1.0.conv2.weight
----------------------------------------------------------------------------------------------------
param_tuple = ('layer1.0.conv2.weight', Parameter containing:
tensor([[[[-8.6159e-02, 1.8507e-04, 5.4006e-03],
[-6.3063e-03, 3.9225e-03, -6.3141e-02],
[-7.0145e-02, -3.9266e-02, 1.9724e-03]],[[ 4.6454e-02, 2.1519e-02, 5.3696e-02],
[ 1.1086e-02, 1.6269e-01, -7.0579e-02],
[-1.1220e-01, -4.9811e-02, -7.5515e-02]],[[ 5.5275e-02, -8.2407e-02, -8.9807e-02],
[ 5.8418e-02, 4.4029e-02, 3.0584e-03],
[ 5.2371e-02, -1.5983e-02, 5.1494e-02]],...,
[[ 7.3441e-02, 4.5401e-02, -1.9175e-02],
[-6.2500e-02, -8.5905e-03, -7.4856e-02],
[-1.6170e-02, 3.7529e-02, -5.1231e-02]],[[ 7.7501e-04, -5.7506e-02, 1.8422e-01],
[ 2.4594e-02, 1.7378e-02, 4.0000e-02],
[-8.6796e-02, -6.0548e-02, 2.6795e-02]],[[ 5.3264e-02, -8.6190e-02, 4.2443e-02],
[-6.8029e-03, -1.6581e-02, 7.8568e-02],
[ 3.2037e-02, -7.3002e-02, 4.9353e-02]]]], requires_grad=True))
...，
********************************************************************************************************************************************************************************************************
name = layer4.1.bn2.bias
----------------------------------------------------------------------------------------------------
param_tuple = ('layer4.1.bn2.bias', Parameter containing:
tensor([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
...,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0.], requires_grad=True))
********************************************************************************************************************************************************************************************************
name = fc.weight
----------------------------------------------------------------------------------------------------
param_tuple = ('fc.weight', Parameter containing:
tensor([[-0.0125, 0.0437, -0.0014, ..., -0.0230, 0.0280, 0.0249],
[-0.0105, 0.0242, 0.0291, ..., 0.0153, 0.0366, -0.0236],
[-0.0315, 0.0306, -0.0216, ..., 0.0387, 0.0403, 0.0056],
...,
[-0.0068, -0.0222, -0.0027, ..., -0.0243, 0.0260, 0.0065],
[ 0.0213, 0.0167, -0.0379, ..., -0.0140, 0.0037, -0.0372],
[ 0.0180, 0.0101, -0.0341, ..., -0.0295, -0.0146, 0.0416]],
requires_grad=True))
********************************************************************************************************************************************************************************************************
name = fc.bias
----------------------------------------------------------------------------------------------------
param_tuple = ('fc.bias', Parameter containing:
tensor([ 3.5711e-02, 3.2682e-02, 7.5932e-03, -3.1623e-02, -9.6316e-03,
-2.4051e-02, -1.0393e-02, 2.3210e-02, -3.6044e-02, 2.3099e-02,
-3.5723e-02, -3.9482e-02, 4.8526e-03, -3.2688e-02, 3.7720e-03,
-2.2014e-02, -4.0935e-02, 4.0533e-02, -4.1172e-02, 3.9513e-02,
-3.0332e-02, 3.2777e-02, 1.3342e-02, 2.3394e-02, 8.2328e-03,
1.3757e-02, -1.7578e-02, -2.7165e-02, 3.8495e-03, -3.2116e-02,
7.9903e-03, 9.9640e-04, -8.3106e-03, 2.5033e-02, -3.0446e-02,
-1.8282e-02, -3.8420e-03, -8.6129e-03, -4.2712e-03, 1.7169e-02,
...,
-5.3570e-05, -3.7353e-02, -9.8633e-03, -9.1069e-03, 3.2688e-02,
2.2457e-02, 7.6379e-03, -3.6287e-02, -1.0444e-02, 2.1669e-02,
2.5270e-02, -4.3881e-02, 2.1960e-02, 2.6293e-02, -3.5049e-02,
-2.0074e-02, -9.7686e-03, -2.3766e-02, -5.0265e-03, -2.1095e-02,
2.0981e-02, -3.5132e-02, 8.6407e-03, 1.8453e-02, 2.4282e-02,
3.8392e-02, -1.7470e-02, 3.6958e-02, -3.7590e-02, -4.1951e-02,
-1.8246e-02, 9.0818e-03, 3.8774e-02, 7.3408e-03, 1.7728e-02,
3.5547e-02, -7.2857e-03, -2.7015e-02, -8.6983e-03, -2.3785e-02],
requires_grad=True))
********************************************************************************************************************************************************************************************************Process finished with exit code 0
二、model.named_parameters()和model.state_dict()差别
它们的差异主要体现在3方面：
- 返回值类型不同
- 存储的模型参数的种类不同
- 返回的值的require_grad属性不同
| named_parameters() | state_dict() |
|---|---|
| 将layer_name : layer_param的键值信息打包成一个元祖然后再存到list当中 | 将layer_name : layer_param的键值信息存储为dict形式 |
| 只保存可学习、可被更新的参数，model.buffer()中的参数不包含在model.named_parameters()中 | 存储的是该model中包含的所有layer中的所有参数 |
| require_grad属性都是True | 存储的模型参数tensor的require_grad属性都是False |
为何model.parameters()迭代出来的所有参数的require_grad属性都是True，因为它们在被创建时，默认的require_grad就是True。
这也符合逻辑，即，使用nn.Parameter()创建的变量是模型参数，本就是要参与学习和更新的
总结
以上为个人经验，希望能给大家一个参考，也希望大家多多支持脚本之家。