PyTorch基础

Posted on | In

与Tensorflow的静态计算图不同，pytorch的计算图是动态的，可以根据计算需要实时改变计算图
基于python，具备GPU加速的张量和动态神经网络深度学习框架。

安装

按照官网指示，https://pytorch.org

python 3.6 cpu 稳定版
pip3 install http://download.pytorch.org/whl/cpu/torch-0.4.1-cp36-cp36m-linux_x86_64.whl
pip3 install torchvision

—可以换个有gpu的服务器么。。。

教程

官方教程

Pytorch张量运算

100多种，包括转置，数学运算，线性代数，索引切分 https://pytorch.org/docs/stable/torch.html

Tensor创建：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# torch.tensor(data) creates a torch.Tensor object with the given data.
# 1D tensor: vector
V_data = [1., 2., 3.]
V = torch.tensor(V_data)

# Creates a matrix
# 2D tensor 矩阵
M_data = [[1., 2., 3.], [4., 5., 6]]
M = torch.tensor(M_data)

# Create a 3D tensor of size 2x2x2.
# 3D tensor
T_data = [[[1., 2.], [3., 4.]],
          [[5., 6.], [7., 8.]]]
T = torch.tensor(T_data)

# Index into V and get a scalar (0 dimensional tensor)
print(V[0])
# Get a Python number from it
print(V[0].item())

# Index into M and get a vector
print(M[0])

# Index into T and get a matrix
print(T[0])

concatenation运算

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# By default, it concatenates along the first axis (concatenates rows)
# 行连接
x_1 = torch.randn(2, 5) #2行5列
y_1 = torch.randn(3, 5) #3行5列
z_1 = torch.cat([x_1, y_1])
print(z_1) #5行5列

# Concatenate columns:
# 列连接
x_2 = torch.randn(2, 3) #2行3列
y_2 = torch.randn(2, 5) #2行5列
# second arg specifies which axis to concat along
z_2 = torch.cat([x_2, y_2], 1)
print(z_2) #2行8列

Reshaping Tensors

1
2
3
4
5
x = torch.randn(2, 3, 4) # 2*3*4的随机tensor
print(x)
print(x.view(2, 12))  # Reshape to 2 rows, 12 columns
# Same as above.  If one of the dimensions is -1, its size can be inferred
print(x.view(2, -1)) #如果某一维是-1，它的具体大小会被自动推断而出。

Torch Tensor与 NumPy array的互相转换

Converting a Torch Tensor to a NumPy array

1
2
3
4
5
6
7
8
9
from __future__ import print_function
import torch
a = torch.ones(5)
b = a.numpy()

# a变化之后b也会跟着变
a.add_(1)
print(a)
print(b)

Converting NumPy Array to Torch Tensor

1
2
3
4
5
6
import numpy as np
a = np.ones(5)
b = torch.from_numpy(a)
np.add(a, 1, out=a) #a变化之后b也会跟着变
print(a)
print(b)

autograd 自动求导

原理

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# Tensor factory methods have a ``requires_grad`` flag
x = torch.tensor([1., 2., 3], requires_grad=True)

# With requires_grad=True, you can still do all the operations you previously
# could
y = torch.tensor([4., 5., 6], requires_grad=True)
z = x + y
print(z)  
# BUT z knows something extra.
print(z.grad_fn) #grad_fn=<AddBackward0>

s = z.sum()
print(s)
print(s.grad_fn) #grad_fn=<SumBackward0> 

# what is the derivative of this sum with respect to the first component of x?
# s knows that it was created as a sum of the tensor z. z knows that it was the sum x + y
# calling .backward() on any variable will run backprop, starting from it.
s.backward()
print(x.grad)  #tensor([1., 1., 1.])

脱离自动求导的方法

z.detach()

该命令将z从计算历史中分离出来。

returns a tensor that shares the same storage as z, but with the computation history forgotten. It doesn’t know anything
about how it was computed.

with torch.no_grad()

将需要停止自动求导的代码库放置于with torch.no_grad() 范围内

1
2
3
4
5
print(x.requires_grad)  #True
print((x ** 2).requires_grad) #True

with torch.no_grad():
    print((x ** 2).requires_grad) #False

Neural networks

Training a Classifier

先使用python包将数据导入成numpy array

  • images: Pillow, OpenCV
  • audio: scipy and librosa
  • text: NLTK and SpaCy

torchvision包提供了计算机视觉领域 Imagenet, CIFAR10, MNIST等数据集的数据入口。
torchvision.datasets
torch.utils.data.DataLoader