Tim Blog

Himmel oder Hölle

tensorflow graph management

tf.reset_default_graph() purpose: to delete all the available node(variables, operations, tensors), and reset the graph to default graph

Posted by neverset on April 11, 2020

tensorflow fully connected layer

fully connected layer is the layer after cnn, its purpose to convert the feature map to the predefined categories tf.nn.xw_plus_b((x, weights) + biases) it is equal to the formulation: mat...

Posted by neverset on April 11, 2020

tensorflow LeNet&AlexNet&VGG

development of CNN LeNet there are 2 conv layers, 2 pooling layers, 2 fc layers and 1 output layer, tanh or sigmoid activation function Conv1: 6 kernels with size of 5x5 , stride=1, ...

Posted by neverset on April 11, 2020

tensorflow LRN

Local Response Normalization (LRN) is first introduced in AlexNet, is used after activation and pooling to increase normalization ablitity LRN maths background physical meaning: the output featu...

Posted by neverset on April 10, 2020

tensorflow ConfigProto&GPU

the usage of gpu can be checked with command watch -n 1 nvidia-smi # check dynamic usage nvidia-smi #check static usage tensorflow ConfigProto log_device_placement=True : log computation...

Posted by neverset on April 7, 2020

tensorflow learning rate decay

to use large learning rate at the beginning, and decays step by step, function tf.train.exponential_decay() can be used large learning rate for simply task can lead to result vibration!(not larger ...

Posted by neverset on April 6, 2020

tensorflow regulazition

regulazition is to add model complexity item in loss function to avoid overfitting, so the new loss function is J(\theta)+\lambda *R(w) Implementation create a regulazition method apply metho...

Posted by neverset on April 6, 2020

tensorflow moving average

google says that use averaged parameters sometimes produce significantly better results than the final trained values. tf.train.ExponentialMovingAverage（decay, num_updates=None, name=’ExponentialM...

Posted by neverset on April 6, 2020

tensorflow Fully connected layer to CNN

fully connected layer+CNN has some drawbacks compared with fully CNN even if they have the same number of parameters and time/space cost the former modell can only classify object for one whole...

Posted by neverset on April 5, 2020

tensorflow Initialization

the initialization of weight(or kernel) can have signifikant effect on the speed and accuracy of the model. there are several way to initialize the weight random xavier He the biase is us...

Posted by neverset on April 5, 2020