Graph & Session

tf.Graph
1. 存了Graph的结构
2. 提供了graph collections机制来存graph的metadata，collection就是op或者tensor（包括Variable）的集合

创建tf.Graph
1. TensorFlow provides a default graph that is an implicit argument to all API functions in the same context
2. 对train，eval，inference 创建不同graph的好处：
	a.The inference graph is usually very different from the other two
	b.The eval graph becomes simpler since it no longer has all the additional backprop ops.
	c.Data feeding can be implemented separately for each graph.
	d.Variable reuse is much simpler. For example, in the eval graph there's no need to reopen variable scopes with reuse=True just because the Training model created these variables already.
	e. In distributed training, it is commonplace to have separate workers perform training, eval, and inference. These need to build their own graphs anyway. So building the system this way prepares you for distributed training.
3. 使用g = tf.Graph()来创建新的图，用with g.as_default()来做context manager。用g = tf.get_default_graph()来检查当前的context

Naming operations
每创建一个op都会被加到graph里，每一个op都有唯一的name和它对应
1. Each API function that creates a new tf.Operation or returns a new tf.Tensor accepts an optional name argument
	tf.constant(42.0, name="answer")，产生一个op，名字为'answer'. 如果已经有'name'的op了，那会被自动加成'name_1'
	tf.name_scope可以创建一个scope，scope里的op的name会自动被添加一个prefix。如果当前scope已经存在同名的subscope，则自动添加'_1','_2'在新建subscope后面

c_0 = tf.constant(0, name="c")  # => operation named "c"

# Already-used names will be "uniquified".
c_1 = tf.constant(2, name="c")  # => operation named "c_1"

# Name scopes add a prefix to all operations created in the same context.
with tf.name_scope("outer"):
  c_2 = tf.constant(2, name="c")  # => operation named "outer/c"

  # Name scopes nest like paths in a hierarchical file system.
  with tf.name_scope("inner"):
    c_3 = tf.constant(3, name="c")  # => operation named "outer/inner/c"

  # Exiting a name scope context will return to the previous prefix.
  c_4 = tf.constant(4, name="c")  # => operation named "outer/c_1"

  # Already-used name scopes will be "uniquified".
  with tf.name_scope("inner"):
    c_5 = tf.constant(5, name="c")  # => operation named "outer/inner_1/c"
给op命名的好处是在visualize graph的时候op会按namescope被组织起来：将用一个name_scope里的op组合成一个super node

Naming tensor
任何一个tensor都是由某个op产生的，tensor的name不可更改，构成为 '<op_name>:<i>' ：
	<op_name>是产生该tensor的op的名字
	<i> is an integer representing the index of that tensor among the operation's outputs.
	
op接受的参数：tensor-like object
能接受tensor作为参数输入的op其实可以接受其他tensor-like object并自动用tf.convert_to_tensor把它们转换成tensor：
	tf.Tensor
	tf.Variable
	numpy.ndarray
	list (and lists of tensor-like objects)
	Scalar Python types: bool, float, int, str
注意：当使用同一个tensor-like object时，tensorflow默认都会创建一个新的tf.Tensor对象。为避免占据太多内存，应手动对tensor-like object调用tf.convert_to_tensor，然后使用返回的tf.Tensor对象

与variable_scope的差别
1.name_scope会给该scope内的op加上prefix。另外创建一个name_scope时，只要name存在，则新创建的scope会被视为是同一个name_scope。
2.name_scope不会给该scope内的variable加上prefix，里面的variable跟随name_scope外的scope名字来命名
3.variable_scope会给该scope内的variable加上prefix，并且按variable是否reuse的规则来运行
4.variable_scope会给该scope内的op加上prefix。另外创建一个同样name的variable_scope时，里面op的prefix会被算作不在同一scope：
with tf.variable_scope('layer456') as vs:
    t1 = tf.constant(1,dtype=tf.int32,shape=[])   #"layer456/Const:0"
    t4 = tf.constant(1,dtype=tf.int32,shape=[])   #"layer456/Const_1:0"

with tf.variable_scope('layer456', reuse=True):
    t2 = tf.constant(2, dtype=tf.int32, shape=[])  #"layer456_1/Const:0"

with tf.variable_scope(vs, reuse=True):
    t3 =  tf.constant(3, dtype=tf.int32, shape=[])  #"layer456_2/Const:0"




tf.Session
1. tf.Session将client program连接到c++核心
2. 创建Session：使用with语句来自动释放Session所占用的资源，不然就手动tf.Session.close
3. tf.Session.__init__:
	target：session所要连接的machine，None的话就是local machine
	graph：所要执行的graph，默认是当前的default graph
	config：一些对于session控制的参数

4.feed机制：向图传入数据，可以是numpy，list等python中常用的数据结构
  fetch机制：从图中取出数据，类型为numpy的数据类型，可以直接在python中进行处理
5.可以监控运行过程中运行了哪些节点
options = tf.RunOptions()
options.output_partition_graphs = True
options.trace_level = tf.RunOptions.FULL_TRACE
metadata = tf.RunMetadata()

sess.run(y, options=options, run_metadata=metadata)

# Print the subgraphs that executed on each device.
print(metadata.partition_graphs)

# Print the timings of each operation that executed.
print(metadata.step_stats)

一个清晰的tf程序应该写成
g = tf.Graph()
with g.as_default():
	// add op node to graph
	
with tf.Session(graph = g) as sess:
	// run computation on the graph