Tuesday March 28, 2017
TensorFlow has a flexible API, and it has automatic differentiation, and it can run on GPUs. But the thing that’s really neat about TensorFlow is that it gives you a fairly general way to easily program across multiple computers.
TensorFlow’s distributed runtime, the big bottom box in this figure from the 2016 paper “TensorFlow: A system for large-scale machine learning”, is the part of TensorFlow that runs the computation graph.
The computation graph, specified with protocol buffers, is much higher level than Java virtual machine (JVM) bytecode. But I think it’s interesting to think of the TensorFlow distributed runtime as a sort of virtual machine. This is not a whole system virtual machine, but a process virtual machine, like the JVM, to “execute computer programs in a platform-independent environment.” The Python API can be like a domain-specific language for programming the TensorFlow graph.
Unlike the JVM, and unlike any other