Deep Learning Tools

Deep Learning is a new area of Machine Learning research, which has been introduced with the objective of moving Machine Learning closer to one of its original goals: Artificial Intelligence.

Here I have included few tools with little information. There are more than 50 tools are available. Please check this for more http://deeplearning.net/software_links/ .

TensorFlow 

 

Description

  • ·         Open source software library for numerical computation using data flow graphs.
  • ·         Nodes in the graph represent mathematical operations, while the graph edges represent the multidimensional data arrays (tensors) communicated between them.
  • ·         The system is general enough to be applicable in a wide variety of other domains as well.
Background

 

  • ·         Developed by researchers and engineers working on the Google Brain Team within Google’s Machine Intelligence research organization for the purposes of conducting machine learning and deep neural networks research.
  • ·         More information  https://www.tensorflow.org
Use Cases

 

  • ·         Fit a neural network on MNIST handwritten digit recognition data.
  • ·         Note: MNIST is a simple computer vision dataset
  • ·         Breathe life into older media – “Remastering Classic Films in TensorFlow with Pix2Pix”
Supports

 

  • ·         General numerical computations on CPU’s, GPU’s and GPU clusters
  • ·         Flexible architecture allows you to deploy computation to one or more CPUs or GPUs in a desktop, server, or mobile device with a single API.

Theano 

 

Description

  • ·         Python library that lets you to define, optimize, and evaluate mathematical expressions, especially ones with multi-dimensional arrays (numpy.ndarray).
  • ·         Using Theano, it’s possible to attain speeds rivaling hand-crafted C implementations for problems involving large amounts of data. It can also surpass C on a CPU by many orders of magnitude by taking advantage of recent GPUs.
  • ·         Theano combines aspects of a computer algebra system (CAS) with aspects of an optimizing compiler.
  • ·         Also generates customized C code for many mathematical operations. This combination of CAS with optimizing compilation is particularly useful for tasks in which complicated mathematical expressions are evaluated repeatedly and evaluation speed is critical. For situations where many different expressions are each evaluated once Theano can minimize the amount of compilation/analysis overhead, and still provide symbolic features such as automatic differentiation
Background

 

  • ·         Theano was written at the LISA lab to support rapid development of efficient machine learning algorithms. Theano is named after the Greek mathematician, who may have been Pythagoras’ wife. Theano is released under a BSD license (link).
  • ·         More information http://deeplearning.net/software/theano/
Use Cases

 

  • ·         Fit a logistic regression model on a synthetic dataset with GPU acceleration.
Supports

 

  • ·         Tight integration with NumPy – Use numpy.ndarray in Theano-compiled functions. Transparent use of a GPU – Perform data-intensive calculations up to 140x faster than with CPU.(float32 only)
  • ·         Efficient symbolic differentiation – Theano does your derivatives for function with one or many inputs.
  • ·         Speed and stability optimizations – Get the right answer for log (1 + x) even when x is really tiny.
  • ·         Dynamic C code generation – Evaluate expressions faster.
  • ·         Extensive unit-testing and self-verification – Detect and diagnose many types of errors.

Caffe

 

Description

  • ·         Deep learning framework designed with expression, speed, and modularity in mind.
Background

 

  • ·         Developed by the Berkeley Vision and Learning Center (BVLC) and by community contributors. Yangqing Jia created the project during his PhD at UC Berkeley. Caffe is released under the BSD 2-Clause license
  • ·         More information  http://caffe.berkeleyvision.org/
Use Cases

 

  • ·         Academic research projects
  • ·         Startup prototypes
  • ·         Large-scale industrial applications in:
  •  Vision  Speech  Multimedia
  • ·         Fit a neural network on MNIST handwritten digit recognition data.
Supports

 

  • ·         Expressive architecture encourages application and innovation. Models and optimization are defined by configuration without hard-coding. Switch between CPU and GPU by setting a single flag to train on a GPU machine then deploy to commodity clusters or mobile devices.
  • ·         Extensible code fosters active development. In Caffe’s first year, it has been forked by over 1,000 developers and had many significant changes contributed back. Thanks to these contributors the framework tracks the state-of-the-art in both code and models.
  • ·         Speed makes Caffe perfect for research experiments and industry deployment. Caffe can process over 60M images per day with a single NVIDIA K40 GPU*. That’s 1 ms/image for inference and 4 ms/image for learning. We believe that Caffe is the fastest convnet implementation available.
  • ·         Community: Caffe already powers academic research projects, startup prototypes, and even large-scale industrial applications in vision, speech, and multimedia. Join our community of brewers on the caffe-users group and Github. With the ILSVRC2012-winning SuperVision model and caching IO. Consult performance details.

CaffeMXNet

 

Description

  • ·         Open-source deep learning framework that allows you to define, train, and deploy deep neural networks on a wide array of devices, from cloud infrastructure to mobile devices.
  • ·         It is highly scalable, allowing for fast model training, and supports a flexible programming model and multiple languages. MXNet allows you to mix symbolic and imperative programming flavors to maximize both efficiency and productivity.
  • ·         MXNet is built on a dynamic dependency scheduler that automatically parallelizes both symbolic and imperative operations on the fly.
  • ·          A graph optimization layer on top of that makes symbolic execution fast and memory efficient.
  • ·         The MXNet library is portable and lightweight, and it scales to multiple GPUs and multiple machines.
Background

 

  • ·         Developed by collaborators from multiple universities and companies.
  • ·          More information  http://mxnet.io/
Use Cases

 

  • ·         Basics

Manipulating arrays, building networks, loading and preprocessing data

  • ·         IO

Data loading, Image IO, Record IO, Custom Image IO

  • ·         Computer Vision

Use machine learning to modify, classify, and segment images and video

Handwritten Digit Classification

Image Classification

Image Segmentation

Object Detection using faster R-CNN

Neural Art: Adding Artistic Style to Images

Large Scale Image Classification

Classifying Real-World Images with a Pre-Trained Model

  • ·         Natural Language Processing

Applications that understand, generate, and summarize text-based data

Character Level LSTM

Text Classification using Convolutional Neural Network

NCE Loss

  • ·         Speech Recognition

Applications that map natural speech to text

Speech LSTM

Baidu Warp CTC

  • ·         Generative Networks

General Adversarial Network

  • ·         Unsupervised Machine Learning

Develop applications for discovering existing structures and relationships in datasets

Matrix Factorization

Auto Encoders

Supports

 

  • ·         Flexible – Supports both imperative and symbolic programming
  • ·         Portable – Runs on CPUs or GPUs, on clusters, servers, desktops, or mobile phones
  • ·         Multiple Languages – Supports multiple languages, including C++, Python, R, Scala, Julia, Matlab and Javascript – All with the same amazing performance.
  • ·         Auto-Differentiation – Calculates the gradient automatically for training a model
  • ·         Distributed on Cloud – Supports distributed training on multiple CPU/GPU machines, including AWS, GCE, Azure, and Yarn clusters
  • Recommendation Systems  Visualization  Understanding the vanishing gradient problem through visualization  Embedded  Develop machine learning applications running on embedded devices,

BigDL

 

Description

  • ·         Distributed deep learning library for Apache Spark.
  • ·          Users can write their deep learning applications as standard Spark programs, which can directly run on top of existing Spark or Hadoop clusters.
Background

 

More information  https://github.com/intel-analytics/BigDL
Use Cases

 

  • ·         Analyze a large amount of data on the same Big Data (Hadoop/Spark) cluster where the data are stored (in, say, HDFS, HBase, Hive, etc.).
  • ·         Add deep learning functionalities (either training or prediction) to your Big Data (Spark) programs and/or workflow.
  • ·         Leverage existing Hadoop/Spark clusters to run your deep learning applications, which can be then dynamically shared with other workloads (e.g., ETL, data
Supports

 

  • ·         Rich deep learning support. Modeled after Torch, BigDL provides comprehensive support for deep learning, including numeric computing (via Tensor) and high level neural networks; in addition, users can load pre-trained Caffe or Torch models into Spark programs using BigDL.
  • ·         Extremely high performance. To achieve high performance, BigDL uses Intel MKL and multi-threaded programming in each Spark task. Consequently, it is orders of magnitude faster than out-of-box open source Caffe, Torch or TensorFlow on a single-node Xeon (i.e., comparable with mainstream GPU).

DistBelief

 

Description

·         Framework for training deep neural networks.
Background

 

·         DistBelief was first presented in the 2012 paper “Large Scale Distributed Deep Networks” by Dean et al. In this paper, the GraphLab distributed computing framework was used to implement DistBelief.

·         More information  https://github.com/alexminnaar/AkkaDistBelief

·         http://alexminnaar.com/implementing-the-distbelief-deep-neural-network-training-framework-with-akka.html

Use Cases

 

  • ·         Supervised Learning
  • ·         Time Series & Clustering
  • ·         Bayesian Inference NLP
  • ·         Distributed Binary Search Tree with Akka
Supports

 

  • ·         Avoids GPUs entirely and performs parallel computing with clusters of commodity machines.
  • ·         Relies heavily on asynchronous message passing which makes the Akka actor framework a suitable alternative
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s