Seide 2016

CNTK: Microsoft's Open-Source Deep-Learning Toolkit
Frank Seide Amit Agarwal

Microsoft Research Microsoft Research
One Microsoft Way One Microsoft Way
Redmond, WA 98052 Redmond, WA 98052
fseide@microsoft.com amitaga@microsoft.com
ABSTRACT
This tutorial will introduce the Computational Network SPEAKER BIOGRAPHIES
Toolkit, or CNTK, Microsoft’s cutting-edge open-source deep- Frank Seide, a native of Hamburg, Germany, is a Senior
learning toolkit for Windows and Linux. CNTK is a powerful Researcher at Microsoft Research. His current research focus
computation-graph based deep-learning toolkit for training and is on deep neural networks for conversational speech
evaluating deep neural networks. Microsoft product groups use recognition; together with co-author Dong Yu, he was first to
CNTK, for example to create the Cortana speech models and show the effectiveness of deep neural networks for recognition
web ranking. CNTK supports feed-forward, convolutional, and of conversational speech. Throughout his career, he has been
recurrent networks for speech, image, and text workloads, also interested in and worked on a broad range of topics and
in combination. Popular network types are supported either components of automatic speech recognition, including
natively (convolution) or can be described as a CNTK spoken-dialogue systems, recognition of Mandarin Chinese,
configuration (LSTM, sequence-to-sequence). CNTK scales to and, particularly, large-vocabulary recognition of
multiple GPU servers and is designed around efficiency. conversational speech with application to audio indexing,
transcription, and speech-to-speech translation. His current
The tutorial will give an overview of CNTK's general focus is Microsoft's CNTK deep-learning toolkit.
architecture and describe the specific methods and algorithms
used for automatic differentiation, recurrent-loop inference Amit Agarwal is a Principal Software Engineer at Microsoft’s
and execution, memory sharing, on-the-fly randomization of Technology and Research division. His current focus is on
large corpora, and multi-server parallelization. We will then building CNTK, Microsoft’s large scale distributed deep
show how typical uses looks like for relevant tasks like image learning platform, to enable unprecedented scale, speed and
recognition, sequence-to-sequence modeling, and speech capacity for training massive deep learning models on
recognition. enormous datasets, used in a wide gamut of speech, image and
text related deep learning tasks at Microsoft and in the
OUTLINE community. Amit Agarwal worked on a wide range of
The tutorial will cover these topics: Microsoft products and at Mentor graphics. He holds 7 patents
related to heterogeneous and GPU programming.
 What is CNTK?
o Computational network introduction
REFERENCES
 How does a typical use of CNTK look like?
[1] Amit Agarwal, Eldar Akchurin, Chris Basoglu, Guoguo
o Defining the Computational Network
Chen, Scott Cyphers, Jasha Droppo, Adam Eversole,
o Configuring data I/O
Brian Guenter, Mark Hillebrand, T. Ryan Hoens,
o SGD hyper-parameters
Xuedong Huang, Zhiheng Huang, Vladimir Ivanov,
o Typical workflows
Alexey Kamenev, Philipp Kranen, Oleksii Kuchaiev,
 Deep Dive into specific technologies Wolfgang Manousek, Avner May, Bhaskar Mitra, Olivier
o Implicit handling of time Nano, Gaizka Navarro, Alexey Orlov, Hari Parthasarathi,
o Minibatching of variable-length sequences Baolin Peng, Marko Radmilac, Alexey Reznichenko,
o Data-parallel training Frank Seide, Michael L. Seltzer, Malcolm Slaney,
 CNTK Library APIs Andreas Stolcke, Huaming Wang, Yongqiang Wang,
o Network, Reader, Learner, SGD API Kaisheng Yao, Dong Yu, Yu Zhang, Geoffrey Zweig (in
o C++ usage alphabetical order), "An Introduction to Computational
o Python usage Networks and the Computational Network Toolkit",
 Hands-on examples, including Microsoft Technical Report MSR-TR-2014-112, 2014..
o ResNet image recognition
o Sequence-to-sequence modeling [2] “CNTK,” https://github.com/Microsoft/CNTK
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full
citation on the first page. Copyrights for third-party components of this work must be
honored. For all other uses, contact the Owner/Author.
Copyright is held by the owner/author(s).
KDD '16, August 13-17, 2016, San Francisco, CA, USA.
ACM 978-1-4503-4232-2/16/08.
http://dx.doi.org/10.1145/2939672.2945397
2135

Seide 2016

Uploaded by

Copyright:

Available Formats

Seide 2016

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Seide 2016

Uploaded by

Copyright:

Available Formats

CNTK: Microsoft's Open-Source Deep-Learning Toolkit

Frank Seide Amit Agarwal

You might also like