July 25, 2014

June 18-21, 2014
IEEE North American School of Information Theory
to be held at

The Bahen Centre, Room 1160 and
Fields Institute, 222 College St, Toronto

Map to venue


Steps 1 must be completed by March 7.
Required, Step 1. Application, must be completed at time of application
Required, Step 2. Submit Poster Abstract,
Optional, Step 3. NSF Scholarship

IEEE + IT Society Member $100
IEEE Member only $150
Other $200
Accommodations: $50 (for 4 nights June 17-21)
Video of Panel Discussion Tutorials Click here to join the IEEE+IT Society.
A student membership for one year costs $50
Confirmed Participants Information for participants
The IEEE North American School on Information Theory delivers interactive education for graduate students in the engineering and mathematical sciences, and the opportunity to meet with distinguished lecturers and peers in a supportive environment. Information Theory studies the transmission, processing, and use of information in many contexts; it draws from and contributes to a large number of disciplines in the mathematical sciences. The areas of study of participants are expected to include pure mathematics, theoretical computer science, and the engineering sciences. The types of research topics presented by students, and the tutorial areas discussed by lecturers, will span foundational theory, algorithmic design, and implementation in practice.

In its seventh year, this event is designed to provide graduate students with opportunities to:

  • Learn from distinguished lecturers in the field who will present long-format (2 1/2 hour) tutorials;
  • Participate in a stimulating and inviting forum of scientists;
  • Present their own work for feedback and potential collaboration;
  • Deepen their connections with the community

The 15th IEEE International Workshop on Signal Processing Advances in Wireless Communications (SPAWC 2014) will be held right after NASIT.


  • Robert Calderbank, Duke
  • Brendan Frey, Toronto
  • Andrea Goldsmith, Stanford
  • Alon Orlitsky, UCSD
  • Henry Pfister, Texas A&M
  • En-hui Yang, Waterloo

Poster Session

As part of NASIT, there will be a poster session each day. The idea of the sessions is to provide an informal and relaxed setting for students to interact with other students and faculty to get experience in presenting and to discuss their ideas and their research; students will be required to present a poster in one of the three sessions. We encourage more junior graduate students, who do not yet have original research to present at NASIT, to present a "tutorial-style" poster on some topic of interest to the student. To more advanced graduate students and post-docs we note that posters are not refereed, and that previously published results are completely acceptable.

In addition, the school will feature interactive panels and poster sessions, as well as social and technical student events.


Tutorial 1 (Wednesday June 18, AM)
Title: Information Theory and Learning: The Grace of Dimensionality
Speaker: Alon Orlitsky, UCSD

Information theory uses data to effectively perform various tasks. In these two lectures we apply it to the machine-learning tasks of estimation, prediction, and classification, over discrete and continuous domains. For discrete domains we show that these tasks require a number of samples that tends to infinity as the domain size increases, and that a simple and natural modification of the tasks requires only a constant number of samples for any domain size. For continuous domains we construct an intuitive spectral estimator that learns high-dimensional Gaussian mixtures with a near-optimal number of samples.
Based on joint work with Jayadev Acharya, Ashkan Jafarpour, and Ananda Theertha Suresh.

Bio: Alon Orlitsky received B.Sc. degrees in Mathematics and Electrical Engineering from Ben Gurion University in 1980 and 1981, and M.Sc. and Ph.D. degrees in Electrical Engineering from Stanford University in 1982 and 1986. From 1986 to 1996 he was with the Communications Analysis Research Department of Bell Laboratories. He spent the following year as a quantitative analyst at D.E. Shaw and Company, an investment firm in New York city. In 1997 he joined the University of California, San Diego, where he is currently a professor of Electrical and Computer Engineering and of Computer Science and Engineering, and directs the Information Theory and Applications Center and the Center for Wireless Communications. His research concerns information theory, statistical modeling, and machine learning. He is a recipient of the 1981 ITT International Fellowship and the 1992 IEEE W.R.G. Baker Paper Award, and co-recipient of the 2006 Information Theory Society Paper Award. He co-authored two papers for which his students received student-paper awards: the 2003 Capocelli Prize and the 2010 ISIT Student Paper Award. He is a fellow of the IEEE, and holds the Qucalcomm Chair for Information Theory and its Applications at UCSD.

Tutorial 2 (Wednesday June 18, PM)
Title: Source Coding Theory and Practice: Making and Connecting Dots
Speaker: En-hui Yang, University of Waterloo

Rene Descartes (1596--1650), the father of modern philosophy and analytic geometry, stated, ``it is necessary that at least once in your life you doubt, as far as possible, all things,'' and tried to use what is now known as ``the method of doubt'' to establish a firm, rational foundation for knowledge. To many, information and knowledge may be regarded as the two sides of a coin. As such, it would be interesting to see if the method of doubt can be used to advance information theory (IT) research as well.
Taking source coding as an example, we will illustrate in this lecture how to use the method of doubt to make and connect dots in source coding theory and practice. After reviewing Shannon's probabilistic approach to source coding, we will use the method of doubt to challenge assumptions made therein, derive the computational approach to both lossless and lossy compression, demonstrate the consistency between the computational approach and probabilistic approach whenever those assumptions are valid, and discuss the recent successful applications of the computational approach to image and video coding, and its impacts on the design of HEVC, the newest video coding standard, and future video coding standards. Using the method of doubt again, we would challenge the existing video coding paradigm, especially in mobile video, and present a new video coding paradigm dubbed causal video coding along with some counter intuitive IT research results. Finally, using the method of doubt, we would connect source coding with the display and management of Big Data and machine learning by pointing out some new IT problems and, if time permitting, demonstrating some of our practical solutions.

Bio: En-hui Yang has been with the Dept. of Electrical and Computer Engineering, University of Waterloo, Ontario, Canada since June 1997, where he is now a Professor and Canada Research Chair in information theory and multimedia compression. He is the founding director of the Leitch-University of Waterloo multimedia communications lab, and a co-founder of SlipStream Data Inc. (now a subsidiary of BlackBerry (formerly Research In Motion)). He currently also serves as an Executive Council Member of China Overseas Exchange Association and an Overseas Advisor for the Overseas Chinese Affairs Office of the City of Shanghai, and is sitting on the Overseas Expert Advisory Committee for the Overseas Chinese Affairs Office of the State Council of China and a Review Panel for the International Council for Science.

Tutorial 3: (Thursday June 19 AM)
Title: Wireless networks: dreams, challenges, and capacity limits
Speaker: Andrea Goldsmith, Stanford University

Wireless networks have fundamentally altered the way we live, work, and play. This talk focuses on the use of information theory to obtain fundamental capacity limits as well as design insights forthese networks. Building up from first principles, we will cover capacity analysis for single-user and multi-user channels as well as ad-hoc networks and cellular systems, including the impact of time-varying fading and ISI. We will also explore how fundamental capacity limits outside "asymptopia," i.e. under finite delay, energy, and sampling constraints,are obtained. The talk concludes by speculating onwhat the wireless future might look like and describing some of the innovations and insights that are required to realize this vision.

Bio: Andrea Goldsmith is the Stephen Harris professor in the School of Engineering and a professor of Electrical Engineering at Stanford University. She was previously on the faculty of Electrical Engineering at Caltech. She co-founded and served as CTO of Accelera, Inc. and of Quantenna Communications, Inc. and has also held industry positions at Maxim Technologies, Memorylink Corporation, and AT&T Bell Laboratories. Dr. Goldsmith is a Fellow of the IEEE and of Stanford, and she has received several awards for her work, including the IEEE Communications Society and Information Theory Society joint paper award, the IEEE Communications Society Best Tutorial Paper Award, the National Academy of Engineering Gilbreth Lecture Award, the IEEE ComSoc Communications Theory Technical Achievement Award, the IEEE ComSoc Wireless Communications Technical Achievement Award, the Alfred P. Sloan Fellowship, and the Silicon Valley/San Jose Business Journal's Women of Influence Award. She is author of the book ``Wireless Communications'' and co-author of the books ``MIMO Wireless Communications'' and "Principles of Cognitive Radio," all published by Cambridge University Press. She received the B.S., M.S. and Ph.D. degrees in Electrical Engineering from U.C. Berkeley.

Dr. Goldsmith has served as editor for the IEEE Transactions on Information Theory, the Journal on Foundations and Trends in Communications and Information Theory and in Networks, the IEEE Transactions on Communications, and the IEEE Wireless Communications Magazine as well as on the Steering Committee for the IEEE Transactions on Wireless Communications. She participates actively in committees and conference organization for the IEEE Information Theory and Communications Societies and has served on the Board of Governors for both societies. She has also been a Distinguished Lecturer for both societies, served as President of the IEEE Information Theory Society in 2009, founded and chaired the student committee of the IEEE Information Theory Society, and chaired the Emerging Technology Committee of the IEEE Communications Society. At Stanford she received the inaugural University Postdoc Mentoring Award and has been active in committees to innovate and revise both graduate and undergraduate education university-wide. She served as Chair of Stanford's Faculty Senate in 2009 and currently serves on its Faculty Senate and on its Budget Group.

Tutorial 4: (Thursday June 19 PM)
Title: Decoding the Genome
Speaker: Brendan Frey, University of Toronto

The digital code that controls the biochemistry of life is embedded within a sequence of DNA letters called a genome. Recently, scientists have figured out the code embedded in the ten thousand-letter HIV genome and have successfully modified the code in a one million-letter bacterial genome. However, we are only beginning to understand the human code, which resides in a three billion-letter genome and makes use of combinatorial complexity to generate hundreds of different cell types comprising trillions of cells in the human body. I will describe recent discoveries made by my colleagues and I, which shed light on how the human code works, how it is used to generate complex cell types, and how it differs from other species, such as chimpanzees. I will also describe how the human code can be used to understand the genetic causes of human disease and even to treat disease.

Bio: Dr. Brendan J Frey is a Professor at the University of Toronto, with appointments in Engineering and Medicine. He conducts research in the fields of genome biology and machine learning. Dr. Frey holds the Canada Research Chair in Biological Computation, and is a Fellow of the Canadian Institute for Advanced Research, the Institute of Electrical and Electronic Engineers and the American Institute for the Advancement of Science. He has received several distinctions, including the John C Polanyi Award, the EWR Steacie Fellowship and Canada's Top 40 Leaders Under 40 Award. Dr. Frey has consulted for several industrial research and development laboratories in Canada, the United States and England, and he is currently on the Technical Advisory Board of Microsoft Research. His former students and postdoctoral fellows include professors, industrial researchers and developers at universities and industrial laboratories from across Canada, the United States and Europe.

Tutorial 5 (Friday June 20 AM)
Title: A Brief Introduction to Spatially-Coupled Codes and Threshold Saturation
Speaker: Henry Pfister, Texas A&M

It is well-known that belief-propagation (BP) decoding of low-density parity-check (LDPC) codes is suboptimal and that the noise threshold of maximum-a-posteriori (MAP) decoding can be larger than that of BP decoding. Recently, Kudekar et al. proved that regular LDPC ensembles can be spatially coupled (SC) so that the BP noise threshold saturates to the MAP noise threshold of the original ensemble. These SC ensembles are instances of LDPC convolutional (LDPCC) codes and the new proof explains an earlier observation by Lentmaier et al. that terminated LDPCC codes allow reliable communication at rates approaching capacity.

The goal of this lecture is to provide a tutorial introduction to these recent advances in coding theory. A short review of LDPC codes and density evolution will be followed by an introduction to SC codes. Next, some standard communication problems will be used to highlight the benefits of SC codes. Finally, the phenomenon of threshold saturation will be discussed and an overview of it.

This lecture is based on joint research with Yung-Yih Jian, Santhosh Kumar, Krishna R. Narayanan, Phong S. Nguyen, and Arvind Yedla.

Henry D. Pfister received his Ph.D. in electrical engineering from UCSD in 2003 and joined the faculty of the School of Engineering at Texas A&M University in 2006, where he is currently an associate professor. Prior to that he spent two years in R&D at Qualcomm, Inc. and one year as a post-doc at EPFL. He received the NSF Career Award in 2008, the Texas A&M ECE Department Outstanding Professor Award in 2010, and was a coauthor of the 2007 IEEE COMSOC best paper in Signal Processing and Coding for Data Storage. He is currently an associate editor in coding theory for the IEEE Transactions on Information Theory.
His current research interests include information theory, communications, and probabilistic graphical models. In particular, he is interested in problems related to sensing (e.g., compressed sensing), communication (e.g., coding for single and multiuser channels), storage (e.g., capacity of recording channels), and inference (e.g., collaborative filtering and learning graphical models).

Tutorial 5 (Saturday June 21 AM)
Title: The Art of Measurement
Speaker: Robert Calderbank

The central goal of compressed sensing is to capture attributes of a signal using very few measurements. In most work to date this broader objective is exemplified by the important special case of classification or reconstruction from a small number of linear measurements. In the first part of this talk we use wireless information theory to derive fundamental limits on compressive classification, on the maximum number of classes that can be discriminated with low probability of error and on the tradeoff between the number of classes and the probability of misclassification.

Next we describe how to use information theory to guide the design of linear measurements by maximizing mutual information between the measurements and the statistics of the source.

Next we introduce the Weyl transform as a powerful framework for representing measurement data. Transform coefficients are connected to the Walsh-Hadamard transform of multiscale autocorrelations, and different forms of dyadic periodicity in a signal are shown to appear as different features in its Weyl coefficients. Finally we demonstrate the effectiveness of the Weyl transform through the example of textured image classification.

Bio: Robert Calderbank is Professor of Electrical Engineering and Director of the Information Initiative at Duke University. Prior to joining Duke as Dean of Natural Sciences in 2010, he directed the Program in Applied and Computational Mathematics at Princeton University. Prior to joining Princeton in 2004 he was Vice President for Research at AT&T, in charge of what may have been the first industrial research lab where the primary focus was Big Data.

Professor Calderbank is well known for contributions to voiceband modem technology, to quantum information theory, and for co-invention of space-time codes for wireless communication. His research papers have been cited more than 30,000 times and his inventions are found in billions of consumer devices. Professor Calderbank was elected to the National Academy of Engineering in 2005 and has received a number of awards, most recently the 2013 IEEE Hamming Medal for his contributions to information transmission.


Map to Venues
View NASIT Summer School map


Stark C. Draper (Toronto)
Warren Gross (McGill)
Ashish Khisti (Toronto)
Patrick Mitran (Waterloo)
Serdar Yüksel (Queen's)

For the 2013 school visit