Developing with Notebooks

not related to computer laptops
(although you can run a Notebook system in a laptop)

Paulo Villegas
TEFCON, 2016

Contents

  1. The explanation
    A description/justification/exposition of Notebooks
  2. The examples
    Walkthrough & execution over a few notebook examples
  3. The proposal
    Initial sketch of how to improve knowledge sharing by using Notebooks

Context

Literate programming

an approach to programming in which a program is given as an explanation of the program logic in a natural language interspersed with snippets of macros and traditional source code, from which a compilable source code can be generated

Example: wcni.c

Reproducible research

data analyses, and more generally, scientific claims, are published with their data and software code so that others may verify the findings and build upon them.

Interactive code development

IDEs with 'shell-like' interfaces for interactively running the code and getting the results

Examples: RStudio, Matlab/Octave GUIs

REPL

read-eval-print loop

Examples

... and many more

Uses

  • instantaneous prototyping
  • mathematical calculation (scientific analysis)
  • interactive software maintenance
  • benchmarking
  • algorithm exploration
  • learning a language

What is a notebook?

Two things at the same time:

  1. A REPL to evaluate code in a given programming language
  2. An editor to create a storyline with reproducible code

The outcome is a rich document (text, graphics, formatting) that can be executed

Notebook history

  • The concept exists since the 80s. Arguably started with Mathematica
  • More recently, the rise of IPython as environment for scientific computation has caused a notebook fever

Notebook frameworks

Jupyter notebook

Origin and structure

  • a split from IPython 4.0 onwards
  • a server exposing notebook functionality as a Web application (allowing document edition and code execution via a Web browser)
  • pluggable kernels for different programming languages

Installation

Sharing

  • As notebook
    for interaction and easy reproduction and play around
  • A formatted version (e.g.) PDF
    for easier portability (nbconvert)
  • Online
    • a rendered & saved notebook can be directly visualized
    • also: Notebook viewer
    • also: GitHub automatically renders uploaded notebooks (see gallery)

Extensions

The Jupyter Notebook framework can be extended with additional functionality

Many notebook extensions (nbextensions) have already been published.

to the code!!

(asuming a local notebook server is running on port 8008)

The proposal

An envisioned workflow

Scene 1: developing

Scene 2: activating

Scene 3: indexing

Scene 4: reusing

the end