the life and times of Linux Containers

Tycho.Andersen @ Canonical .com

Who am I?

  • Functional programmer for a while
  • Canonical-er since mid 2013 working on Cloud Stuff
    • LXD core developer
    • LXC contributor
    • CRIU contributor
    • Openstack & related

What are containers?


How did we get here?

First there were namespaces

  • mount namespace in 2000, unshare(CLONE_NEWNS)
  • OpenVZ released in 2005
  • meiosis checkpoint & restore tool for *nixen
  • lxc2 work begins inside IBM around 2007
  • meiosis pid virtualization patch for Linux
  • UTS namespaces (allow each container to have its own hostname)
  • IPC namespaces (POSIX queues)
  • PID namespaces joint work between IBM & Parallels

Then there were CGroups

  • Google begin upstreaming CGroups in 2007
  • Called "process controllers"
  • Resource monitoring in addition to control
  • 2010 Kernel summit: agreed upon that there will be one Linux Container API

So... what are containers?

  • Containers are OS level virtualization
  • Containers are implemented with a collection of kernel APIs
  • Container engines use these APIs to create a container

Container Security

  • User namespaces: root inside the container is not root outside
  • SECCOMP, AppArmor, etc. also necessary
  • CAP_SYS_ADMIN and other capabilities
  • Better than virtualization? A short story.

What about migration?

Linus Torvalds on CRIU (kernel commit 09946950, 2012):

“A note on this: this is a project by various mad Russians to perform c/r mainly from userspace, with various oddball helper code added into the kernel where the need is demonstrated... However I'm less confident than the developers that it will all eventually work!”

CRIU History

  • Pre 2011: OpenVZ has some migration support in the kernel
  • 2011: CRIU born as a Parallels project
  • 2012: First patches get merged upstream
  • 2013: OpenVZ and mainline parity
  • 2014: LXC 1.1 released, lxc-checkpoint

Containers today


Containers today: a dichotomy

System containers

  • LXD
  • OpenVZ
  • Designed with full Linux "virtualization" in mind
    • Migration is a first class primitive
    • APIs for manipulating filesystems, running commands
  • /sbin/init

App containers

  • Docker
  • Rocket
  • Designed with apps in mind
    • (mostly) immutable filesystem
    • Dockerfile to set up app
  • /usr/bin/apache2


  • Announced in 2014
  • Based on Linux Containers (LXC)
  • Secure by default: user namespaces, cgroups, AppArmor, etc.
  • A REST API for managing system containers
  • A daemon that can do hypervisor-y things
  • A framework for managing container base images

Creating a container

wget --no-check-certificate --certificate=~/.config/lxc/client.crt --private-key=~/.config/lxc/client.key -O - -q --method=POST --body-data='{"name": "manual", ...}'


containers endpoint

wget --no-check-certificate --certificate=~/.config/lxc/cert.pem --private-key=~/.config/lxc/key.pem -O - -q

   {"name":"foo", "config":[], "profiles":[],
       "status":{"state":"RUNNING", ...}}

lxc move container otherhost:container

Container Migration




Thanks to James Bottomley for the container/vm drawings