--- title: ROS Development with LXD description: Linux containers allow for easy isolation of developer environments. If you’re often working with a bunch of different ROS versions, it’s a lot easier to do your work entirely in containers. Getting Started You’ll first need to install LXD using snap. Note Throughout this guide I will be using the hostname to distinguish which machine […] url: https://canonical.com/blog/ros-development-with-lxd?format=md --- 1. [Blog](https://canonical.com/blog) 2. Article --- [Ted Kern](https://canonical.com/blog/author/tkern "More about Ted Kern") 6 March 2020 # ROS Development with LXD [LXC](https://canonical.com/blog/tag/lxc) [LXD](https://canonical.com/blog/tag/lxd) [robotics](https://canonical.com/blog/tag/robotics) [ROS](https://canonical.com/blog/tag/ros) [ROS2](https://canonical.com/blog/tag/ros2) --- Share the article Linux containers allow for easy isolation of developer environments. If you’re often working with a bunch of different ROS versions, it’s a lot easier to do your work entirely in containers. ## Getting Started You’ll first need to install LXD using snap. ``` ubuntu@lxhost:~$ sudo snap install lxd lxd 3.21 from Canonical✓ installed ``` > Note > > Throughout this guide I will be using the hostname to distinguish which machine I’m running commands on. > `lxhost` represents the bare metal machine you’ll be creating containers in. > `ros1-live` is the container we’ll be creating later > `remote-pc` is a different machine on the same LAN as lxhost > Pay attention to the value after the `@` in the shell prompts to make sure you run commands on the right machine. If you haven’t added `/snap/bin` to your PATH yet, you’ll want to in order to call the programs snap installs. ``` ubuntu@lxhost:~$ export PATH=/snap/bin:$PATH ``` You’ll need to initialize LXD before your first run. I’ll be using default settings for this example, but you should look into the settings to make sure of what you’re getting into. The exact printout may vary depending on your system, as well as what version of LXD you’re using ``` ubuntu@lxhost:~$ lxd init Would you like to use LXD clustering? (yes/no) [default=no]: Do you want to configure a new storage pool? (yes/no) [default=yes]: Name of the new storage pool [default=default]: Would you like to connect to a MAAS server? (yes/no) [default=no]: Would you like to create a new local network bridge? (yes/no) [default=yes]: What should the new bridge be called? [default=lxdbr0]: What IPv4 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: What IPv6 address should be used? (CIDR subnet notation, “auto” or “none”) [default=auto]: Would you like LXD to be available over the network? (yes/no) [default=no]: Would you like stale cached images to be updated automatically? (yes/no) [default=yes] Would you like a YAML "lxd init" preseed to be printed? (yes/no) [default=no]: ubuntu@lxhost:~$ ``` This handled setting a number of configuration options for us, as well as creating a storage pool for any containers we make and a network bridge (named `lxdbr0` above) that will connect our containers to the network. ### Heads Up LXD runs as a daemon on your system that is accessible to `root` and the `lxd` group on your system. Users with access to LXD can attach host devices and filesystems, presenting a security risk. Only add users you’d trust with root access to lxd. You can read more about LXD security in its [documentation](https://linuxcontainers.org/lxd/docs/master/security). ## The LXC command LXD provides an `lxc` program you’ll use to actually manage the containers. You *can* invoke this with `lxd.lxc`, but lxc works just fine. Make sure your PATH resolves to the correct lxc. ``` ubuntu@lxhost:~$ which lxc /snap/bin/lxc ``` If you installed lxc via `apt` before, and your `/usr/bin` directory comes before `/snap/bin` in PATH, you’ll run the wrong lxc command. You probably should go ahead and uninstall that lxc. LXC provides a number of verbs: ``` ubuntu@lxhost:~$ lxc alias copy file info manpage operation publish restart start cluster delete help init monitor pause query restore stop config exec image launch move profile remote shell storage console export import list network project rename snapshot version ``` (Output from tab completion) We’ll go through some of these, but you can always use `lxc --help` to get more details (or `man lxc.`, but those are currently just autogenerated from the help text). ## Creating a container We’ll start with a ROS Melodic container. I’m currently running Ubuntu 20.04 Focal preview, so this is actually the easiest way in my case to work with an older ROS. ``` ubuntu@lxhost:~$ lxc launch ubuntu:18.04 ros1-live Creating ros1-live Starting ros1-live ... ``` Just like that, we’ve spun up a container. An Ubuntu 18.04 machine named ros1-live is now available. `lxc list` will show it. ``` ubuntu@lxhost:~$ lxc list +---------------+---------+-----------------------+-----------------+-----------+-----------+ | NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS | +---------------+---------+-----------------------+-----------------+-----------+-----------+ | ros1-live | RUNNING | 10.141.221.65 (eth0) | | CONTAINER | 0 | +---------------+---------+-----------------------+-----------------+-----------+-----------+ ``` To work in it, we’ll use `lxc exec` to run a login shell as the default ubuntu user: ``` ubuntu@lxhost:~$ lxc exec ros1-live -- sudo -iu ubuntu bash ubuntu@ros1-live:~$ ``` ## SSH Access Depending on how we like to develop, we may prefer to ssh in (for instance, to use vscode’s remote tools). We’ll need to add our keys to the container to do so. If our public key is stored at `~/.ssh/id_rsa.pub`, we can add it in a number of ways. Here we’ll simply create an authorized\_keys file with our public key in it, and push it ``` ubuntu@lxhost:~$ cp ~/.ssh/id_rsa.pub /tmp/authorized_keys ubuntu@lxhost:~$ lxc file push /tmp/authorized_keys ros1-live/home/ubuntu/.ssh/authorized_keys -p ``` The `-p` argument creates parents, as per `mkdir -p`. We can now ssh into our container from the host and other containers. You can get the ip address using `lxc list` ``` ubuntu@lxhost:~$ lxc list +---------------+---------+-----------------------+-----------------+-----------+-----------+ | NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS | +---------------+---------+-----------------------+-----------------+-----------+-----------+ | ros1-live | RUNNING | 10.141.221.65 (eth0) | | CONTAINER | 0 | +---------------+---------+-----------------------+-----------------+-----------+-----------+ ubuntu@lxhost:~$ ssh ubuntu@10.141.221.65 ubuntu@ros1-live:~$ ``` You may want to add this address to your `.ssh/config` file ``` Host ros1-live HostName 10.141.221.65 User ubuntu IdentityFile ~/.ssh/id_rsa ForwardAgent Yes ForwardX11 Yes ``` ### Remote Access If you’d like to access the container from a remote pc, the default bridged network setup makes things tricky, as the containers are behind a NAT on the host. The easiest way to ssh in is to use the host as a jumphost. ``` ubuntu@remote-pc:~$ ssh -J ros1-live ``` or in the config file ``` ubuntu@remote-pc:~$ cat ~/.ssh/config Host ros1-live HostName 10.141.221.65 User ubuntu ProxyJump IdentityFile ~/.ssh/id_rsa ForwardAgent Yes ForwardX11 Yes ubuntu@remote-pc:~$ ssh ros1-live ubuntu@ros1-live:~$ ``` If your remote PC is running Windows, you’re likely using a version of ssh without the ProxyJump command. I recommend downloading the [latest release of OpenSSH](https://github.com/PowerShell/openssh-portable/releases) and putting it at the front of your Windows PATH variable, so VSCode and other tools can use it. If you need more direct access, you can add an lxc proxy device to the container to forward a port into it ``` ubuntu@lxhost:~$ lxc config device add rosl-live proxy22 proxy connect=tcp:127.0.0.1:22 listen=tcp:0.0.0.0:2222 ubuntu@lxhost:~$ lxc config device show ros1-live proxy22: connect: tcp:127.0.0.1:22 listen: tcp:0.0.0.0:2222 type: proxy ``` This will listen on the host port 2222 and forward connections to the container’s port 22. You can learn more about proxy devices [here](https://linuxcontainers.org/lxd/docs/master/instances#type-proxy). If you don’t want the container behind a NAT on the host, you can specify a different bridge configuration or add an IPVLAN or MACVLAN `nic` network device to the container. You can read more about `nic` devices [here](https://linuxcontainers.org/lxd/docs/master/instances#type-nic). ## Sharing Files Transparently You can neatly share directories between the host and the container using `disk` devices. Disk devices can be a bind-mount of an existing file or directory, a regular mount of a block device, or one of several other source types. You can read about `disk` devices [here](https://linuxcontainers.org/lxd/docs/master/instances#type-nic). Lets create a directory and share it with the container: ``` ubuntu@lxhost:~$ mkdir share && cd share ubuntu@lxhost:~/share$ mkdir ros1-live ubuntu@lxhost:~/share$ lxc config device add ros1-live share disk source=~/share/ros1-live path=/home/ubuntu/share ``` In order to truly share access, we’ll want to use the `raw.idmap` config option for the container to map your UID and GID. Assuming your UID and GID on the host is 1000 (the default for a single user Ubuntu installation), you’ll use the following command to set the option: ``` ubuntu@lxhost:~$ lxc config set ros1-live raw.idmap "both 1000 1000" ``` Now any files you make and modify in the host and the container will be indistinguishable, permission-wise. To verify the directory is shared: ``` ubuntu@lxhost:~$ ssh ros1-live ubuntu@ros1-live:~$ ls share ``` ## Set up ROS For once, setting up ROS will be the easy part. Just follow the instructions at the [ROS wiki](http://wiki.ros.org/melodic/Installation/Ubuntu) like normal. ## Developing in the Container Create a `workspace` directory in the share directory, and use it as normal to develop. ### Devices If you intend to interface with actual hardware, you’ll need to attach devices to your container. I’ve linked to the `device` configuration option several times before, but you’ll want to look at it in depth for configuring your hardware. For example, if you’re controlling an OpenManipulatorX via a U2D2, you’ll need to communicate via serial. This is done via a Unix character device at `/dev/ttyUSB0` or similar. To add it to the container as a [unix-char device](https://linuxcontainers.org/lxd/docs/master/instances#type-unix-char), use ``` ubuntu@lxhost:~$ lxc config device add ros1-live ttyusb0 unix-char source=/dev/ttyUSB0 path=/dev/ttyUSB0 ``` You can also forward the entire usb device using the vendorid and productid of the device as you would for a udev rule. See the entry on [usb devices](https://linuxcontainers.org/lxd/docs/master/instances#type-usb). ### GUIs You may notice GUI programs don’t work, either well or at all, in containers. Simos Xenitellis wrote a [wonderful guide](https://blog.simos.info/how-to-easily-run-graphics-accelerated-gui-apps-in-lxd-containers-on-your-ubuntu-desktop/) on how to fix this and you’ll be able to run Gazebo, RViz, et al in your containers. Create a new `profile` for your containers as follows: (**Note**: this will use vim, if you do not know vim, use the alternative further below): ``` ubuntu@lxhost:~$ lxc profile create gui ubuntu@lxhost:~$ lxc profile edit gui ``` and add the following to the config file that opened in vim ``` config: environment.DISPLAY: :0 raw.idmap: both 1000 1000 user.user-data: | #cloud-config runcmd: - 'sed -i "s/; enable-shm = yes/enable-shm = no/g" /etc/pulse/client.conf' - 'echo export PULSE_SERVER=unix:/tmp/.pulse-native | tee --append /home/ubuntu/.profile' packages: - x11-apps - mesa-utils - pulseaudio description: GUI LXD profile devices: PASocket: path: /tmp/.pulse-native source: /run/user/1000/pulse/native type: disk X0: path: /tmp/.X11-unix/X0 source: /tmp/.X11-unix/X0 type: disk mygpu: type: gpu name: gui used_by: ``` Alternatively, if you do not know or like touching vim, save the above to a temporary file, say, `/tmp/foo.yaml`, and do the following instead: ``` ubuntu@lxhost:~$ lxc profile create gui ubuntu@lxhost:~$ lxc profile edit gui < /tmp/foo.yaml ``` (see documentation on profiles [here](https://linuxcontainers.org/lxd/docs/master/profiles)). To summarize the above, we tell lxc that every device with a `gui` profile on it should * Do the `raw.idmap` config from earlier. * Set `DISPLAY=:0` * On cloud-init, disable shm in `/etc/pulse/client.conf` * Set the pulse audio server to a socket `/tmp/.pulse_native` * Install `x11-apps`, `mesa-utils`, and `pulseaudio` * Mount the pulseaudio socket from the host to to `/tmp/.pulse_native` * Mount the X11 socket * Mount your gpu We apply this profile to the existing container using: ``` ubuntu@lxhost:~$ lxc profile add ros1-live gui ``` Which adds the `gui` profile alongside the already applied `default` profile to the container. Restart the container using: ``` ubuntu@lxhost:~$ lxc restart ros1-live ``` And after ~30 seconds of downloading and installing the packages, the container should be up and ready to use. You can get GUI apps over ssh with X11 forwarding, or just from the `lxc exec` method of getting a shell. *This article originally appeared on [Ted Kern’s blog.](https://www.arnatious.com/blog/lxc/)* ## Sign up for our newsletter Get the latest Canonical news and updates in your inbox. Work email: \*I agree to receive information about Canonical's products and services. By submitting this form, I confirm that I have read and agree to [Canonical's Privacy Policy](https://canonical.com/legal/dataprivacy). Sign up ## Share on --- ## Related posts [### Extending ROS Noetic Support with ESM-Enabled Content Snaps](https://canonical.com/blog/extending-ros-noetic-support-with-esm-enabled-content-snaps) Canonical has now extended its ESM (Expanded Security Maintenance) for ROS coverage to ROS Noetic content-sharing snaps. With ESM for ROS now available in both deb and snap... [Florencia Cabral Berenfus](https://canonical.com/blog/author/flor-cabral) 17 December 2025 [### Discover your fully open source robotics observability at ROSCon 2025](https://canonical.com/blog/roscon-2025) Another year, another ROSCon! This year we’re heading to Singapore, and Canonical is once again thrilled to sponsor this important community event. Just like last year in... [Gabriel Aguiar Noury](https://canonical.com/blog/author/g-aguiar-noury) 22 October 2025 [### Canonical is now a platinum member in the Open Source Robotics Alliance](https://canonical.com/blog/osra-membership) Ubuntu is the home of ROS. The very first ROS distribution, Box Turtle, launched on Ubuntu 8.04 LTS, Hardy Heron, and since then, Ubuntu and ROS have grown hand in hand. With... [Gabriel Aguiar Noury](https://canonical.com/blog/author/g-aguiar-noury) 20 August 2025 [### ROS Noetic is EOL – take action to maintain fleet security](https://canonical.com/blog/ros-noetic-is-eol-take-action-to-maintain-fleet-security) As of May 2025, the Robot Operating System (ROS) Noetic Ninjemys officially reached its end of life (EOL). First released in 2020 as the final ROS (1) distribution, ROS Noetic... [Florencia Cabral Berenfus](https://canonical.com/blog/author/flor-cabral) 8 August 2025