File Sharing with Docker Desktop

One of the topics that Docker Desktop users often ask us about is file sharing. How do I see my source code inside my container? What is the difference between size and anchor base? Why is file sharing slower than Linux, and how can I speed it up? In this blog post, I’ll cover your options, some tips and tricks, and finish with a quick preview of what we’re currently working on.

Hook mounts and sizes

Let’s say you run an Ubuntu container with docker run -it ubuntu bash. You will quickly find that (1) the container has its own file system, based on the file system in the Ubuntu image; (2) You can create, delete, and modify files, but your changes are local to the container and are lost when the container is deleted; (3) The container cannot access any other files on the host computer.

So the next natural questions are, how do I see my other files? And how can my container write data that I can read later and possibly use in other containers? This is where peg sizes and bindings come in. Both use the . extension -v flag for docker run To select some files to share with the container.

The first option most people encounter is tethering, where a portion of the local file system is shared with the container. For example, if you run

docker run -it -v /users/stephen:/my_files ubuntu bash

Then the files in /users/stephen It will be available at /my_files In the container, you can read and write them there. This is very simple and convenient, but if you are using Docker Desktop, a named volume may have better performance, for reasons that I will explain in the next section.

The second option, a named folder, is a file system managed by Docker. You can create a folder named with a command like docker volume create new_vol, then share it to the container using a file -v Teach again:

docker run -it -v new_vol:/my_files ubuntu bash

These volumes persist even after the container is deleted, and can be shared with other containers. You can also browse its contents from the Docker Desktop UI, using the Volumes tab we added recently (it’s now free for all users including Docker Personal).

performance considerations

To understand the performance differences between the options, we first have to talk briefly about how Docker Desktop works. Many people imagine Docker Desktop as just a user interface on top of some open source tools, but that’s only a small part of what it is. Docker Desktop is basically an environment for developing and running Linux containers on your Mac or Windows, with seamless integration with the host so that it looks like it’s running locally. It does this by setting up a Linux VM (or optionally a WSL 2 environment on Windows) to run your Docker drive and containers, and then passing CLI commands, networks, and files between the host and the virtual machine.

Unfortunately, it is in the nature of virtualization that there is always a small inevitable overhead in crossing the boundaries of the host VM. It’s only small, but in a development environment with a huge source tree and a lot of reads and writes, it adds up and can obviously affect performance. And because Docker Desktop does a good job of hiding the underlying virtual machine, it’s not clear why this happens. On Linux, the container has direct access to the file system installed on the tethering system, and because the implementation on Mac and Windows “looks native”, people would intuitively expect the same performance there.

Named folders don’t have the same problem because they are created inside the VM’s file system, so they’re as fast as Linux machines. In WSL 2, Windows manages file sharing rather than Docker manages it, but the same performance considerations apply: Files loaded from the Windows file system can be slow, and named volumes are fast, but in this case there is another option: files stored in the Windows file system Linux files are also inside the WSL VM, so it’s fast too.

Best practices and tips

This gives us the main tip in improving performance. It is convenient to use tie-down mounts initially, and you may find that they are suitable for your use case. But if performance becomes an issue, (1) make sure you only share what you want to share, and (2) consider what can be shared in a way other than linking. You have several options for keeping files inside the virtual machine, including a named folder, Linux files in WSL, and a container file system: which use depends on the use case. For example:

  • The source code that you are actively editing is convenient to use for syntax binding
  • Large, static dependency libraries or trees can be moved to a named folder, a WSL, or even stored in a container image.
  • More suitable databases in a folder named or WSL
  • Cache directories and log files should be placed in a named volume or WSL (if you need to keep them after a container has stopped) or in the container’s file system (if they can disappear when the container is gone).
  • Files that the container doesn’t need at all should not be shared. In particular, don’t share your entire home directory. We’ve seen some people do this on a regular basis so they always have access to whatever files they need, but unlike Linux it’s not “free”.

One of the remaining options if you really need an anchor for some large or high traffic directory is a third party caching/sync solution, for example Mutagen or docker-sync. These basically copy your files inside the virtual machine for quick read/write access, and handle synchronization (in one or both directions) between the copy and the host. But it does include an additional element of management, so named sizes are still preferred where possible.

the future

We’ve used a variety of file sharing apps over the years (Samba and gRPC FUSE on Windows Hyper-V; osxfs and gRPC FUSE on Mac; Windows 9P uses on WSL 2). We’ve made some performance improvements over time, but none of them have been able to match the original performance. But we are currently testing a very promising file sharing app based on the creative stuff. Virtiofs is a new technology designed specifically for sharing files between a host and a virtual machine. It is able to achieve significant performance gains by using the fact that the virtual machine and the host are running on the same machine, not across the network. In our experiments, we have seen some very promising results.

We’ve already released a preview of this technology for Docker Desktop for Mac, which you can get from our public roadmap (requires macOS 12.2), and we’re also planning to use it in our upcoming Docker Desktop for Linux. We think it would be able to make link loads a lot faster (although we still recommend named sizes or container filesystems for appropriate use cases). We’d love to hear your experiences about it.

Next steps

If you’d like to dig deeper on these topics, I recommend a talk given by one of Docker’s leaders, Jacob Howard, at DockerCon 2021 titled A hands-on tour of Docker file systems. Lots of great information and practical advice in just 26 minutes!

To follow the progress of our current work on creators, sign up for the ticket on our public roadmap. This is where we post preview builds, we’d love for you to try them out and give us your feedback.

Dukercon 2022

Join us at DockerCon2022 on Tuesday, May 10th. DockerCon is a free, one-day virtual event that is a unique experience for developers and development teams building the next generation of modern applications. If you want to learn how to move from code to the cloud quickly and solve your development challenges, DockerCon 2022 offers live content to help you build, share, and run your applications. Register today at https://www.docker.com/dockercon/

Leave a Comment