Running A stateful app on ECS
Although AWS ECS is not really designed to run stateful apps, sometimes you’ve got no choice. I ran into one of these situations recently where I wanted to run an IRC bot called Limnoria which didn’t rely on any standard database that I could get managed from any cloud provider like AWS. Instead, the data was stored in the filesystem in various flat-file formats. Obviously, it was not feasable to have the data lost across ECS instances, so I needed to find ’true’ persistent storage.
As it turns out, reasonably recently AWS released the ability to use custom drivers as docker backends. One of those backends is NFS which in turn allows us to mount an EFS filesystem (which are based an NFS) as a docker volume for persistent cross-ECS instances. As a bonus, doing so isn’t even too difficult.
Obviously, you’ll need to start out by creating an EFS filesystem. The only thing to check at this step is that it is created in the same VPC as your ECS cluster. Once created, make note of the volume url, and go to your EFS task definition and start adding a new volume.
Tick the “Specify a volume driver” button and put “local” into the driver textbox. Select “Task” as the scope, and then we can start setting the driver options. Add the first option with the key “type” and the value nfs
, add another with the key device
and the value is the url of the volume. Finally, add one with the key o
and the value addr=<filesystem address>,nfsvers=4.0,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2
Once done, you can mount this volume like any other ECS volume into a container and it all data written to it will be persistently stored on EFS, and will persist across ECS instanes. Furthermore, you can mount this to multiple running containers to share data (such as images uploaded to a site).