Wednesday, 24 May 2017

How to change resource limit of a running docker container dynamically

Say we are running a docker container constraining it’s resource usage. After some time we discovered that the container needs more resource and we have to relax the resource limit for that container without stopping that container. 
I am writing two methods for changing the resource limit of a container.
For example we are running a container with a memory limit of 128MB
docker run -it -m 128m ubuntu /bin/bash
You can find all the information about the memory under /sys/fs/cgroup/memory/docker/<full container id>
We can get the full Container ID by running the docker ps command with --no-trunc option.
docker ps --no-trunc
Memory limit for a container can be found from the file
/sys/fs/cgroup/memory/docker/<container full id>/memory.limit_in_bytes
#cat /sys/fs/cgroup/memory/docker/b2bebfd78782ff345c92a6e44535e61d001187a2f15ce171679729eebfd7c327/memory.limit_in_bytes
We can check the memory utilization by the container using the docker stats command:
# docker stats b2bebfd78782
Let’s run stress tool in the container and check the utilization:
# stress --vm 1 --vm-bytes 512M
Checking the resource utilization again:
Although we specified 512MB in the stress command but as the container has a limit of 128MB RAM, so stress command is unable to get 512MB RAM and currently occupying full 128MB RAM.
Let’s increase the RAM to 1GB:
Method 1:
We can directly change the value of /sys/fs/cgroup/memory/docker/<container full id>/memory.limit_in_bytes to number of bytes, and this will change the memory limit to the value we want.
echo 1073741824 > /sys/fs/cgroup/memory/docker/b2bebfd78782ff345c92a6e44535e61d001187a2f15ce171679729eebfd7c327/memory.limit_in_bytes
Again we will check the memory utilization:
Yes, we can see that the memory limit has been increased to 1G
This change is temporary and once the container is restarted, it takes whatever memory setting was specified while container was created.
Method 2:
Another simple way is to change the resource limit is to use the docker update command. For example say we want to change the memory limit to 512MB:
# docker update b2bebfd78782 -m 512M
This will update the memory limit for the container permanently.
Usage: docker update CONTAINER [CONTAINER...]
Update configuration of one or more containers
--blkio-weight Block IO (relative weight), between 10 and 1000
-c, --cpu-shares CPU shares (relative weight)
--cpu-period Limit CPU CFS (Completely Fair Scheduler) period
--cpu-quota Limit CPU CFS (Completely Fair Scheduler) quota
--cpuset-cpus CPUs in which to allow execution (0-3, 0,1)
--cpuset-mems MEMs in which to allow execution (0-3, 0,1)
--help Print usage
--kernel-memory Kernel memory limit
-m, --memory Memory limit
--memory-reservation Memory soft limit
--memory-swap Swap limit equal to memory plus swap: '-1' to enable unlimited swap
--restart Restart policy to apply when a container exits

Friday, 19 May 2017

MongoDB Recipes: Disable replica set chaining

By default MongoDB allows replica set chaining. That means it allows a secondary member to sync from another secondary. Suppose we want our secondaries only to sync from the primary not from any other secondary. In that case we can disable replica set chaining.
  • Save replica set configuration in a variable:
    repl1:PRIMARY> cfg = rs.conf()
  • If settings sub-document is not present in the config, then add it:
    repl1:PRIMARY> cfg.settings = {}
  • Set the chainingAllowed property to false from default value true in the cfg variable
    repl1:PRIMARY> cfg.settings.chainingAllowed = false
  • Set the new configuration from cfg variable
    repl1:PRIMARY> rs.reconfig(cfg)
  • Check the new settings:
    repl1:PRIMARY> rs.conf()

Tuesday, 16 May 2017

Encrypting the shell scripts

Sometimes we need to encrypt a shell script for security reasons, for example if the script contains some sensitive information like password etc.
For this task I am going to use the shc tool ( to convert my text shell script file into a binary file . Download the source code of shc tool from the link and extract the GZIP compressed tar archive file. Here I am going to use the 3.8.9 version.
Note: I used Ubuntu 14.04 for this example.
If make is not installed, then install make
# apt-get install make
Go inside the shc-3.8.9 source folder.
# cd shc-3.8.9
# make

Now install shc
#make install
If installation fails with directory not found error, create the /usr/local/man/man1 directory and run the command again.

#mkdir /usr/local/man/man1
# make install

Remove the shc source folder after it is installed
# cd ..
# rm -rf shc-3.8.9/

Our shc tool is installed, we are now going to convert our shell script into binary.
Go to the folder where the shell script is stored. My script name is mysql_backup.
Create binary file of the shell script using the following command:
# shc -f mysql_backup
shc command creates 2 additional files
# ls -l mysql_backup*
-rwxrw-r-- 1 pranab pranab 149 Mar 27 01:09 mysql_backup
-rwx-wx--x 1 pranab pranab 11752 Mar 27 01:12 mysql_backup.x
-rw-rw-r-- 1 pranab pranab 10174 Mar 27 01:12 mysql_backup.x.c
mysql_backup is the original unencrypted shell script.
mysql_backup.x is the encrypted shell script in binary format.
mysql_backup.c is the C source code of the mysql_backup file. This C source code is compiled to create the above encrypted mysql_backup.x file.
We will remove the original shell script (mysql_backup) and c file (mysql_backup.x.c) and rename the binary file (mysql_backup.x) into the shell script (mysql_backup).
# rm -f  mysql_backup.x.c
# rm -f mysql_backup
# mv mysql_backup.x mysql_backup
Now we have our binary shell script, the contents of this file can not be easily seen as it is a binary file.