Morning Musings

I'm not ready to wake up yet...

Squashed Truecrypt Archive

| Comments

This post presents a script that can create a squashfs filesystem inside of a truecrypt container.
This has many benefits over encrypted zip files as well as normal truecrypt containers:

  • Resulting file can be mounted and accessed directly
    • No need to “unzip” to the hard drive
    • No chance for leaking unencrypted data to the hard drive
  • Achieve both good compression as well as strong encryption
    • Better compression ratio than NTFS, BTRFS, GZIP, BZIP2
  • Truecrypt container is only as large as it needs to be
    • No need to guess the approximate size of the compressed result before compressing
  • Resulting file is immutable
    • Making changes is still possible with an easy workaround, described below

First, the script. It is named star, for “Squashed Truecrypt ARchive”. You will need the following to run it:

  • squashfs-tools
  • truecrypt
  • aufs-tools
(star) download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
#!/bin/bash

# star
# Copyright (C) 2015 Joe Ruether jrruethe@gmail.com
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.

# Creates a squashed truecrypt archive (star)

# $1 = Directory to star

# Stop on any error
set -e

# Declare an array of tasks to perform on exit
declare -a on_exit_items

# This function is run on exit
function on_exit()
{
    for i in "${on_exit_items[@]}"
    do
        eval $i
    done
}

# Add to the list of tasks to run on exit
function add_on_exit()
{
    local n=${#on_exit_items[*]}
    on_exit_items[$n]="$*"
    if [[ $n -eq 0 ]]; then
        trap on_exit EXIT
    fi
}

# Make sure enough arguments were specified
if [ -z $1 ]; then
   echo "Usage: $0 <directory> [name]"
   exit 1
fi

# Make sure the first argument is an existing directory
if [ ! -d $1 ]; then
   echo "Directory does not exist: $1"
   exit 1
fi

# Save some variables
directory=${1%/}
name=${2-$directory}
sfs=$name.sfs
star=$name.star

# Get the password
echo -n 'Enter the password to use:'
read -s password
echo
echo -n 'Repeat the password:'
read -s password_repeat
echo

# Check that the passwords match
if [ "$password" != "$password_repeat" ]; then
   echo 'Passwords do not match'
   exit 1
fi

# Squash the directory
mksquashfs $directory $sfs -noappend -b 1048576 > /dev/null 2>&1

# Clean this file up on exit
add_on_exit shred -f $sfs

# Get the size of the squashfs, add 512 blocks of size 512
sfssize=`stat --printf="%s" $sfs`
starsize=`expr $sfssize + 262144`

# Make a truecrypt volume
sudo truecrypt -t --non-interactive -c $star --size=$starsize --filesystem=none --encryption=AES --hash=SHA-512 --password=$password > /dev/null 2>&1

# Mount the truecrypt volume
sudo truecrypt -t --non-interactive --filesystem=none --password=$password $star > /dev/null 2>&1

# Get the truecrypt device name
starloc=`readlink -f $star`
devname=`truecrypt -t --non-interactive -l | grep $starloc | awk '{print $3}'`

# Unmount the volume on exit
add_on_exit sudo truecrypt -t --non-interactive -d $starloc > /dev/null 2>&1

# Copy the squashfs into the truecrypt volume
sudo dd if=$sfs of=$devname bs=64K > /dev/null 2>&1

# Set permissions
sudo chmod 755 $star

# Synchronize the filesystem
sync

echo 'Success'

Here is an example of how to use it:

$ mkdir secret_stuff
$ dd if=/dev/urandom bs=1MB count=10 > secret_stuff/secret_data.bin
10+0 records in
10+0 records out
10000000 bytes (10 MB) copied, 0.741577 s, 13.5 MB/s

$ ls
secret_stuff  star

$ ./star 
Usage: ./star <directory> [name]

$ ./star secret_stuff
Enter the password to use:
Repeat the password:
Success

$ ls
secret_stuff  secret_stuff.star  star

$ mkdir mnt
$ sudo truecrypt -t secret_stuff.star 
Enter mount directory [default]: mnt
Enter password for /home/joe/Downloads/temp/secret_stuff.star: 
Enter keyfile [none]: 
Protect hidden volume (if any)? (y=Yes/n=No) [No]: 

$ ls mnt/
secret_data.bin

$ sha256sum secret_stuff/secret_data.bin 
5342d4e85a221df35c5beda80e7b93b609fca732b908b6fd43febfcc89c324ea  secret_stuff/secret_data.bin
$ sha256sum mnt/secret_data.bin 
5342d4e85a221df35c5beda80e7b93b609fca732b908b6fd43febfcc89c324ea  mnt/secret_data.bin

The resulting file is immutable. This may seem like a downside at first, but it can be beneficial. For example, it can be mounted by multiple users simultaneously when shared via Dropbox or Bittorrent Sync, and it is easy to version control.

Making secure modifications to the archive is possible because the archive allows direct access via mounting. This is something that cannot be easily done with a normal zip file or tarball. By using a tmpfs filesystem as a writable aufs layer on top of the archive, edits can be made in memory that never touch the hard drive, so your encrypted data stays secure. Then, a new archive can be created from that tmpfs layer.

Layers can be kept separate and treated as diffs (similar to how Docker containers operate), or they can be “resquashed” together for maximum compression. It all depends on the user’s needs.

Here is a script that mounts a tmpfs aufs layer on top of the archive to allow edits.

(mount_star.sh) download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
#!/bin/bash

# mount_star.sh
# Copyright (C) 2015 Joe Ruether jrruethe@gmail.com
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.

# Mounts a squashed truecrypt archive

# $1 = Star to mount

# Stop on any error
set -e

# Declare an array of tasks to perform on exit
declare -a on_exit_items

# This function is run on exit
function on_exit()
{
    for i in "${on_exit_items[@]}"
    do
        eval $i
    done
}

# Add to the list of tasks to run on exit
function add_on_exit_reverse()
{
    on_exit_items=("$*" "${on_exit_items[@]}")
    if [[ $n -eq 0 ]]; then
        trap on_exit EXIT
    fi
}

# Make sure enough arguments were specified
if [ -z $1 ]; then
   echo "Usage: $0 <*.star>"
   exit 1
fi

# Make sure the first argument is an existing file
if [ ! -f $1 ]; then
   echo "File does not exist: $1"
   exit 1
fi

# Save some variables
star_file=$1
star_name=${star_file%.star}
star_old=${star_name}_old
star_changes=${star_name}_changes
star_new=${star_name}_new

# Get the password
echo -n 'Enter the password:'
read -s password
echo

# Make the directories
mkdir -p $star_old
add_on_exit_reverse rmdir $star_old

mkdir -p $star_changes
add_on_exit_reverse rmdir $star_changes

mkdir -p $star_new
add_on_exit_reverse rmdir $star_new

# Mount the star file
sudo truecrypt -t --non-interactive --password=$password $star_file $star_old
add_on_exit_reverse sudo truecrypt -t --non-interactive -d $star_old > /dev/null 2>&1

# Mount the tmpfs
sudo mount -t tmpfs tmpfs $star_changes
add_on_exit_reverse sudo umount -lf $star_changes

# Mount the aufs
sudo mount -t aufs -o dirs=$star_changes=rw:$star_old=ro aufs $star_new
add_on_exit_reverse sudo umount -lf $star_new

# Wait for user to continue
echo "Star is now mounted. Press Enter to unmount and exit"
read

Use it like so:

$ ./mount_star.sh 
Usage: ./mount_star.sh <*.star>

$ ./mount_star.sh ./secret_stuff.star 
Enter the password:
Star is now mounted. Press Enter to unmount and exit

Now in another terminal, you can interact with the mounted volumes:

$ ls
mount_star.sh  secret_stuff_changes  secret_stuff_old  starsecret_stuff_new  secret_stuff.star

$ touch secret_stuff_old/lala
touch: cannot touch ‘secret_stuff_old/lala’: Read-only file system

$ touch secret_stuff_new/lala

$ ls secret_stuff_new
lala  secret_data.bin

$ ls secret_stuff_changes/
lala

Creating a new archive is as simple as star’ing the name_new branch. Creating a “patch” is as simple as star’ing the name_changes branch:

$ dd if=/dev/urandom bs=1MB count=10 > secret_stuff_new/new_secret_data.bin10+0 records in
10+0 records out
10000000 bytes (10 MB) copied, 0.756202 s, 13.2 MB/s
$ ls secret_stuff*
secret_stuff.star

secret_stuff_changes:
lala  new_secret_data.bin

secret_stuff_new:
lala  new_secret_data.bin  secret_data.bin

secret_stuff_old:
secret_data.bin

$ ./star secret_stuff_changes/
Enter the password to use:
Repeat the password:
Success

$ ls
mount_star.sh  secret_stuff_changes.star  secret_stuff.star  star

Patch archives can be applied using aufs layers with the following script:

(patch_star.sh) download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
#!/bin/bash

# patch_star.sh
# Copyright (C) 2015 Joe Ruether jrruethe@gmail.com
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.

# Mounts a squashed truecrypt archive

# $1 = Star to mount

# Stop on any error
set -e

# Declare an array of tasks to perform on exit
declare -a on_exit_items

# This function is run on exit
function on_exit()
{
    for i in "${on_exit_items[@]}"
    do
        eval $i
    done
}

# Add to the list of tasks to run on exit
function add_on_exit_reverse()
{
    on_exit_items=("$*" "${on_exit_items[@]}")
    if [[ $n -eq 0 ]]; then
        trap on_exit EXIT
    fi
}

# Make sure enough arguments were specified
if [ -z $1 ]; then
   echo "Usage: $0 <base *.star> <patch *.star>"
   exit 1
fi

# Make sure the first argument is an existing file
if [ ! -f $1 ]; then
   echo "File does not exist: $1"
   exit 1
fi

# Make sure the second argument is an existing file
if [ ! -f $2 ]; then
   echo "File does not exist: $2"
   exit 1
fi

# Save some variables
base_star=$1
base_name=${base_star%.star}

# Patch directory will be hidden
patch_star=$2
patch_name=.${patch_star%.star}

# Get the passwords
echo -n "Enter the password for $1 :"
read -s base_password
echo
echo -n "Enter the password for $2 :"
read -s patch_password
echo

# Make the directories
mkdir -p $base_name
add_on_exit_reverse rmdir $base_name

mkdir -p $patch_name
add_on_exit_reverse rmdir $patch_name

# Mount the base
sudo truecrypt -t --non-interactive --password=$base_password $base_star $base_name
add_on_exit_reverse sudo truecrypt -t --non-interactive -d $base_name > /dev/null 2>&1

# Mount the patch
sudo truecrypt -t --non-interactive --password=$patch_password $patch_star $patch_name
add_on_exit_reverse sudo truecrypt -t --non-interactive -d $patch_name > /dev/null 2>&1

# Apply the patch
sudo mount -t aufs -o dirs=$patch_name=ro:$base_name=ro aufs $base_name
add_on_exit_reverse sudo umount -lf $base_name

# Wait for user to continue
echo "Star is now mounted and patched. Press Enter to unmount and exit"
read

Here is how it works:

$ ./patch_star.sh 
Usage: ./patch_star.sh <base *.star> <patch *.star>

$ ./patch_star.sh secret_stuff.star secret_stuff_changes.star 
Enter the password for secret_stuff.star :
Enter the password for secret_stuff_changes.star :
Star is now mounted and patched. Press Enter to unmount and exit

ls
mount_star.sh  patch_star.sh  secret_stuff  secret_stuff_changes.star  secret_stuff.star  star
$ ls secret_stuff
lala  new_secret_data.bin  secret_data.bin

This can be very handy for transferring small edits to a large archive across the network in a secure manner, without needing to retransfer the whole archive. The layering effect also acts as a poor man’s version control. In the future, I would like to expand on this idea and write a script to manage the layers more convieniently and effectively, perhaps in a manner similar to Git or Docker.

Comments