Projects Meta Quick Hacks About

TilePaper: Generate OS/X "Shifting Tiles" style wallpapers

Nov 22, 2015

On all my Mac’s I have a folder of pictures, for the screensaver I used the tiled images option which generates a really nice scrolly thing with all the images on which I really like, but my desktop background is just a rotation of those same images.

Now having the images full size as my wallpaper is fine, but I’d much rather see multiple images in the same screen, so I threw together this quick project to generate those.

TL;DR it looks something like this

Example

And you can get the code on Github here

Terms

There are a few terms I use when talking about this when I think about how it works, so for simplicity I’ll explain them here so I don’t loose people;

  • Grid - A single output image comprised of multiple tiles
  • Tile - A single image on a grid

How it works

So when I started thinking about how to implement this, I was wondering how to generate an “algorithm” to generate the tiles. I spent a bit of time watching the way my Apple TV did it and thought about a few of the key components I wanted;

  • Correctly sizing portrait and landscape photos with grow-cropping to the tile size
  • Generating multiple tiles with the same image in to make it more interesting
  • Random large/small images, with some images taking up 2x the tile size

I realized there would be a little bit of randomness needed to get nice results, when thinking about how to architect the application there were 4 main classes I could think of

  • Grid - Hold the structure for a single grid, understand the positions and generate the layout
  • Image - Understand the actual source images, deal with abstracting some of the PIL stuff
  • Render - Create the output JPEG laying out the images as per the Grid
  • Tiler - General helper classes for loading the images and generating the tiles based on the config

I think the Grid is probably the most interesting part, since it contains most of the logic for making things look nice. The flow is actually relatively simple;

First, the Tiler passes a list of Images to Grid, along with useful grid properties like the size (the Grid doesn’t actually care about the output size as that’s dealt with by Render, or the source image size beyond if their portrait or landscape, which it can find by asking Image).

Then it shuffles the list of Images, so we get a nice random selection. The Tiler deals with only passing us images that have been used less than the configured inclusions, so Grid can use any of the images passed to it.

Then, we basically just iterate through every Image, try to place it on the grid. There’s a bit of a hack I used to improve the positioning and reduce the number of times we generate a grid with blank spaces, in the getImageSize function in Grid, 20% of the time a tile will be enlarged to take up twice the space.

def getImageSize(self, im, small=False):
    if im.portrait:
        height = 2
        width = 1
    else:
        if random.randint(0, 10) <= 2 and not small:
            height = 2
            width = 2
        else:
            height = 1
            width = 1

But when the grid is already mostly full, a large tile is more likely to not be able to find a position, so when generating the layout, Grid will retry any image that doesn’t find a position with the small parameter which prevents a large tile being generated. If a small image doesn’t fit anywhere on the grid the Grid function will return because it assumes that ether

  • The grid is full
  • There are no valid positions on the grid left

This isn’t necessarily the best system, as it’s possible for the image it’s trying to place to be portrait (so taking up 2x vertical tile space), but there might be room on the grid for a single normal sized image.

But this process of generating a Grid with Tiles occupied from top left across and down results in a Grid generation process that looks a little like this;

tile-gen-preview tile-gen-preview tile-gen-preview tile-gen-preview tile-gen-preview tile-gen-preview tile-gen-preview tile-gen-preview tile-gen-preview tile-gen-preview tile-gen-preview

So you can see the process of positioning images then going back and filling in any gaps.

After each Grid generation, the Tiler looks at the images that were used in the Grid and marks them to make sure we don’t go over the per image inclusion rates. It also used the .full helper on Grid to make sure there are no blank spaces left on the grid. Even on a full grid, Tiler still marks the images as used, this maximizes the number of Grids that can be generated within the constraints. When Tiler runs out of images it moves on to rendering.

The Render step is relatively simple, infact the TileRenderer class is less than 20 lines

class TileRenderer(object):
    def __init__(self, grid, tileSize, cellSize, border, filePath):
        self.tileSize = tileSize
        self.cellSize = cellSize
        self.im = Img.new("RGB", tileSize)
        self.filePath = filePath

        for i, tile in enumerate(grid.grid):
            sizeX = (cellSize[0]*tile['width'] - border['size'])
            sizeY = (cellSize[1]*tile['height'] - border['size'])

            r = tile['image'].resize(sizeX, sizeY)
            box = (
                int(cellSize[0]*tile['x']+border['size']/2),
                int(cellSize[1]*tile['y']+border['size']/2)
            )
            self.im.paste(r, box)

        self.im.save(filePath, 'JPEG')

Given Grid already tells us all the positions, it quite simply pastes the Image into the correct place. There’s a resize helper on Image which simply hands off to PIL to do a resize and fit operation, this is basically a filling crop without changing the aspect ratio.

I ran into a fun issue with the Image class, original the file loading was done in the __init__ method, it looked a little like this.

class Image(object):
    """
    Image object
    """

    def __init__(self, filePath):
        self.file = filePath
        self.im = Img.open(self.file)
        self.size = self.im.size

Then as part of the image loading process we detect the mime type (using libmagic) and add it to our image list if the mime types match image/jpeg or image/png. However when I was testing I noticed some unusual behavior, my source image directory had almost 500 images in, but tilepaper was only picking up around 250;

Sams-MacBook-Air:wallpaper-tiles sr$ ls -l ~/Pictures/Wallpapers | wc -l
  470
Sams-MacBook-Air:wallpaper-tiles sr$ ENV/bin/python tilepaper/cli.py --source=~/Pictures/Wallpapers/ --destination=test-output/ --config=example-config.yml
INFO:root:Found 252 images

Weird. The mime detection code looks like this

with magic.Magic(flags=magic.MAGIC_MIME_TYPE) as m:
    for fileName in os.listdir(directory):
        fullPath = os.path.join(directory, fileName)

        if os.path.isdir(fullPath):
            images += self.findImages(fullPath)
        else:
            mimeType = m.id_filename(fullPath)
            if mimeType in self.imageTypes:
                images.append(Image(fullPath))
            else:
                logging.debug(
                    "Skipping file %s with mime %s"
                    % (fullPath, mimeType)
                )

But running in --verbose mode I saw a lot of debug lines like

DEBUG:root:Skipping file /Users/sr/Pictures/Wallpapers/151031-180039-00030-1139.jpg with mime writable, regular file, no read permission

Hmm, no read permission? But you can read other files in that directory. Also why are you returing this error as the mime type? Remember PEP-20

Errors should never pass silently. Unless explicitly silenced.

Don’t give me a string with an error, give me an exception and let me deal with it *tableflip*

Anyway, enough complaining, lets get to the bottom of this problem. I was contemplating getting out strace/dtrace and playing around a bit, but there are certain advantages to having been working with *NIX for nearly 10 years, and my first hunch turned out to be right. I develop on a Macbook, running OS/X, and generally desktop systems tend to have much lower system limits. Also seeing permission denied errors on files that you know you can read generally comes down to one thing;

Sams-MacBook-Air:wallpaper-tiles sr$ ulimit -n
256

Yup, max open files is set to 256. After a little bit of investigation, it turns out PIL doesn’t actually need a file descriptor open on the image to do a lot of stuff, you can load it, then close the FD and still get access to a lot of stuff like size and metadata. With a few quick changes, I gave my Image class __enter__ and __exit__ methods, as well as passing PIL a file handle we can explicitly close. Now we can use with Image as im whenever we need access to the full PIL object, such as during rendering, then the rest of the code doesn’t have to care about the file descriptor being opened/closed, and we don’t run out of open files on desktop systems.

class Image(object):
    """
    Image object
    """

    def __init__(self, filePath):
        self.file = filePath
        _f = self.rawFile
        self.im = Img.open(_f)
        self.size = self.im.size
        _f.close()

    def __enter__(self):
        self._file = self.rawFile
        return Img.open(self._file)

    def __exit__(self, *sp):
        self._file.close()

    @property
    def rawFile(self):
        return open(self.file, 'rb')

Then in, for example, the resize function which needs a full PIL object with an FD we can do

def resize(self, width, height):
    """
    Resize the image to the specified size
    Resizes with croping/scaling
    """
    with self as im:
        return ImageOps.fit(im, (width, height), centering=(0.5, 0.5))

And all is happy. Of course it would have been better if the creators of the magic module raised an exception properly instead of expecting other people to deal with the fallout of permissions errors, but anyway.

I’ll probably still be working on this occasionally, if you’ve got any ideas for how to make it better hit me up.

Usage

Installing and running should be fairly simple for anyone familiar with Python projects. Just do a pip install tilepaper to get the latest version. You might wanna install it in a virtualenv so it stays out of the way.

Once it’s installed, you can run the example by just doing

tilepaper –source=./example-images/ –destination=./test-output –config=./example-config.yml –verbose

Configuration

The example config file should be a good place to start, there are a few configurable options to make it fit how you want it to work.

sizes:
  - 1366x768
  - 1920x1280
  - 1024x768
  - 1280x720
border:
  size: 2
  color: 000
grid:
  1024x768: [4,3]
  default: [6,4]
inclusions: 5

Most of the options should be fairly easy to understand but I’ll explain some of the less well named ones;

  • Sizes - List of sized outputs to generate, based on pixel size. You should set this to the size of your monitor(s). The output folder will contain a folder for each size specified here
  • Border - Details about the border between tiles, size is the gap to leave, in pixles, and color is the HEX code for the background colour. Black works best IMO but you can customize it if you want.
  • Grid - This is the size of the tile grid, you can specify a default then customize for each individual size tile. It’s a list of width, height in number of tiles. The size of the tiles is calculated by the number of tiles, border and grid size. It’s generally best to keep the aspect ratio as close to the aspect ratio of the output image size, as images will be cropped to fit.
  • Inclusions - This is the maximum number of times each image in the input folder should be used. An image will never be used twice on the same grid, but will be used in multiple grids.

Have a play around see what options work for you.