Embroidered Animation! Finally!
Nina and I have been working forever on a project to create a short animation where every frame is rendered in embroidery. It's finally finished and ready for the world! You can read Nina's Blog post about it, or watch it here.
Every frame in the video is simply a photograph of an object that exists in the real world (in fact, you can buy them). There is no Photoshop manipulation. All the close-up animation of the figures is photos of the 6-frame sheets as you see at the start of the film.
The song is something to do with passover. Sorry, not my thing, so I can't tell you much about what it all means (see Nina's blog). All I know is that it's a folk song, kind of like the spider-that-ate-the-fly song, and that it's sung in Hebrew and Aramaic (which sounds as ancient as it is).
There's a good bit of technology that went into creating the stitched animation. I've given several talks about it, but never posted about it online, because we were waiting until the film was done.
There are two reasons it's not easy to make this kind of animation (other than that it's not easy to get started on something so obviously more work than it's worth). First, embroidery machines only go so fast, and even a small bit of animation requires a lot of frames (this film has 516 individually stitched images running at 12 frames per second, with some looping and repetition). It just takes time to do that much stitching (about a month in this case).
The second problem is that the software available for creating embroidery designs is not up to the job. It's stuck, both conceptually and stylistically, somewhere in the late 1980's. In particular, it is utterly impervious to attempts at automation.
Automation is crucial in animation, both because of the scale of the problem, and because if you want consistency from frame to frame, you want to avoid manual steps that introduce variability.
So we created all the embroidery designs using custom software we wrote specifically for animation embroidery, which allowed us to control the exact placement of stitches, not one frame at a time, but over all the frames as a whole.
The process starts with Nina's animation (which she does using Macromedia Flash, because that's her tool of choice, and because it can export animation sequences in vector file format, unlike crap Flash after Adobe ruined it). Here's what a frame looks like at the start:
Here's what it looks like when imported into Mathematica (my tool of choice). This is Mathematica's beautiful rendering of its Region[] objects, which allows you to see that we've preserved the full polygons for each moving part of the figure.
To see this better, we can move one of them to the side:
Mathematica's region functionality is quite powerful. RegionIntersection[] gives us clipped polygons that include only what's visible:
We start with the unclipped polygons because they are used to determine the direction of stitches (always perpendicular to the major axis of the polygon). Because, in Nina's style of animation, the polygons don't change shape, this means the stitch direction will follow the movement of the figure, making the final animation look like a single stitched figure is moving, rather than a hodgepodge of separate figures.
This is the result of filling each clipped polygon with stitches based on the orientation of its unclipped form:
You can see all the zig-zig stitches (called satin stitching), but notice that there are also some lines haphazardly crossing the middles of patchs? Those are traveling lines that let the machine get from one patch of color to another of the same color, without having to stop and cut the thread. This is crucial to maintaining fast, efficient stitching. You don't see them in the final result because they are always underneath later satin stitching. This turns into an optimization problem: Figure out which colors to do in what order, to minimize total thread cutting. For example, these were the first two steps in making the above figure: Notice how the traveling lines will all get covered later. Our code optimizes this automatically.
Here is the stitched result. Notice how stitch direction makes the different parts of the leg clearly stand out from the body, and from each other, even though they are exactly the same color. The interplay of light and thread is a delightful aspect of this style of animation!
Even just filling a patch is far from trivial. You can't just start at one end and stitch to the other, because you might stitch yourself into a corner. Imagine a painter who is painting the floor of a house with a complicated set of connecting, branching hallways. If you start painting down one dead-end hallway, you'll paint yourself into a corner and not be able to get out to finish the rest of them. Instead you have to walk to the end of each side-branch and paint backwards, until you reach a branching point, then walk down the next branch, etc. Filling with stitches is just like that, except you're only allowed to move the brush in one direction, which further complicates the issue. Here is a little animation I made testing the code we wrote for solving this problem (which turns out to be largely one of graph theory). The lines are "walking stitches" used to get to the ends of the "hallways" and the filled patches are the satin stitches that will cover them up.
Of course it matters what direction the stitches are going. This next clip shows how completely different the filling order is as you change the direction:
It took a while to get all this code right, but of course it's crucial that it cover every possible case, because when you're doing animation, it's not about getting one frame right, it's about having an automated process that always gets every frame right, not just once, but every time there is an iteration of the animation requiring a re-render of the frames. We must have generated tens of thousands of frames (just generated the files, not actually stitched them!) before it was all looking good.
You can see the finished animation at the top of this blog post, and below is a version that is just a run through of all 516 frames (with just a bit of looping, but no cut-aways), in case you want to concentrate on watching the stitches.
In keeping with Nina's free-everything copyright-is-evil philosophy, the embroidery files used to create every frame of this movie are available under Creative Commons license. You can download them (in PES format) from https://archive.org/details/ChadGadyaEmbroideryFiles_20150710