Movie layers provide a way to associate a Quicktime movie file with a specific Studio Artist layer. You can then work with individual frames in the movie layer like you would with any Studio Artist canvas layer, so you can paint, image process, warp, or otherwise manipulate the individual movie frames.

You can make a new movie layer or convert an existing layer to a movie layer (which will start by making a new movie file for the layer on your hard disk), or you can open a new movie layer from an existing movie file. All of these choices are available as menu commands in the Canvas : Movie Layer main menu.

In either of these cases the movie layer needs to be the same size dimensions as the existing layer stack.

You could also use the File : Import : Open Movie as New Canvas Layer menu command to close out the existing layer stack and start a new movie layer from an existing movie file. In this case the new movie layer will be sized to match the imported movie file. This would be equivalent to the old File : Open Canvas Movie menu command available in previous versions of Studio Artist.

To show off how you might use movie layers, we’re going to perform some hand painting on a simple paint animation generated in a previous tip on sequential keyframe recording. Here’s the paint animation generated at the end of this previous tip post.

Click here to view the embedded video.

I wanted to augment this simple paint animation by performing some additional hand painting on top of the existing movie frames. I started by using the Canvas : Movie Layer : Open Movie as New Layer menu command to open the movie file associated with the paint animation shown above. This generated a second movie layer in the layer stack. I then deleted my original layer 1 so that i had one movie layer in the layer stack called ex2.

As shown above, a movie transport appears at the bottom of the layers palette when the current layer is a movie layer. You can use the movie transport controls to play the movie in the Studio Artist canvas. The up down arrows associated with the current frame display can be used to move forwards or backwards through individual frames one at a time.

My goal for this simple demo was to hand paint in red dots to augment the existing painted eyes for each frame in my paint animation movie file. You could of course get much more elaborate about the kind of hand paint touchup and artistic effects you want to do when trying this out yourself. I set the source area to the fixed color picker and choose the shade of red i wanted to  hand paint with. I then used a wacom pen to hand paint in 2 red dots in the center of each eye in frame 1.

After finishing my hand paint touchup for frame 1, i then needed to record the changes i made to frame 1. Any changes you make to a movie layer are not stored in the movie itself unless you explicitly record the changes.  To do this you need to either press the R button (R stands for Record) in the movie layer transport, or run the Canvas : Movie Layer : Record to Existing Frame menu command.

If you don’t explicitly record any changes made to a frame image displayed in a movie layer, then if you change the current frame position and then change it back your frame editing changes will be gone. You can either record over the existing frame as discussed above, or you can insert a new frame after the current frame position (either by shift clicking the R button in the movie layer transport or by running the Canvas : Movie Layer : Insert as New Frame menu command).

I then used the up arrow control associated with the movie layer transport current frame display to advance to frame 2. I then painted in each eye with red paint, pressed the R button to record my changes to frame 2, then advanced to frame 3.  I continued this process for all 135 frames in the paint animation movie file. The finished result with the hand painted changes is shown below.

Click here to view the embedded video.

At any point in time you can use the movie layer transport controls to play your edited movie layer to see if the resulting motion is what you want or not. You can always go back to frames you have already edited and modify them again if you wish to.

Once you are finished with your editing, it’s a good idea to output your edited movie file as a new flattened movie file. What the flattening step does is generate a new recompressed movie file that only contains frames used in the actual movie presentation.

Movie layer editing works behind the scenes by adding a new frame image to a Quicktime movie’s media track, and then adding a nonlinear edit to the Quicktime movie’s video track that references the new frame image in the media track at a particular frame time. Any new frames added to the media track are added as single uncompressed frames.  So 2 things are going on when you make an edit to a movie layer.  First, you are adding additional frame images and an associated nonlinear video edit to reference them at the frame in the video track they should be displayed at. Second, they are added as non-temporally compressed video frames. Flattening your edited movie file when you are done with your editing will recompress the entire movie frame by frame. So you end up with a new clean flattened movie file that is temporally compressed with no extraneous frames in it’s underlying media track.

For more Information

An Overview of Movie Layers is available here.

A tip on working with Movie Layers in Depth is available here.

The simple paint animation movie used as the starting point for the hand painting performed in this tip was generated in another tip post that discusses sequential keyframe recording.

We could have added two additional action steps to the PASeq used in this other tip post to generate the red eye coloring we manually painted above as a part of the initial paint sketches used for PASeq interpolation. An advantage of this alternate approach is that we would have only needed to paint in the red dots for the limited set of keyframe sketches generated in the sequential keyframe recording process ( as opposed to hand painting each individual frame like we did above).

In this tutorial we’ll be exploring the use of Studio Artist 4’s Temporal Image Operations for processing movie files.  Temporal or time based image operations are effects that use multiple frames in a source movie to build each effected output frame. The example above is a single output frame that was generated using a temporal rank filter.

Studio Artist processes movie files with Paint Action Sequences (PASeq).  You can think of a PASeq as a script that details the series of processing steps that are need to produce a certain artistic effect. So any time you want to process a movie file you need to build a PASeq to implement the a particular processing effect you want to achieve.

Studio Artist 4 includes a new temporal image operation mode. Temporal means time based.  These new processing effects work with multiple video frames over time to implement their effects. For an introduction to temporal image operations check out this tip.

When working with temporal image operations you need to have a loaded source movie. Temporal image operations also don’t have the normal Ip Source options associated with standard image operations in Studio Artist since they are currently designed to only use the source movies as input (not the canvas or other layers). If you don’t have a movie file loaded as your source they won’t seem to do anything other than pass the source through to the canvas.

The section below shows off a few of the different temporal image operations used to process a source movie file. These are just a few of the different temporal processing effects you can work with.  All of the video processing examples we’ll discuss below were generated using the same source video footage of a skater on a black background.

Each of these examples was created with a simple PASeq that contained one temporal image operation action step. The PASeq used for the first example is shown below. The others are similar, the only difference is that they use different temporal image operations, but they are all just single step PASeqs.

Temporal Difference Matte

An example of processing the skater source movie with the temporal difference matte is shown below.

Click here to view the embedded video.

This effect tries to extract motion taking place in a series of video frames and encapsulate it into a single output frame.

Temporal Rank Filter

An example of processing the skater source movie with the temporal rank filter is shown below.

Click here to view the embedded video.

This particular example was generated using a max algorithm for the temporal rank filter. So it’s combining together the brightest information across multiple frames into a single frame. This particular setting works well for this example’s source video due to the light skater moving on a black background. Different algorithm settings will dramatically change the visual appearance of the effect.

Temporal Time AxisRotate

An example of processing the skater source movie with the temporal time axis rotate effect is shown below.

Click here to view the embedded video.

This effect essentially rotates the stack of video frames so that what was the time axis becomes a spatial axis and vice versa.  It’s great for ethereal special effects when working with video that has motion.  Depending on the scan direction you can extract moving objects while removing background detail, or generate more abstract motion effects that encapsulate motion over time like the one seen in this example.

Temporal Scan Tracker

An example of processing the skater source movie with the temporal scan tracker is shown below.

Click here to view the embedded video.

A let to right scan setting was used for this particular scan tracker effect.The scan direction is going to interact with movement in the video.  Movement in the direction of the scan motion will be extended, while movement in the opposite direction of the scan direction will be deemphasized.  A vertical scan would encapsulate the horizontal motion in the video very differently.

The scan tracker is also great for generating static panorama images from video pan sequences.

For More Information

An introduction to temporal image operations can be found here.

A Studio Artist News post that provides an overview of what you can do with temporal image operations is available here.

The PASeq presets used in this movie processing tutorial series are available for download here.

This tutorial has focused on using temporal image operations for processing movie files to generate movie file output. But temporal image operations are also great tools for 2D artists that allow you to generate static images derived from video sequences. You can do things like encapsulate motion in a video over time into a single static 2d image, or create panorama images from video pan sequences.

To process a movie file in Studio Artist you first record a Paint Action Sequence (PASeq) and then process a movie file with the PASeq. A paint strategy is the series of processing steps you need to implement in your PASeq to build up a particular visual effect.

This discussion will focus on some technique to construct cartoon style auto-rotoscope animation effects. What we mean by auto-rotoscope is that you work with one frame of a movie to build up a particular stylistic paint effect encapsulated in a PASeq preset and then Studio Artist automatically paints the rest of the frames in the visual effect style you created in your custom PASeq.

Here’s an example of a finished cartoon style animation effect generated by processing a source movie with a PASeq. The Vectorizer was used to generate this particular stylistic effect.

Click here to view the embedded video.

Studio Artist  has the capability to create a wide range of different stylistic cartoon effects. In this tutorial we’ll discuss a few of the many different processing approaches you can take to create cartoon style animation effects. There are also a lot of prebuilt factory PASeq presets that you can use to get started quickly creating cartoon animation image or movie processing effects.

When building up a PASeq for processing video you always want to have a paint strategy in mind. Paint Strategies are discussed in much more detail in the first part of this tutorial series, but the basic idea is to formulate the components of the visual effect style you are trying to achieve and then record action steps in your PASeq to implement the different steps necessary to build up the desired effect.

My initial thinking was that i wanted to generate a soft cartoon style that looked stylized but still retained a lot of the original source video characteristics. I used multiple applications of the Smooth image operation to generate this effect. The first Smooth action step processed the Source Image. The rest of the Smooth action steps process the Current Layer. The initial PASeq i constructed is shown below.

The Smart Blur ip op step was added for some additional edge smoothing. The Blur ip op step at the end is using an Edge 1 Composite setting and is actually a contrast boost sharpening step to boost the contrast and sharpness of the softening effect. The Simplify ip op is another processing effect that is very useful for generating soft cartoon styles.

The soft cartoon style animation i generated with the PASeq created above is shown below.

Click here to view the embedded video.

I liked the fact that the soft cartoon effect retained a lot of the source video characteristics. But after viewing it i wanted to try a more flat color cell animation cartoon style. To achieve this i used the Vectorizer to convert the soft cartoon canvas image into a flat color vector rendition. My modified PASeq with the additional vectorizer step is shown below.

I could have just run the vectorizer in a new single step PASeq by processing the original source image. But by processing a modified version of the source image i was able to achieve a more simplified flat color cartoon effect. Preprocessing your source footage to remove undesirable features or enhance desirable features prior to effect processing or painting is a useful tool to add to your paint strategy arsenal.

The flat color cartoon animation generated by my modified PASeq is shown below.

Click here to view the embedded video.

After viewing the flat color cartoon animation output i decided that i liked it better for this particular sequence than my initial concept of a soft cartoon rendition. I then thought it would be interesting to try adding some additional vector paint sketching on top of the flat color cartoon rendition.

To achieve this i added the following action steps to my cartoon style PASeq. All of these steps were implemented by running different Path menu commands. First i erased the bezier path layer.

Then i generated a set of bezier paths derived from the source image edges.

Then i picked a thin vector stroke paint synthesizer preset. I used a Blend Composite setting with a low Blend % setting in the Paint Fill Apply control panel so that the vector sketch paint strokes i was adding would be more subtle when placed on top of the flat color cartoon canvas.

I then painted in the bezier path buffer using the vector paint preset i had just edited.

My final PAseq after recording the additional processing steps described above is shown below.

When i processed my source movie with this modified PASeq i generated the following cartoon style animation.

Click here to view the embedded video.

Additional Cartoon Styles

The Vectorizer is capable of a wide range of different flat color cartoon styles and effects. The cartoon example at the very top of this tutorial post was generated with a single vectorizer action step in a PASeq that processed the source image. A vectorizer Region Effect was used used to stylize the generated vector regions in that animation.

Studio Artist 4 also allows for mix compositing when using the vectorizer so you can use repeated applications of the vectorizer with partial transparency to build up vector gradient effects that have some shading incorporated into them.

The Color Simplify image operation is also a very useful tool for generating cartoon styles. The cartoon style animation example below was generated using a single Color Simplify ip op action step in a PASeq.

Click here to view the embedded video.

Many cartoon styles incorporate some form of thin line sketch outlining in addition to rendering image mass with soft gradient or flat color areas. There are many different ways to generate thin outline sketch effects in Studio Artist. Some image processing options include the Color Edge, Color Simplify, Edge, Gradient, Smart Contrast, Sketch Edge, and Watershed Regionize ip ops. The Color Edge and Sketch Edge ip ops can also directly generate bezier paths that could then be painted in by the paint synthesizer. The paint synthesizer is also capable of a wide variety of different sketch paint effects by itself.

For More Information

This tutorial is the second part in a series of tutorials on movie processing strategies. The first part of the tutorial is available here. The second part of this tutorial series is available here.

The PASeq presets i described above are included in the new Movie Processing Strategies category. These PASeq presets showcase some additional version 4 features that work well for movie processing.

For some additional information on the mechanics of processing a movie file with a paint action sequence check out this tip.

The PASeq presets used in this movie processing tutorial series are available for download here.

Studio Artist processes movie files with Paint Action Sequences (PASeq). A PASeq can be thought of as a script that details the different processing steps needed to achieve a certain stylistic effect. PASeqs consist of a series of action steps recorded into a list. When PASeq recording is turned on, then any processing actions you run in Studio Artist will be recorded as individual action steps in the PASeq. This includes automatic actions as well as manual paint steps.

To begin, you need to open the source video you wish to process. To do this run the File : New Source and Canvas… menu command.

A standard file dialog will come up that allows you to choose a Quicktime movie file of your choice. Navigate the file system in the file dialog and select the movie file you wish to process.

After selecting the movie file you wish to process a New Canvas sizing dialog will come up. This dialog shows the size of your movie source frames as well as an adjustable canvas size. Set the canvas size to what you want your output movie frame size to be and then press OK.

For this particular example we choose the canvas size to be the same size as the source video. But Studio Artist lets you choose any canvas size you want. This allows you to take low resolution source video and generate video processing effects or paint animations at a higher resolution. So you could take normal video resolution as an input and generate HD or even IMAX processed output frame resolutions if you wish.

After selecting the source movie you wish to work with and setting a working canvas size you are ready to get started. The screenshot belows shows how i configured my Studio Artist workspace for this example. The working canvas is in the center of the workspace in white. The Source area is at the top left below the main operation toolbar. The Editor palette is positioned below the source area. The Paint Action Sequence palette is docked on the right side of the workspace.

You can of course configure the workspace any way you wish. You will need access to the Source Area, Editor, and Paint Action Sequence palette for this particular tutorial. You can turn any of the Studio Artist palettes on or off in the workspace by using the appropriate Windows menu command.

Your selected source movie should be displayed in the Source Area under the Image tab. When a source movie is loaded a set of movie transport control buttons become available under the image display. You can use these to play the movie file as well as more around between different frames in the movie.

There are a wide variety of different image operations (ip op) to choose from. Each one has different editable parameter options that can totally change the visual appearance of the ip op effect. To enter image operation mode, use the operation popup in the left side of the main operation toolbar at the top of the workspace to select Image Operation.

When you do this the Editor will reconfigure to show parameters available to edit for image operation effects. The popup at the top of the Editor will display the different image operation effects you can choose from. For this particular tutorial we’ll be working with the Threshold image operation effect. So select Threshold in the popup at the top of the Editor palette. Be aware, if you are not in Image Operation mode, then the Editor will display control panels available for whatever operation mode you are currently in. The operation popup in the main toolbar next to the Action button tells you which operation mode you are currently in.

Once you are in the Threshold ip op in Image Operation mode, you can press the Action button to run the effect.

The particular effect you will generate is a function of the different editable parameters available in the Editor for the Threshold ip op effect.

The IP Source parameter selects what is going to be processed by the ip op effect. In the screen snap above it is set to Current Layer. This means that what ever image is in the current canvas layer will be sent through the Threshold ip op effect and then the result will be placed back in the canvas.

For video processing you will want to process the source image rather than the current layer. So change the IP Source option to Source Image. This will insure that your source video frames are used as the input to the Threshold ip op effect.

Now if you press the Action button you can see what the Threshold ip op will do with it’s current parameter settings.

With the default settings the Threshold ip op generates a  black and white hard threshold effect. By modifying the editable parameters you can generate a wide range of more sophisticated threshold or black and white film noire processing effects.

Play around with the effect settings and put together a processing effect you want to apply to your source video. When you are happy with your edited effect, you are ready to record it into a Paint Action Sequence. Turn on recording in the Paint Action Sequence palette by clicking on the Rec checkbox located at the top of the Paint Action Sequence palette.

I used the modified settings below to generate a black and white film noire look.

After turning on PASeq recording and then clicked the Action button. After the threshold action runs a single action step labeled Threshold is recorded in the Paint Action Sequence. You can now turn off PASeq recording by unchecking the Rec checkbox at the top of the Paint Action Sequence palette.

You have now successfully recorded a custom paint action sequence that runs a single Threshold ip op that you also custom edited. You are now ready to process your source video file.

To process a source video file with the current paint action sequence, run the Action : Process with Paint Action Sequence : Source to Movie… menu command.

A standard file dialog will appear that lets you name your processed output movie file. Name it what you want and position it where you want it to be placed in the file system on your computer. The click OK and Studio Artist will render out your processed movie file frame by frame.

Note that Studio Artist is generating a new processed movie file. It is not modifying your existing source movie file at all, which remains untouched.

If you click cancel in the Create Movie Stream file dialog then the movie processing will still preview in your workspace but no movie file will be generated. This is useful if you want to preview the processing. To stop movie file processing for any reason, just press the spacebar. A dialog will come up asking you if you want to stop processing or continue on. Click the appropriate button to stop or continue.

If You Run Into Problems

If your output video is generated with an unchanging blank frame, then you probably didn’t setup your image operation to process the Source Image.  You need to make sure that the IP Source setting is set to Source Image for your ip op effect.

If IP Source is set to Current Layer for your ip op effect then the ip op is processing the canvas, not the source frames in the video. So if the canvas is empty then the empty canvas would be processed by the effect for each output movie frame.

An alternate approach to constructing your PASeq would be to first set the canvas to the source image.  You could then run an image operation that uses an IP Source option set to Current Layer. To set the canvas to the source image, you can either drag the source to the canvas or you can hold down the mouse on the Eraser in the operation toolbar until a list of possible erase to options comes up and then choose Source Image.

If your processed output movie file seems to have a different frame timing than your source movie file you may have setup your movie preferences to change the frame timing. In the Movie tab of the Preferences dialog there’s a preference option called Process Frame Timing.  You want the Process Frame Timing option set to  Source Movie to pass through the source movie timing to your processed output movie.  If it’s set to MovieCodec FPS then the processed output movie is retimed to a specific FPS (frames per second) setting specified in the File : Movie Codec dialog.

The movie preferences also include skip frame settings.  Skipping or duplicating frames would also change your output movie file timing, so these settings should be set to 0 if you don’t want that to happen.

Codec stands for the kind of movie compression format that is used to encode the output movie frames. You specify the movie compression codec used by the settings in File : Movie Codec dialog. Your processed output movie file will have it’s frame images compressed using your current Studio Artist movie codec settings. So your output movie file may be compressed differently than your input movie file.  This could result in a much larger movie file size on your had disk, especially if you have specified an uncompressed movie code file format for your output movie processing.

For More Information

For an introduction to working with Image Operation effects check out this tip.

Here’s another getting started tip that discusses processing a movie with a PASeq.

For more information on configuring the Studio Artist workspace check out this tip.

For a more advanced tutorial that discusses paint strategies for generating paint animation effects in processed movies check out this tip.

Studio Artist uses Paint Action Sequences (PASeq) to process movies. So to implement your paint strategy for building a particular paint animation effect you need to record a series of action steps in a PASeq that work together to build the particular processing effect you are trying to create. Designing an effective paint strategy and associated PASeq processing preset is often an iterative process.

Here’s an example of a finished paint animation processing effect. What i like about this example is that i started out with an initial design goal, but in the process of implementing it and watching the end result i was inspired to move in a slightly different direction for what ended up being my final paint animation.

Click here to view the embedded video.

The examples below give some insight into the design process that went into creating the animation above. My initial design goal was fairly straight forward. I wanted to create a simple paint animation effect using a single wet paint preset. I used multiple applications of the wet paint preset with reduced brush size for each auto paint iteration to first rough in the image and then build detail in the painting in the later passes as the brush size gets smaller.

The PASeq below shows my initial design attempt.

I started off by erasing the canvas to black. I option m clicked keyframe 2 for this first action step to record a black mute keyframe. Doing this insures that this first action step only plays back for the first frame in the animation. I did this because i wanted to overdraw on the previous output frame for all frames except the first frame of the animation.

I then recorded 5 auto paint passes using the same wet paint preset. For each pass i reduced the brush size by adjusting the Max Size Range control in the Brush Modulation control panel of the paint synthesizer. Reducing the brush size over time is a typical paint strategy to build detail in a painting.

The paint animation generated from this first simple PASeq is show below.

Click here to view the embedded video.

After watching this initial paint animation i decided that i wanted to change my initial design goals for the animation. I decided that my initial strategy of overpainting the entire video frame resulted in an animation that was a little more frenetic than i wanted. So i edited the individual auto paint action steps so that they focused on starting paint strokes in high textured areas of the source video only.

I did this by making some simple edits in the Path Start control panel of the paint synthesizer shown below. I set the Probability option there to Texture Prob Weight. And i turned on the Texture Range control and set the Texture Min to 30 and the Texture Max to 255. These editing changes in sure that the paint preset will only start a paint path in a high texture or edge area of the source video.

I also wanted to modify the previous output frame prior to painting on top of it. I wanted my previous paint strokes to soften and dissipate over time in the animation. I added a Smart Blur image operation action step to my PAseq to achieve this goal. I used the Polar variation setting to achieve a stylized polar blur of the painted background over time in the animation.

Note that i dragged the SmartBlur action step to the top of the PAseq list shown below so that it processes the previous output frame prior to the auto paint steps painting on top of the processed previous frame. This PAseq overdraws on top of the previous output frame because we don’t erase the canvas except when processing the initial first frame of the animation.

The paint animation that results from processing my source movie with the modified PAseq above is shown below.

Click here to view the embedded video.

Note that the visual appearance of this second paint animation is very different than my first attempt at building the effect. Visual flicker has been reduced, and the painting now takes place in the foreground on top of a dissipating blurred painted backdrop.

After watching this second attempt at creating my effect i noticed that i actually liked the look of the larger brush size paint steps more than the final painted appearance after the small detail generating brush steps. Adding detail made the painting more recognizable but gave it  a somewhat flatter visual appearance for this particular paint preset. So i removed the last  3 Auto paint action steps from my PASeq.

I liked this new paint look achieved from using fewer paint steps with larger brush sizes. But i still wanted to add more emphasis on the features present in the original source video. I used 2 different techniques to boost the source features in the painted canvas. I first used the Equalizer ip op to generate a high spatial frequency representation of the source image. I set the ip op Composite control to Replace Region Selection. I did this so that the output of the Equalizer ip op was routed to the current layer’s selection buffer rather than added to the canvas.

I then used the Selection Modulate Rotate interactive warp to slightly warp the painted canvas. This warp is adaptively modulated based on the contents of the current selection buffer. It’s an interesting technique that can be used to add spatial feature detail to an abstract image or painting.

I then used the Edge ip op to add some additional source edge modulated sharpening to the painted canvas. Note from the Edge ip op control panel settings below that i’m using the Source Image as the input to an Edge Strength Coloring effect that is then add scaled composited into the painted canvas. Doing this adds additional source video feature detail to the painting.

The resulting PASeq with these editing changes is shown below.

This final PASeq implements the following paint strategy. It starts by erasing the first frame only of the animation to black. Then the contents of the canvas (the previous output frame) are then blurred. Two auto paint steps then paint in the features of the source video frame. I then generate a high frequency representation of the source image, place it in the selection buffer, and then warp the painted canvas using the selection buffer to modulate the warp. I then boost the source frame edge features in the painted canvas.

The resulting paint animation generated from this final PASeq design iteration is shown below

Click here to view the embedded video.

What i like about this particular example is that it shows how your initial design goals can sometimes change as a part of the overall design process based on the visual feedback you get from viewing your initial attempts at constructing an animation. By going with the flow and trying out different ideas as a part of my design process i was able to move in some different directions and end up with a more interesting animation than what i initially set out to create.

For More Information

This tutorial is the second part in a series of tutorials on movie processing strategies.  The first part of the tutorials is available here. The third part of the tutorials is available here.

The PASeq presets i described above are included in the new Movie Processing Strategies category. These PASeq presets showcase some additional version 4 features that work well for movie processing.

For some additional information on the mechanics of processing a movie file with a paint action sequence check out this tip.

The trick i described above for adding source features to an abstract image or painting is described in more detail in this following tutorial.

The PASeq presets used in this movie processing tutorial series are available for download here.

Overview

Movie Processing in Studio Artist is achieved by building a processing script called a Paint Action Sequence (PASeq) and then processing a video file with the PASeq to generate a new effect rendered output movie.  The individual action steps recorded in the PASeq work together to build up a particular artistic effect. The Paint Action Sequence palette shown above is accessible from the main Windows menu and is where you record and edit a PASeq.

You can open a source movie in Studio Artist’s source area and work with a single frame of your source movie to build and record a PASeq.  When PASeq recording is turned on any processing actions (manual or automatic) you perform in Studio Artist will be recorded as individual action steps in the PASeq. When finished recording you PASeq, you can then run one of the Action menus to process a movie file (or folder of numbered frame images) with the PASeq to render out your new processed output movie file.

For more information on recording custom paint action sequence and processing a movie file with it check out this tip.

Building a Paint Strategy

Every successful paint action sequence is built up from implementing some kind of paint strategy. A paint strategy can be thought of as the set of steps necessary to build up a particular visual effect. An artist always follows some kind of a strategy or plan to build up an artistic image. The combination of all of the steps in the strategy act together to generate the desired visual effect.

When processing video there are additional perceptual factors to take into account besides the artistic appearance of a single image.  A video is composed of a series of frame images presented sequentially over time. When processing a series of video frames you need to be aware of how any image processing or painting you are doing is going to look when the processed video frames are sequentially presented to a viewer as the rendered video file plays back.

One important part of any paint animation sequence involves what to do with the animating canvas background. There are several different approaches to this component of a paint strategy for building a paint animation. The traditional thing to do would be to erase the background of the canvas to white before drawing each frame. This simulates traditional paint or sketch animation where the animator always draws on a clean piece of paper.

A visual side effect of erasing the background each frame is that the resulting animations usually have a perceived flicker. This is especially true when an animation is built up from generic autopainting within Studio Artist. There may not be any real visual continuity between paint strokes from frame to frame when autopainting with some arbitrary paint synthesizer preset.  If there is lack of temporal continuity from painted frame to frame then that results in perceived flicker in the processed output movie when you watch it.

The paint  strategy you will follow when building your paint action sequence depends on what you are trying to achieve stylistically in a given animation. If you want it to look like it was hand drawn animation, then a style with some visual flicker from frame to frame might be just what you want. If you don’t, then there are various strategies to use to eliminate flicker in a paint animation.

Eliminating flicker involves introducing some form of temporal continuity from frame to frame. By temporal continuity we mean that there is some visual coherence between sequential frame images in the output movie. One approach to generating temporal continuity in an animation is to modify or transform the previously painted output frame in some fashion, and then paint on top of the modified canvas.

Making sure that individual paint strokes have continuity from frame to frame is important for generating temporal continuity. Painting on top of the previous output frame is one way of doing this. But you could also use other techniques like time particles or bezier path interpolation to build paint stroke continuity from frame to frame in an animation.

Building paint paths from source frame features (like edges) is also an approach that can work well for reduced flicker in paint animation. This is because paint strokes that track the edges in the source video frames will track the object movement taking place in the video over time. A technique like this can be used in conjunction with overdrawing on a modified previous output frame image to reduce or eliminate flicker.

Many Studio Artist paint presets also incorporate various kinds of randomization to help achieve a more organic paint look.  This can take the form of color randomization as well as modulation of different characteristics of the paint paths and brush over the drawing of a single paint stroke as well as over the course of drawing multiple paint strokes.  Sometimes you need to dial back or eliminate artistic randomization effects in a paint preset when processing video to avoid introducing temporal variance from frame to frame.

Typically you will be working with source video that is 24 or 30 frames a second.  But traditional animation is often generated at a lower frame rate. Reducing the frame rate of an animated output sequence (like to 10 – 15 fps) is another great way to reduce the visual perception of flicker. Studio Artist has movie preferences you can use to automatically skip source frames when processing with a PASeq if you want to reduce the output frame rate without stretching out the timing of the resulting animated movie.

Another key point is that often a successful paint strategy and it’s associated paint action sequence can be a combination of paint effects and image processing effects. One of the strengths of Studio Artist is that you can combine together different paint effects, image processing effects, warping or morphing effects, region masking, and resolution independent texture synthesis within a given paint Action Sequence script. Most successful animations are based on scripts involving several different paint or image processing presets that work together to build the particular desired effect.

Wet Flow Paint Animation Example

Here’s an example of a simple paint animation effect i put together to showcase some of the principals of building a paint strategy as discussed above. The PASeq incorporates overdrawing on a modified previously painted canvas (as opposed to erasing the canvas at the beginning of each new frame). I was trying to achieve a rough fairly abstract paint style with some liquid flow in the background. I also didn’t want the painting to be so abstract that you couldn’t distinguish what the original source content was.

Click here to view the embedded video.

The video example above shows off the end result of my iterative design process to build the desired paint animation style i described above.

I started by choosing a rough paint preset that focuses on laying down paint in high texture regions of the source image.  I then recorded four paint passes using this preset with progressively smaller brush size for each pass. My initial PASeq (shown below) also erased the canvas to white before painting each frame.

The video example belows shows what you get when you process the skater source video with this initial PASeq.

Click here to view the embedded video.

Because the canvas is first erased to white for each frame prior to painting it in there’s a fair amount of visual flicker associated with the paint animation.

My next attempt at modifying this initial PASeq was the one shown below. A black mute keyframe was inserted at frame 2 of the first Set Canvas – White action step in the PASeq list. You hold down the m hotkey while option clicking a keyframe to record a black mute keyframe. I then choose a radial flow canvas spreader paint preset and used it to spread the paint strokes on the canvas out from the center towards the edges.  After recording this additional paint step into the PAseq i then dragged it at the top of the PASeq list right after the initial set canvas to white action step that only runs for the first frame of the animation.

So this PASeq erases the canvas to white for the first frame only.  The canvas is then spread out by the flow paint, and then the original paint steps that draw in the skater take place. Since the canvas is not erased except at frame 0 the previous output frame will always be in the canvas when the next frame starts drawing. The video example belows shows what you get when you process the skater source video with this second PASeq.

Click here to view the embedded video.

I liked the fact that overdrawing on top of the flow modified previous output frames reduced the visual perception of flicker in the resulting paint animation. But i felt that the foreground of the moving skater was a little hard to distinguish from the smeared paint background. So, i introduced an additional processing step that modified the previous output frame by  slightly hue shifting it. The modified PASeq with the addition of a HueShift image operation action step is shown below.

The video example belows shows what you get when you process the skater source video with this third PASeq. Note how the addition of the hue shift step helps to distinguish the foreground from the background.

Click here to view the embedded video.

At this point in designing my paint strategy i wanted to make the source video features a little more recognizable in the paint animation.  And i wanted to reduce the emphasis of the flowing smeared background a little and increase focus on the foreground painting.  I also decided to start with a black background instead of a white background.  My final PASeq for my iteratively designed paint strategy is shown below.

Once again, here’s the final paint animation video below generated with the final PASeq shown above.

Click here to view the embedded video.

Note how i worked iteratively to build my final paint strategy.  I took my initial goals and worked to build my paint action sequence step by step to achieve those goals. This is usually a more effective way to work than trying to do everything in a single pass, especially when just getting started with Studio Artist. As an experienced Studio Artist user i find that i still modify and tweak my initial PASeq designs to hone a particular effect and to fine tune it to the characteristics of my source video.

Being organized and focused on building the different components of your desired paint strategy step by step is the best way to build your own custom paint animation effects. Don’t be afraid to experiment.  You can always delete a recorded PASeq action step if you’re not happy with what it’s doing. You can also export PASeq presets as you’re working, so that you can go back to a particular iteration of the effect if you get off track in your design process.

Getting Started

Studio Artist comes with thousands of different paint synthesizer and PASeq presets to get you started. They are arranged in different Collections and Categories. All of the presets are completely editable, so you can modify them or build your own. If you don’t want to build your own, you can use the ones we provide to get started. If you want to build your own, there are an unlimited number of presets that can be generated within the program by custom editing your own presets.

Combining together different paint presets and image processing effects within a paint action sequence is how you build up a particular paint strategy. Don’t be afraid to experiment.  Often it’s best to start out fairly simple and then add additional action steps one at a time to build a more sophisticated effect.

Where to go from here

We have provided a tutorial folder of specific paint action sequence presets in the Process Movie Tutorial category designed to show off a series of different simple paint strategies.  The Process Movie Tutorial pdf helps explain how these different PASeq presets were built. This particular set of tutorial PASeq presets is a little old but still useful for getting started with building PASeqs for movie processing.

The presets i described above are included in the newer Movie Processing Strategies category. These PASeq presets showcase some additional version 4 features that work well for movie processing.

For some additional information on the mechanics of processing a movie file with a paint action sequence check out this tip.

This tutorial is the first part of a series of tutorials on building paint strategies for processing movies. The second part of the tutorials is available here. The third part of the tutorials is available here.

The PASeq presets used in this movie processing tutorial series are available for download here.

Movie layers can be used to paint or modify individual frames of a movie file. You could use movie layers to construct hand drawn animation a frame at a time, or to add paint or image processing effects to specific frames in an existing movie file. Movie layers are a new feature in v4 that replace the functionality associated with the old canvas movie features available in earlier versions of Studio Artist. This tip will discuss how to work with movie layers in Studio Artist 4.

Movie layers associate a Quicktime movie file with a specific canvas layer. You can have multiple movie layers in the canvas as a part of the overall layer stack. Any specific movie layer can be played in real time in the canvas by using the movie transport controls located at the bottom of the Layer palette. A composited stack of multiple movie layers cannot currently be played back in real time. But you could flatten the layer stack to a composited output movie and then play that movie output in real time.

The movie transport controls (shown below) appear at the bottom of the Layer palette whenever your current working layer is a movie layer. The transport controls vanish if your current working layer is not a movie layer.

You can use the transport control buttons to move to the beginning or end of the movie file, play the movie forward or in reverse, stop a playing movie, and record over an existing frame or insert a new frame. The frame counter shows the current frame (its frame 1 in the screenshot above). You can click on the up/down arrows to single step the current frame forward or backwards.

You can also use arrow keys to reposition the current frame time if the canvas is the current keyboard focus. Up/down keys single step forward/backwards. Left/right keys move to the beginning or the end of the movie file. The +/- keys also single step forwards/backwards.

You can think of a movie layer as being composed of an image buffer directly associated with the layer and a movie file sitting on disk. When you playback a given movie layer or single step through it’s frames using the layer movie transport controls you are loading the layer’s image buffer with a particular frame image from the movie file on disk. Any changes you make while working with the layer in Studio Artist are made to the image buffer associated with the layer, not to the actual movie sitting on disk. You need to explicitly record any changes you make to the image buffer associated with the movie layer back into the movie file on disk if you want those changes to be stored in the movie file.

In order to record a modified frame image into the movie file associated with a given movie layer you need to record your modified canvas into a given frame position. To record overwrite an existing frame you can either press the R (record) button in the layer movie transport or you can use the appropriate Canvas : Movie Layer menu command.

You can also insert a new frame rather than recording over an existing frame. To insert a new frame after the current frame position you can either shift click the R button in the layer movie transport controls, or you can use the appropriate Canvas : Movie Layer menu command.

Building an animation from scratch

You can use a movie layer to build hand painted animation from scratch.  To start out you need to first start a new movie layer or convert your existing layer to a movie layer.  You can use the appropriate Canvas : Movie Layer menu command to do that. In either case you’ll be asked to name a new movie file on your hard disk, which will be the movie file associated with the movie layer you are generating.

Once you have your working movie layer, you can then start generating your individual frame images. After generating your first frame image, you can press the record button to record the current contents of the movie layer’s image buffer into the first frame of the movie layer’s associated movie file.  You can then generate your second frame image.  You will need to insert a new frame after your first frame, which you can do by shift clicking the record button. Shift clicking the record button will insert a new frame after the current frame position. There are also Canvas : Movie Layer menu commands for both of these operations (that include command key options to run them).

You will then continue to generate new frame images one at a time and insert them into the associated movie file as new frames to build up your desired animation. You can use the movie transport controls at any time to playback your recorded movie file changes in real time. You can also retouch previous frames or add additional painting to previous frames if building up animation from scratch.

There are additional Canvas : Movie Layer menu commands to insert blank space between each of the frames of the movie in your movie layer. So you could construct a series of key frames, then insert blank frames between each key frame, and then build intermediate animation images between the previously drawn keyframes.

Canvas Movie Onion Skin

There are canvas movie onion skin options to help you build intermediate animation frames. You can specify 2 different onion skins, which could be positioned before and after the current frame, or both before or after the current frame at different frame times. By onion skin we mean a partially transparent view of another frame time image(s) displayed on top of the current frame image that is used as a drawing reference for building up animation.

An additional canvas movie onion skin toolbar appears in the Layer palette below the movie transport control toolbar if you turn on the canvas movie onion skin option. There’s a Canvas Movie Onion Skin Toolbar onion skin preference that turns on or off the display of the Layer palette onion skin toolbar, so that  preference option needs to be turned on to use this feature.

You can use the Canvas : Onion Skin : Toggle Onion Skin menu (and it’s associated command key option) to toggle the onion skin view on or off. When the canvas movie onion skin option is toggled on then the onion skin toolbar control become active. There are 2 check boxes to turn on or off each of the 2 onion skin views. The associated numeric fields allow you to specify the number of frames before and after the current frame for the 2 onion skin offset views. By using a negative number for the frame offset you can move the onion skin offset to the opposite side of the current frame (for example -1 for the before offset moves it to after 1 frame).

You could potentially work off of a second loaded source movie file if you wished by using the source onion skin option. The canvas movie onion skin is useful for building new frames off of previously drawn frame images. The source onion skin is useful if you are hand roto-scoping a source movie in some cartoon or paint style.

Studio Artist also includes many additional features for automatically rotoscoping source movies in an endless variety of different paint or cartoon animation styles.  This is discussed in more detail in the section on ‘when not to use movie layers’ at the end of this tip.

Flattening a Movie Layer

Any edits you make to a movie layer are stored as non-destructive non-linear frame edits within the associated movie file. For example, suppose you record over a given frame 5 times over the course of building up an animation. Each of those 5 new frame images is added to the associated movie’s media track, and then a non linear edit performed to insert the new frame data into the appropriate frame time position in the movie’s video playback track.

So over the course of building up an animation you can potentially build up a lot of extraneous media track frames. Also, new recorded frames are added as animation codec uncompressed frame data with no time compression.  After generating a finished animation you may wish to flatten the associated movie file for optimal compression and file size.

There’s a Canvas : Movie Layer menu command to Save a Flattened Canvas Movie as.  You can use this menu command to generate a new flattened movie file that gets rid of any extraneous media track frames that are no longer used. The flattened movie output is also compressed with your current File : Movie Codec Settings menu dialog settings.

Saving a flattened canvas movie also allows you to composite multiple movie layers into a single playable flattened movie file. So flatten refers to potentially flattening the Studio Artist layers as well as flattening the internal media frames stored in the movie file internally.

The current view mode for layers is used when flattening a canvas movie. This basically means that whatever you see displayed on the screen is what is going to be generated for the flattened movie file. So if you want to only flatten the current layer and not a composted view make sure you are in view current layer mode. If you want to flatten multiple layers into a single flattened movie file use the view all mode.

Movie Layer Settings

There are some additional Canvas Movie Layer : Settings menu flag options that are useful when working with movie layers.

Loop Play causes the movie layers to loop during playback if the option is checked. If the option is unchecked then playback stops at the end of the movie.

Advance on Record will automatically advance the current frame position after recording a frame if the option is checked. This option is useful if you are sequentially moving through the frames of a movie layer adding some additional painted feature to each frame.

Synch to Source synches the movie layer(s) to the current source movie frame position if checked.

Synch Layers synches the current frame position of multiple movie layers if checked. If unchecked then different movie layers could be positioned independently to different frame positions.

When not to use movie layers

Movie layers are really designed to build hand drawn animation frame by frame, or to perform touch up or editing on individual frames in a movie.

If your goal is to process an entire existing movie file with a Studio Artist effect you are better off not using movie layers. Instead, you would record a Paint Action Sequence (PASeq) that generates the particular effect you want to process you movie file with. You could then use one of the Action menus to process your movie file with the PASeq.

Here’s a tip that talks in more detail about processing a movie with a paint action sequence.

An alternative approach to generating hand drawn animation is to work with a paint action sequence and build the animation there. You can record a series of individual paint steps in the PASeq. You can then use sequential keyframing in the PASeq timeline editor to record additional keyframes for an animation. You can then use the keyframe animation functionality of the PASeq timeline to generate interpolated animation off of your recorded PASeq keyframes. This approach could potentially save you a lot of drawing time as opposed to if you hand draw each individual frame from scratch.

Individual sketches or paintings can also be encapsulated into a single PASeq action step. A series of these encapsulated paintings can then be keyframe interpolated to generate an animation. An advantage of encapsulated keyframe animation is that an arbitrary number of paint strokes can be used for each keyframe sketch or painting. With sequential keyframing you typically have one PASeq action step for each individual paint stroke. But an advantage of sequential kayframing is that you can use different presets to build up the sketch or painting (as opposed to a single one which would be the case for an encapsulated keyframe animation). Of course you could combine together different encapsulated PASeq action steps in one PASeq to build an encapsulated keyframe animation using different paint presets.

Here’s a tip that talks about encapsulated keyframe animation.

This tip discusses how to create a keyframed paint animation from a series of freely sketched pencil sketches. Each sketch can have an arbitrary number of pencil strokes. Each sketch is encapsulated into a single paint action step recorded in a Paint Action Sequence (PASeq). The final animation is generated by animating the keyframed single step PASeq into a movie file. By working with keyframes as opposed to hand drawing each frame in the animation you can generate a completed animation in a fraction of the time it would take to draw each frame manually.

Bezier encapsulation is a powerful feature of Studio Artist. Bezier paths are vector paths that underly each paint stroke in Studio Artist. Each layer contains a Bezier path frame in addition to a raster frame buffer. A set of multiple Bezier paths can be encapsulated into a single paint action step, and the bezier paths can then be keyframed in a PASeq timeline.

This tip will discuss how to generate keyframed animation from encapsulated bezier paths. One nice feature associated with encapsulated bezier paths is that an entire drawing composed of multiple paint strokes can be recorded into a single paint preset or paint action step in a PASeq. There are also PASeq Timeline menu commands that can manipulate encapsulated bezier paths so that a series of sketches made with arbitrary number of paths can be configured to smoothly animate over time.

Making your first sketch

The source onion skin is a useful feature that can be used as an aid when manually sketching a source image. I used a source onion for all of my keyframed sketches in this example.

To start, load the source image you want to use as your first sketch. Make sure the canvas is the size you want it to be for your animation.  You can either do this by using the File : New Source and Canvas menu to load your source image and resize the canvas in one step, or you can use the Canvas : Resize Canvas menu.

Make sure the Canvas : Onion Skin is set to Source and then turn on the onion skin by either running the Canvas : Onion Skin : Toggle Onion Skin menu or running cmnd t which is it’s keyboard shortcut. You should see a semi-transparent view of the source image in your drawing canvas, this is the source onion skin.

Nw choose the paint preset you will be drawing with. I used the Default : Pencil : Round1 paint preset, which draws like a soft graphite pencil.

Make sure the paint preset you are working with does not have any path randomization. You can check this by going to the Path Randomize control panel in the Editor palette and making sure both Displace options are set to None. Path randomization interferes with the Bezier path conversion process, so you should avoid it if working with Bezier encapsulation.

Since we want to record the bezier paths associated with our drawing, we need to make sure the Path : Path Layer Record Options : Pen Draw menu flag is turned on. When this menu flag is turned on then any manual drawing will be converted into Bezier paths and the paths will be stored in the current Bezier path frame.

Erase the canvas before drawing to white. You want to make sure you do a full erase, so make sure the default color is set to white and then shift click the erase button in the main operation toolbar. Shift clicking erase runs a full erase, which also erases the Bezier path frame for the layer in addition to erasing the raster frame buffer to the default background setting (white in this case).

Now draw a simple free sketch of the source image. Only what you draw will show up in the final sketch, the onion skin is there for a guide only. When you are finished sketching you can turn the onion skin off (run the Toggle Onion Skin menu discussed above).

To verify the Bezier paths associate with your drawing were recorded into the Bezier path frame switch to Bezier operation mode. You should see the paths appear on top of your sketch strokes. A shortcut to drop into and out of Bezier operation mode is to use the cmnd 5 keyboard shortcut.

Switch back to paint synthesizer operation mode. make sure you haven’t changed the paint synthesizer from the preset you were using for your sketch drawing. We’re now ready to encapsulate the recorded Bezier path frame into a single pain action step.

Bezier Encapsulation

To encapsulate the recorded Bezier paths in the Bezier path frame into the current paint synthesizer preset, you run the Path : Convert to Paint Synthesizer : as AutoDraw Embedded Bezier Frame menu.

This menu command captures the contents of the current Bezier path frame into the internal Bezier memory in the paint synthesizer. It also runs a paint synthesizer macro edit that configures it’s editable parameters so that it will play back the paths stored in it’s internal path memory.

Now we’re ready to record the encapsulated paint preset in a Paint Action Sequence. Open the PASeq palette and erase it’s contents by pressing the erase button in the toolbar at the top of the PASeq palette. Turn on recording by checking the record checkbox in that same toolbar. Then press the action button. The encapsulated paint step should now playback your sketch and a single paint action step should record in the PASeq. Turn off PASeq recording by unchecking the record checkbox.

Recording the remaining keyframes

You can now record the remaining keyframes associated with your sketch animation. You will perform the following cycle of steps for each remaining keyframe.

1. open a new source image
2. turn on the onion skin
3. perform a full erase of the canvas
4. draw you sketch
5. switch to bezier operation mode
6. option click a PASeq keyframe cell to record the keyframe

I prefer to record adjacent keyframes, and then use the Action : PASeq Timeline : Time Compress/Expand menu command to expand the timeline when i have finished recording all my keyframes.

Generating an animation movie from your recorded keyframes

Since i wanted to have the PASeq erase the canvas first for each frame and then draw the sketch, i recorded a second PASeq action step that sets the background to white. I did this by turning on PASe recording, pressing the erase button, turning off PASeq recording, and dragging the recorded Set Canvas- White action step to the top of the list.

The screen snap below shows my finished PASeq for the animation. I recorded 4 keyframes associated with 4 sketches. I wanted to have 10 frames between each keyframe, which leads to a total of 31 frames in the animation, since keyframes are recorded at frames 1,11,21,31.

At this point you can generate an animation movie from your recorded PASeq. To do this you run the Action : Animate with Paint Action Sequence : to Movie menu. This menu will ask you to name the generated movie file and will then run the animation rendering the frames out to that movie file.

If you just want to preview the animation without rendering it to a movie file you can just press the Animate button in the PASeq toolbar. You can always use the spacebar to stop an animation or a rendering movie process.

If you now watch the generated animation movie you will notice a few problems associated with it.  The animation between keyframes will not appear to be smooth, and there will be abrupt jumps at keyframe points where paint strokes jump into or out of existance.

There are 2 reasons for the problems associated with the animation. The first is that each keyframe was recorded with an arbitrary number of sketch strokes. But to have a smooth animation we would want to have the same number of paint strokes for each sketch. The second reason is that the stroke transitions between keyframes are totally arbitrary, when ideally we would want spatially adjacent paint strokes in adjacent keyframe times to interpolate.

The solution to both of these problems is to run some Action : PASeq Timeline menu commands on our recorded PASeq prior to rendering out the final animation. The first menu to run is the Action : PASeq Timeline : Embedded Bezier Frame : Normalize Split menu. Both normalize command options insure that each keyframe in the PASeq timeline has the same number of embedded bezier paths. Normalize Split splits the longest paths to normalize the path count. Normalize Recycle duplicates paths.

The second menu command to run is the Action : PASeq Timeline : Embedded Bezier Frame : Reorder Minimal Distance menu command. This menu command reorders the path indexes of the recorded paths to insure that the interpolated paths are spatially adjacent, which leads to a smoother animation. This second menu command may be slow to run if you have a large number of embedded Bezier paths or a large number of keyframes.

3 movie examples are shown below. The first is the original keyframed animation. The second is the split normalized animation. The third is the reordered by minimal distance animation.

original animation

split normalized animation

minimal distance reordered animation

Watching each of these 3 animations should give you a feel for what the various processing steps do.

If you want to create even smoother animation, you can break your sketch up into parts and record each part as a separate PASeq action step. Each of the action steps will have associated keyframed encapsulated Bezier paths. For a face ketch, you could break the sketch up into action steps for each of the facial components (eyes, nose, mouth, hair). Building the sketch this way insure that the keyframe interpolation will only be associated with the transformation of each facial part and not transforming between parts, leading to smoother automatic motion of the component paint strokes.

You can also generate alternative keyframed Bezier animations based on various automatic path generation operations, like generating the sketch paths from the source edges. One advantage of this kind of approach is that there’s no manual drawing, the entire process used to generate the animation is fully automatic. There’s a tip that describes how to do this in more detail.