Studio Artist 4 lets you work off of a source image or start from scratch when building up a digital painting. For beginning users, working with a source image is the best way to get a feel for how Studio Artist works.

To get started, double click on the Studio Artist application icon. When the Studio Artist interface first launches it will be in the Paint Synthesizer operation mode. The Preset browser will display a series of factory paint presets.

The Source Area initially starts up with the Fixed Color option, displaying a color picker you can use to choose the current source color you want to paint with.

Using your mouse or pressure sensitive pen, try painting a few quick paint strokes in the canvas.

By clicking on different presets in the preset browser you can load them as the current preset.  Each preset has a different appearance and characteristic behavior when drawing.

Working with a Source Image

Studio Artist’s paint synthesizer is able to intelligently analyze a source image and then use that visual analysis to intelligently paint the source image in a variety of different styles. The visual characteristics of the loaded source image can also influence the appearance and behavior of paint presets designed for manual drawing as well. So you should try painting with a loaded source image to really appreciate the full potential that the factory paint presets have to offer.

Now lets get started by loading a source image. Use the File : Open Source menu command to open a source image.

If you are just getting started there are some source images that come with Studio Artist you can use for your first introductory painting sessions. When you run the Open Source menu command a standard file dialog will come up. Try going to the A Few Source Images folder that is in the same folder as the Studio Artist application. Choose the A Sunflower image to get started.

After you select your source image the Source Area will change to the Image tab and you will see a small representation of the source image you selected displayed in the source area.

Now try painting a few paint strokes. Note that the coloring of your paint strokes is now being chosen automatically from your source image. The color that is selected depends on where you are drawing in the canvas and is chosen to match the color of your source image in an equivalent spatial location. The source image will automatically map to the full canvas dimensions, even if they are different sizes.

Some paint presets will drag their initial coloring for the length of the entire paint stroke. Other presets may clone color, which means that the coloring of the paint stroke will adjust as you move the pen while drawing to best reflect the associated source coloring.

Automatic Painting

Studio Artist can intelligently paint a source image or a complete video file in a variety of different visual paint styles with no assistance from a user. This is called auto-painting, or auto-rotoscoping when working with movie files. Studio Artist uses the results of it’s intelligent visual analysis of the source image or source movie file to control drawing behind the scenes.

Click here to view the embedded video.

If you are just getting started with digital painting then Studio Artist’s intelligent assisted drawing will really open up new creative doors for you by enabling you to quickly be able to create artistic pieces without having to spend years learning to draw. Even if you are comfortable working with traditional manual painting you will find that automatic painting is an amazing source of creative inspiration, and a useful new tool for building artistic effects. And no matter what your background is, auto-painting is just plain fun.

Now, lets try automatic painting. Press the Action button.  Watch how Studio Artist automatically paints in the source image. To stop automatic painting, just press the spacebar.

Each paint preset has a different automatic drawing behavior in addition to the different physical appearance of the paint.  Try selecting different paint presets and then use the Action button to see how they automatically paint.

Paint Presets

Studio Artist presets are hierarchically organized by Collection And Category. There are 2 popup controls at the top of the factory preset browser that you can use to choose different collections of presets and the different categories associated with each collection.  Every time you load a new preset category the list of presets in the factory preset browser changes to display the individual presets included in that category.

Studio Artist ships with thousands of different factory presets. They encompass a vast range of different artistic effects. Rather than restrict your creative options to a few generic art presets we try to offer a very wide range of different aesthetic choices.

Some presets try to emulate traditional art media or techniques. Others create never seen before digital paint effects. All of the factory presets are completely editable, so you can modify and tweak them if desired to suit your own artistic needs.By editing your own presets, and building your own preset collections and categories you can totally customize Studio Artist to your own personal artistic vision.

Spend some time exploring the different factory paint presets that ship with Studio Artist 4. Some presets may have been designed primarily for auto painting or for manual painting, so it’s worth both manually painting a few paint strokes and then pressing the action button and watching the preset auto-paint to get a sense of what an individual paint preset can do.

If you do a lot of manual hand painting the CD_hand_Drawn_media collection was specifically designed for manual painting as opposed to action painting.

Working with the Source Area

The source area determines how coloring is applied to paint strokes. Depending on how you adjust the source area controls you can dramatically change the coloring behavior of your painting.

There are 4 different tab option available in the source area

• Image colors based on a loaded source image or movie
• Color allows for manual color selection using a color picker
• Palette allows for coloring based on a palette of colors
• Gradient allows for coloring based on a color gradient

Most of the main source area tab options also include a secondary coloring option you can adjust with a popup control. The Image source area tab offers the following secondary coloring options, which are all derived in some way from the source image coloring..

• Paint uses the source image directly for coloring
• Colorize maps the source image into the current source color hue
• Palettize maps the source image into the current source palette colors
• Fixed uses the current source color (but you can c click the source image to select a new source color directly from the source image)

Changing the Pen Mode

The paint synthesizer offers a large number of editable parameter options that can change the appearance and drawing behavior of any given paint preset. You can adjust all of the different editable paint synthesizer parameters in the Editor palette.

One very useful adjustable control is the pen mode. Any given paint preset will have radically different drawing behavior based on the pen mode setting. So learning how to adjust the pen mode is a great way to start to get familiar with editing the paint synthesizer to customize how paint presets look and behave.

The Pen Mode option is located in the Pen Mode control panel.

There’s a tutorial tip that discusses the different pen mode options and how to adjust them here.

Paint Synthesizer Macro Edits

There are a large number of different editable controls available in the paint synthesizer for adjusting the look, feel and automatic drawing capabilities of a paint preset. While this extreme amount of control ultimately is extremely useful for both customization and for providing the underlying capability to produce rich,lush organic paint effects, it can be somewhat intimidating to a beginning user.

Macro edits are a collection of single step editing commands that can be used to easy modify or edit paint presets. The paint synthesizer macro edits are available in the Help Browser as a series of active link commands.

To edit a paint preset with a specific macro edit command, just click the black arrow icon on the left side of the command. Pressing the arrow icon will run the active link command, which will automatically edit the current paint synthesizer settings to correspond to the editing change you requested.

The screen shot above shows a user running the Clone Source Color macro edit. Running this macro edit command will adjust the current paint preset so it clones color as opposed to dragging color.

Many macro edits adjust a series of different internal editable parameters to make the requested editing change. You could of course do this manual editing yourself by adjusting the series of editable controls in the paint synthesizer editor. The point of the macro edits is to provide an easy single click editing interface to common editing commands as an alternative to getting under the hood of the pant synthesizer.

For More Information

An introductory overview to the design philosophy behind the Paint Synthesizer is available here.

A short tip on how to change the brush size is available here.

A short tip on how to adjust the paint opacity is available here.

The Studio Artist User Guide pdf is the ultimate technical reference for all of the functionality available in Studio Artist. There’s a very detailed Paint Synthesizer chapter in the User Guide pdf that goes into great detail about all of the editable controls available in the paint synthesizer. The User Guide pdf is available in your main Studio Artist folder.

In this tutorial we’ll be exploring the use of Studio Artist 4’s Temporal Image Operations for processing movie files.  Temporal or time based image operations are effects that use multiple frames in a source movie to build each effected output frame. The example above is a single output frame that was generated using a temporal rank filter.

Studio Artist processes movie files with Paint Action Sequences (PASeq).  You can think of a PASeq as a script that details the series of processing steps that are need to produce a certain artistic effect. So any time you want to process a movie file you need to build a PASeq to implement the a particular processing effect you want to achieve.

Studio Artist 4 includes a new temporal image operation mode. Temporal means time based.  These new processing effects work with multiple video frames over time to implement their effects. For an introduction to temporal image operations check out this tip.

When working with temporal image operations you need to have a loaded source movie. Temporal image operations also don’t have the normal Ip Source options associated with standard image operations in Studio Artist since they are currently designed to only use the source movies as input (not the canvas or other layers). If you don’t have a movie file loaded as your source they won’t seem to do anything other than pass the source through to the canvas.

The section below shows off a few of the different temporal image operations used to process a source movie file. These are just a few of the different temporal processing effects you can work with.  All of the video processing examples we’ll discuss below were generated using the same source video footage of a skater on a black background.

Each of these examples was created with a simple PASeq that contained one temporal image operation action step. The PASeq used for the first example is shown below. The others are similar, the only difference is that they use different temporal image operations, but they are all just single step PASeqs.

Temporal Difference Matte

An example of processing the skater source movie with the temporal difference matte is shown below.

Click here to view the embedded video.

This effect tries to extract motion taking place in a series of video frames and encapsulate it into a single output frame.

Temporal Rank Filter

An example of processing the skater source movie with the temporal rank filter is shown below.

Click here to view the embedded video.

This particular example was generated using a max algorithm for the temporal rank filter. So it’s combining together the brightest information across multiple frames into a single frame. This particular setting works well for this example’s source video due to the light skater moving on a black background. Different algorithm settings will dramatically change the visual appearance of the effect.

Temporal Time AxisRotate

An example of processing the skater source movie with the temporal time axis rotate effect is shown below.

Click here to view the embedded video.

This effect essentially rotates the stack of video frames so that what was the time axis becomes a spatial axis and vice versa.  It’s great for ethereal special effects when working with video that has motion.  Depending on the scan direction you can extract moving objects while removing background detail, or generate more abstract motion effects that encapsulate motion over time like the one seen in this example.

Temporal Scan Tracker

An example of processing the skater source movie with the temporal scan tracker is shown below.

Click here to view the embedded video.

A let to right scan setting was used for this particular scan tracker effect.The scan direction is going to interact with movement in the video.  Movement in the direction of the scan motion will be extended, while movement in the opposite direction of the scan direction will be deemphasized.  A vertical scan would encapsulate the horizontal motion in the video very differently.

The scan tracker is also great for generating static panorama images from video pan sequences.

For More Information

An introduction to temporal image operations can be found here.

A Studio Artist News post that provides an overview of what you can do with temporal image operations is available here.

The PASeq presets used in this movie processing tutorial series are available for download here.

This tutorial has focused on using temporal image operations for processing movie files to generate movie file output. But temporal image operations are also great tools for 2D artists that allow you to generate static images derived from video sequences. You can do things like encapsulate motion in a video over time into a single static 2d image, or create panorama images from video pan sequences.

To process a movie file in Studio Artist you first record a Paint Action Sequence (PASeq) and then process a movie file with the PASeq. A paint strategy is the series of processing steps you need to implement in your PASeq to build up a particular visual effect.

This discussion will focus on some technique to construct cartoon style auto-rotoscope animation effects. What we mean by auto-rotoscope is that you work with one frame of a movie to build up a particular stylistic paint effect encapsulated in a PASeq preset and then Studio Artist automatically paints the rest of the frames in the visual effect style you created in your custom PASeq.

Here’s an example of a finished cartoon style animation effect generated by processing a source movie with a PASeq. The Vectorizer was used to generate this particular stylistic effect.

Click here to view the embedded video.

Studio Artist  has the capability to create a wide range of different stylistic cartoon effects. In this tutorial we’ll discuss a few of the many different processing approaches you can take to create cartoon style animation effects. There are also a lot of prebuilt factory PASeq presets that you can use to get started quickly creating cartoon animation image or movie processing effects.

When building up a PASeq for processing video you always want to have a paint strategy in mind. Paint Strategies are discussed in much more detail in the first part of this tutorial series, but the basic idea is to formulate the components of the visual effect style you are trying to achieve and then record action steps in your PASeq to implement the different steps necessary to build up the desired effect.

My initial thinking was that i wanted to generate a soft cartoon style that looked stylized but still retained a lot of the original source video characteristics. I used multiple applications of the Smooth image operation to generate this effect. The first Smooth action step processed the Source Image. The rest of the Smooth action steps process the Current Layer. The initial PASeq i constructed is shown below.

The Smart Blur ip op step was added for some additional edge smoothing. The Blur ip op step at the end is using an Edge 1 Composite setting and is actually a contrast boost sharpening step to boost the contrast and sharpness of the softening effect. The Simplify ip op is another processing effect that is very useful for generating soft cartoon styles.

The soft cartoon style animation i generated with the PASeq created above is shown below.

Click here to view the embedded video.

I liked the fact that the soft cartoon effect retained a lot of the source video characteristics. But after viewing it i wanted to try a more flat color cell animation cartoon style. To achieve this i used the Vectorizer to convert the soft cartoon canvas image into a flat color vector rendition. My modified PASeq with the additional vectorizer step is shown below.

I could have just run the vectorizer in a new single step PASeq by processing the original source image. But by processing a modified version of the source image i was able to achieve a more simplified flat color cartoon effect. Preprocessing your source footage to remove undesirable features or enhance desirable features prior to effect processing or painting is a useful tool to add to your paint strategy arsenal.

The flat color cartoon animation generated by my modified PASeq is shown below.

Click here to view the embedded video.

After viewing the flat color cartoon animation output i decided that i liked it better for this particular sequence than my initial concept of a soft cartoon rendition. I then thought it would be interesting to try adding some additional vector paint sketching on top of the flat color cartoon rendition.

To achieve this i added the following action steps to my cartoon style PASeq. All of these steps were implemented by running different Path menu commands. First i erased the bezier path layer.

Then i generated a set of bezier paths derived from the source image edges.

Then i picked a thin vector stroke paint synthesizer preset. I used a Blend Composite setting with a low Blend % setting in the Paint Fill Apply control panel so that the vector sketch paint strokes i was adding would be more subtle when placed on top of the flat color cartoon canvas.

I then painted in the bezier path buffer using the vector paint preset i had just edited.

My final PAseq after recording the additional processing steps described above is shown below.

When i processed my source movie with this modified PASeq i generated the following cartoon style animation.

Click here to view the embedded video.

Additional Cartoon Styles

The Vectorizer is capable of a wide range of different flat color cartoon styles and effects. The cartoon example at the very top of this tutorial post was generated with a single vectorizer action step in a PASeq that processed the source image. A vectorizer Region Effect was used used to stylize the generated vector regions in that animation.

Studio Artist 4 also allows for mix compositing when using the vectorizer so you can use repeated applications of the vectorizer with partial transparency to build up vector gradient effects that have some shading incorporated into them.

The Color Simplify image operation is also a very useful tool for generating cartoon styles. The cartoon style animation example below was generated using a single Color Simplify ip op action step in a PASeq.

Click here to view the embedded video.

Many cartoon styles incorporate some form of thin line sketch outlining in addition to rendering image mass with soft gradient or flat color areas. There are many different ways to generate thin outline sketch effects in Studio Artist. Some image processing options include the Color Edge, Color Simplify, Edge, Gradient, Smart Contrast, Sketch Edge, and Watershed Regionize ip ops. The Color Edge and Sketch Edge ip ops can also directly generate bezier paths that could then be painted in by the paint synthesizer. The paint synthesizer is also capable of a wide variety of different sketch paint effects by itself.

For More Information

This tutorial is the second part in a series of tutorials on movie processing strategies. The first part of the tutorial is available here. The second part of this tutorial series is available here.

The PASeq presets i described above are included in the new Movie Processing Strategies category. These PASeq presets showcase some additional version 4 features that work well for movie processing.

For some additional information on the mechanics of processing a movie file with a paint action sequence check out this tip.

The PASeq presets used in this movie processing tutorial series are available for download here.

Studio Artist processes movie files with Paint Action Sequences (PASeq). A PASeq can be thought of as a script that details the different processing steps needed to achieve a certain stylistic effect. PASeqs consist of a series of action steps recorded into a list. When PASeq recording is turned on, then any processing actions you run in Studio Artist will be recorded as individual action steps in the PASeq. This includes automatic actions as well as manual paint steps.

To begin, you need to open the source video you wish to process. To do this run the File : New Source and Canvas… menu command.

A standard file dialog will come up that allows you to choose a Quicktime movie file of your choice. Navigate the file system in the file dialog and select the movie file you wish to process.

After selecting the movie file you wish to process a New Canvas sizing dialog will come up. This dialog shows the size of your movie source frames as well as an adjustable canvas size. Set the canvas size to what you want your output movie frame size to be and then press OK.

For this particular example we choose the canvas size to be the same size as the source video. But Studio Artist lets you choose any canvas size you want. This allows you to take low resolution source video and generate video processing effects or paint animations at a higher resolution. So you could take normal video resolution as an input and generate HD or even IMAX processed output frame resolutions if you wish.

After selecting the source movie you wish to work with and setting a working canvas size you are ready to get started. The screenshot belows shows how i configured my Studio Artist workspace for this example. The working canvas is in the center of the workspace in white. The Source area is at the top left below the main operation toolbar. The Editor palette is positioned below the source area. The Paint Action Sequence palette is docked on the right side of the workspace.

You can of course configure the workspace any way you wish. You will need access to the Source Area, Editor, and Paint Action Sequence palette for this particular tutorial. You can turn any of the Studio Artist palettes on or off in the workspace by using the appropriate Windows menu command.

Your selected source movie should be displayed in the Source Area under the Image tab. When a source movie is loaded a set of movie transport control buttons become available under the image display. You can use these to play the movie file as well as more around between different frames in the movie.

There are a wide variety of different image operations (ip op) to choose from. Each one has different editable parameter options that can totally change the visual appearance of the ip op effect. To enter image operation mode, use the operation popup in the left side of the main operation toolbar at the top of the workspace to select Image Operation.

When you do this the Editor will reconfigure to show parameters available to edit for image operation effects. The popup at the top of the Editor will display the different image operation effects you can choose from. For this particular tutorial we’ll be working with the Threshold image operation effect. So select Threshold in the popup at the top of the Editor palette. Be aware, if you are not in Image Operation mode, then the Editor will display control panels available for whatever operation mode you are currently in. The operation popup in the main toolbar next to the Action button tells you which operation mode you are currently in.

Once you are in the Threshold ip op in Image Operation mode, you can press the Action button to run the effect.

The particular effect you will generate is a function of the different editable parameters available in the Editor for the Threshold ip op effect.

The IP Source parameter selects what is going to be processed by the ip op effect. In the screen snap above it is set to Current Layer. This means that what ever image is in the current canvas layer will be sent through the Threshold ip op effect and then the result will be placed back in the canvas.

For video processing you will want to process the source image rather than the current layer. So change the IP Source option to Source Image. This will insure that your source video frames are used as the input to the Threshold ip op effect.

Now if you press the Action button you can see what the Threshold ip op will do with it’s current parameter settings.

With the default settings the Threshold ip op generates a  black and white hard threshold effect. By modifying the editable parameters you can generate a wide range of more sophisticated threshold or black and white film noire processing effects.

Play around with the effect settings and put together a processing effect you want to apply to your source video. When you are happy with your edited effect, you are ready to record it into a Paint Action Sequence. Turn on recording in the Paint Action Sequence palette by clicking on the Rec checkbox located at the top of the Paint Action Sequence palette.

I used the modified settings below to generate a black and white film noire look.

After turning on PASeq recording and then clicked the Action button. After the threshold action runs a single action step labeled Threshold is recorded in the Paint Action Sequence. You can now turn off PASeq recording by unchecking the Rec checkbox at the top of the Paint Action Sequence palette.

You have now successfully recorded a custom paint action sequence that runs a single Threshold ip op that you also custom edited. You are now ready to process your source video file.

To process a source video file with the current paint action sequence, run the Action : Process with Paint Action Sequence : Source to Movie… menu command.

A standard file dialog will appear that lets you name your processed output movie file. Name it what you want and position it where you want it to be placed in the file system on your computer. The click OK and Studio Artist will render out your processed movie file frame by frame.

Note that Studio Artist is generating a new processed movie file. It is not modifying your existing source movie file at all, which remains untouched.

If you click cancel in the Create Movie Stream file dialog then the movie processing will still preview in your workspace but no movie file will be generated. This is useful if you want to preview the processing. To stop movie file processing for any reason, just press the spacebar. A dialog will come up asking you if you want to stop processing or continue on. Click the appropriate button to stop or continue.

If You Run Into Problems

If your output video is generated with an unchanging blank frame, then you probably didn’t setup your image operation to process the Source Image.  You need to make sure that the IP Source setting is set to Source Image for your ip op effect.

If IP Source is set to Current Layer for your ip op effect then the ip op is processing the canvas, not the source frames in the video. So if the canvas is empty then the empty canvas would be processed by the effect for each output movie frame.

An alternate approach to constructing your PASeq would be to first set the canvas to the source image.  You could then run an image operation that uses an IP Source option set to Current Layer. To set the canvas to the source image, you can either drag the source to the canvas or you can hold down the mouse on the Eraser in the operation toolbar until a list of possible erase to options comes up and then choose Source Image.

If your processed output movie file seems to have a different frame timing than your source movie file you may have setup your movie preferences to change the frame timing. In the Movie tab of the Preferences dialog there’s a preference option called Process Frame Timing.  You want the Process Frame Timing option set to  Source Movie to pass through the source movie timing to your processed output movie.  If it’s set to MovieCodec FPS then the processed output movie is retimed to a specific FPS (frames per second) setting specified in the File : Movie Codec dialog.

The movie preferences also include skip frame settings.  Skipping or duplicating frames would also change your output movie file timing, so these settings should be set to 0 if you don’t want that to happen.

Codec stands for the kind of movie compression format that is used to encode the output movie frames. You specify the movie compression codec used by the settings in File : Movie Codec dialog. Your processed output movie file will have it’s frame images compressed using your current Studio Artist movie codec settings. So your output movie file may be compressed differently than your input movie file.  This could result in a much larger movie file size on your had disk, especially if you have specified an uncompressed movie code file format for your output movie processing.

For More Information

For an introduction to working with Image Operation effects check out this tip.

Here’s another getting started tip that discusses processing a movie with a PASeq.

For more information on configuring the Studio Artist workspace check out this tip.

For a more advanced tutorial that discusses paint strategies for generating paint animation effects in processed movies check out this tip.

Studio Artist uses Paint Action Sequences (PASeq) to process movies. So to implement your paint strategy for building a particular paint animation effect you need to record a series of action steps in a PASeq that work together to build the particular processing effect you are trying to create. Designing an effective paint strategy and associated PASeq processing preset is often an iterative process.

Here’s an example of a finished paint animation processing effect. What i like about this example is that i started out with an initial design goal, but in the process of implementing it and watching the end result i was inspired to move in a slightly different direction for what ended up being my final paint animation.

Click here to view the embedded video.

The examples below give some insight into the design process that went into creating the animation above. My initial design goal was fairly straight forward. I wanted to create a simple paint animation effect using a single wet paint preset. I used multiple applications of the wet paint preset with reduced brush size for each auto paint iteration to first rough in the image and then build detail in the painting in the later passes as the brush size gets smaller.

The PASeq below shows my initial design attempt.

I started off by erasing the canvas to black. I option m clicked keyframe 2 for this first action step to record a black mute keyframe. Doing this insures that this first action step only plays back for the first frame in the animation. I did this because i wanted to overdraw on the previous output frame for all frames except the first frame of the animation.

I then recorded 5 auto paint passes using the same wet paint preset. For each pass i reduced the brush size by adjusting the Max Size Range control in the Brush Modulation control panel of the paint synthesizer. Reducing the brush size over time is a typical paint strategy to build detail in a painting.

The paint animation generated from this first simple PASeq is show below.

Click here to view the embedded video.

After watching this initial paint animation i decided that i wanted to change my initial design goals for the animation. I decided that my initial strategy of overpainting the entire video frame resulted in an animation that was a little more frenetic than i wanted. So i edited the individual auto paint action steps so that they focused on starting paint strokes in high textured areas of the source video only.

I did this by making some simple edits in the Path Start control panel of the paint synthesizer shown below. I set the Probability option there to Texture Prob Weight. And i turned on the Texture Range control and set the Texture Min to 30 and the Texture Max to 255. These editing changes in sure that the paint preset will only start a paint path in a high texture or edge area of the source video.

I also wanted to modify the previous output frame prior to painting on top of it. I wanted my previous paint strokes to soften and dissipate over time in the animation. I added a Smart Blur image operation action step to my PAseq to achieve this goal. I used the Polar variation setting to achieve a stylized polar blur of the painted background over time in the animation.

Note that i dragged the SmartBlur action step to the top of the PAseq list shown below so that it processes the previous output frame prior to the auto paint steps painting on top of the processed previous frame. This PAseq overdraws on top of the previous output frame because we don’t erase the canvas except when processing the initial first frame of the animation.

The paint animation that results from processing my source movie with the modified PAseq above is shown below.

Click here to view the embedded video.

Note that the visual appearance of this second paint animation is very different than my first attempt at building the effect. Visual flicker has been reduced, and the painting now takes place in the foreground on top of a dissipating blurred painted backdrop.

After watching this second attempt at creating my effect i noticed that i actually liked the look of the larger brush size paint steps more than the final painted appearance after the small detail generating brush steps. Adding detail made the painting more recognizable but gave it  a somewhat flatter visual appearance for this particular paint preset. So i removed the last  3 Auto paint action steps from my PASeq.

I liked this new paint look achieved from using fewer paint steps with larger brush sizes. But i still wanted to add more emphasis on the features present in the original source video. I used 2 different techniques to boost the source features in the painted canvas. I first used the Equalizer ip op to generate a high spatial frequency representation of the source image. I set the ip op Composite control to Replace Region Selection. I did this so that the output of the Equalizer ip op was routed to the current layer’s selection buffer rather than added to the canvas.

I then used the Selection Modulate Rotate interactive warp to slightly warp the painted canvas. This warp is adaptively modulated based on the contents of the current selection buffer. It’s an interesting technique that can be used to add spatial feature detail to an abstract image or painting.

I then used the Edge ip op to add some additional source edge modulated sharpening to the painted canvas. Note from the Edge ip op control panel settings below that i’m using the Source Image as the input to an Edge Strength Coloring effect that is then add scaled composited into the painted canvas. Doing this adds additional source video feature detail to the painting.

The resulting PASeq with these editing changes is shown below.

This final PASeq implements the following paint strategy. It starts by erasing the first frame only of the animation to black. Then the contents of the canvas (the previous output frame) are then blurred. Two auto paint steps then paint in the features of the source video frame. I then generate a high frequency representation of the source image, place it in the selection buffer, and then warp the painted canvas using the selection buffer to modulate the warp. I then boost the source frame edge features in the painted canvas.

The resulting paint animation generated from this final PASeq design iteration is shown below

Click here to view the embedded video.

What i like about this particular example is that it shows how your initial design goals can sometimes change as a part of the overall design process based on the visual feedback you get from viewing your initial attempts at constructing an animation. By going with the flow and trying out different ideas as a part of my design process i was able to move in some different directions and end up with a more interesting animation than what i initially set out to create.

For More Information

This tutorial is the second part in a series of tutorials on movie processing strategies.  The first part of the tutorials is available here. The third part of the tutorials is available here.

The PASeq presets i described above are included in the new Movie Processing Strategies category. These PASeq presets showcase some additional version 4 features that work well for movie processing.

For some additional information on the mechanics of processing a movie file with a paint action sequence check out this tip.

The trick i described above for adding source features to an abstract image or painting is described in more detail in this following tutorial.

The PASeq presets used in this movie processing tutorial series are available for download here.

Overview

Movie Processing in Studio Artist is achieved by building a processing script called a Paint Action Sequence (PASeq) and then processing a video file with the PASeq to generate a new effect rendered output movie.  The individual action steps recorded in the PASeq work together to build up a particular artistic effect. The Paint Action Sequence palette shown above is accessible from the main Windows menu and is where you record and edit a PASeq.

You can open a source movie in Studio Artist’s source area and work with a single frame of your source movie to build and record a PASeq.  When PASeq recording is turned on any processing actions (manual or automatic) you perform in Studio Artist will be recorded as individual action steps in the PASeq. When finished recording you PASeq, you can then run one of the Action menus to process a movie file (or folder of numbered frame images) with the PASeq to render out your new processed output movie file.

For more information on recording custom paint action sequence and processing a movie file with it check out this tip.

Building a Paint Strategy

Every successful paint action sequence is built up from implementing some kind of paint strategy. A paint strategy can be thought of as the set of steps necessary to build up a particular visual effect. An artist always follows some kind of a strategy or plan to build up an artistic image. The combination of all of the steps in the strategy act together to generate the desired visual effect.

When processing video there are additional perceptual factors to take into account besides the artistic appearance of a single image.  A video is composed of a series of frame images presented sequentially over time. When processing a series of video frames you need to be aware of how any image processing or painting you are doing is going to look when the processed video frames are sequentially presented to a viewer as the rendered video file plays back.

One important part of any paint animation sequence involves what to do with the animating canvas background. There are several different approaches to this component of a paint strategy for building a paint animation. The traditional thing to do would be to erase the background of the canvas to white before drawing each frame. This simulates traditional paint or sketch animation where the animator always draws on a clean piece of paper.

A visual side effect of erasing the background each frame is that the resulting animations usually have a perceived flicker. This is especially true when an animation is built up from generic autopainting within Studio Artist. There may not be any real visual continuity between paint strokes from frame to frame when autopainting with some arbitrary paint synthesizer preset.  If there is lack of temporal continuity from painted frame to frame then that results in perceived flicker in the processed output movie when you watch it.

The paint  strategy you will follow when building your paint action sequence depends on what you are trying to achieve stylistically in a given animation. If you want it to look like it was hand drawn animation, then a style with some visual flicker from frame to frame might be just what you want. If you don’t, then there are various strategies to use to eliminate flicker in a paint animation.

Eliminating flicker involves introducing some form of temporal continuity from frame to frame. By temporal continuity we mean that there is some visual coherence between sequential frame images in the output movie. One approach to generating temporal continuity in an animation is to modify or transform the previously painted output frame in some fashion, and then paint on top of the modified canvas.

Making sure that individual paint strokes have continuity from frame to frame is important for generating temporal continuity. Painting on top of the previous output frame is one way of doing this. But you could also use other techniques like time particles or bezier path interpolation to build paint stroke continuity from frame to frame in an animation.

Building paint paths from source frame features (like edges) is also an approach that can work well for reduced flicker in paint animation. This is because paint strokes that track the edges in the source video frames will track the object movement taking place in the video over time. A technique like this can be used in conjunction with overdrawing on a modified previous output frame image to reduce or eliminate flicker.

Many Studio Artist paint presets also incorporate various kinds of randomization to help achieve a more organic paint look.  This can take the form of color randomization as well as modulation of different characteristics of the paint paths and brush over the drawing of a single paint stroke as well as over the course of drawing multiple paint strokes.  Sometimes you need to dial back or eliminate artistic randomization effects in a paint preset when processing video to avoid introducing temporal variance from frame to frame.

Typically you will be working with source video that is 24 or 30 frames a second.  But traditional animation is often generated at a lower frame rate. Reducing the frame rate of an animated output sequence (like to 10 – 15 fps) is another great way to reduce the visual perception of flicker. Studio Artist has movie preferences you can use to automatically skip source frames when processing with a PASeq if you want to reduce the output frame rate without stretching out the timing of the resulting animated movie.

Another key point is that often a successful paint strategy and it’s associated paint action sequence can be a combination of paint effects and image processing effects. One of the strengths of Studio Artist is that you can combine together different paint effects, image processing effects, warping or morphing effects, region masking, and resolution independent texture synthesis within a given paint Action Sequence script. Most successful animations are based on scripts involving several different paint or image processing presets that work together to build the particular desired effect.

Wet Flow Paint Animation Example

Here’s an example of a simple paint animation effect i put together to showcase some of the principals of building a paint strategy as discussed above. The PASeq incorporates overdrawing on a modified previously painted canvas (as opposed to erasing the canvas at the beginning of each new frame). I was trying to achieve a rough fairly abstract paint style with some liquid flow in the background. I also didn’t want the painting to be so abstract that you couldn’t distinguish what the original source content was.

Click here to view the embedded video.

The video example above shows off the end result of my iterative design process to build the desired paint animation style i described above.

I started by choosing a rough paint preset that focuses on laying down paint in high texture regions of the source image.  I then recorded four paint passes using this preset with progressively smaller brush size for each pass. My initial PASeq (shown below) also erased the canvas to white before painting each frame.

The video example belows shows what you get when you process the skater source video with this initial PASeq.

Click here to view the embedded video.

Because the canvas is first erased to white for each frame prior to painting it in there’s a fair amount of visual flicker associated with the paint animation.

My next attempt at modifying this initial PASeq was the one shown below. A black mute keyframe was inserted at frame 2 of the first Set Canvas – White action step in the PASeq list. You hold down the m hotkey while option clicking a keyframe to record a black mute keyframe. I then choose a radial flow canvas spreader paint preset and used it to spread the paint strokes on the canvas out from the center towards the edges.  After recording this additional paint step into the PAseq i then dragged it at the top of the PASeq list right after the initial set canvas to white action step that only runs for the first frame of the animation.

So this PASeq erases the canvas to white for the first frame only.  The canvas is then spread out by the flow paint, and then the original paint steps that draw in the skater take place. Since the canvas is not erased except at frame 0 the previous output frame will always be in the canvas when the next frame starts drawing. The video example belows shows what you get when you process the skater source video with this second PASeq.

Click here to view the embedded video.

I liked the fact that overdrawing on top of the flow modified previous output frames reduced the visual perception of flicker in the resulting paint animation. But i felt that the foreground of the moving skater was a little hard to distinguish from the smeared paint background. So, i introduced an additional processing step that modified the previous output frame by  slightly hue shifting it. The modified PASeq with the addition of a HueShift image operation action step is shown below.

The video example belows shows what you get when you process the skater source video with this third PASeq. Note how the addition of the hue shift step helps to distinguish the foreground from the background.

Click here to view the embedded video.

At this point in designing my paint strategy i wanted to make the source video features a little more recognizable in the paint animation.  And i wanted to reduce the emphasis of the flowing smeared background a little and increase focus on the foreground painting.  I also decided to start with a black background instead of a white background.  My final PASeq for my iteratively designed paint strategy is shown below.

Once again, here’s the final paint animation video below generated with the final PASeq shown above.

Click here to view the embedded video.

Note how i worked iteratively to build my final paint strategy.  I took my initial goals and worked to build my paint action sequence step by step to achieve those goals. This is usually a more effective way to work than trying to do everything in a single pass, especially when just getting started with Studio Artist. As an experienced Studio Artist user i find that i still modify and tweak my initial PASeq designs to hone a particular effect and to fine tune it to the characteristics of my source video.

Being organized and focused on building the different components of your desired paint strategy step by step is the best way to build your own custom paint animation effects. Don’t be afraid to experiment.  You can always delete a recorded PASeq action step if you’re not happy with what it’s doing. You can also export PASeq presets as you’re working, so that you can go back to a particular iteration of the effect if you get off track in your design process.

Getting Started

Studio Artist comes with thousands of different paint synthesizer and PASeq presets to get you started. They are arranged in different Collections and Categories. All of the presets are completely editable, so you can modify them or build your own. If you don’t want to build your own, you can use the ones we provide to get started. If you want to build your own, there are an unlimited number of presets that can be generated within the program by custom editing your own presets.

Combining together different paint presets and image processing effects within a paint action sequence is how you build up a particular paint strategy. Don’t be afraid to experiment.  Often it’s best to start out fairly simple and then add additional action steps one at a time to build a more sophisticated effect.

Where to go from here

We have provided a tutorial folder of specific paint action sequence presets in the Process Movie Tutorial category designed to show off a series of different simple paint strategies.  The Process Movie Tutorial pdf helps explain how these different PASeq presets were built. This particular set of tutorial PASeq presets is a little old but still useful for getting started with building PASeqs for movie processing.

The presets i described above are included in the newer Movie Processing Strategies category. These PASeq presets showcase some additional version 4 features that work well for movie processing.

For some additional information on the mechanics of processing a movie file with a paint action sequence check out this tip.

This tutorial is the first part of a series of tutorials on building paint strategies for processing movies. The second part of the tutorials is available here. The third part of the tutorials is available here.

The PASeq presets used in this movie processing tutorial series are available for download here.

This tip is going to discuss some Studio Artist features useful for building up tone in an image. Tone or mass can be thought of as representing the visual weight of a sketch or artistic rendition and relates to building some kind of representation of the original source luminance in the artistic rendition of that source image.  In this tip we’ll be looking at paint synthesizer and image operation features that are useful for this topic.

Visual perception of imagery can be thought of in terms of different visual sensory modalities. People’s perception of visual imagery can be broken down into raw visual components like line or edge, tone or mass, color, movement, etc. We perceive these component aspects of visual imagery when we examine an image or painting.  And artistic representational techniques take advantage of this by how they work to represent (or ignore) these visual sensory components.

Tone or mass in a sketch works to represent the low spatial frequency luminance of the original source image being recreated in a sketch. A sketch may try to accurately reproduce the original luminance values, or it may radically distort the original tone values to accent certain image features while ignoring or suppressing others. Tone or mass in a sketch can also work to accent positive and negative space in the original source image. Or it may be distorted to better reproduce or enhance certain edge structure or salient visual features in the original source image while acting to simplify other non-essential source image features.

Many Studio Artist paint presets work by adding visual information to the working canvas. A simplistic view of this process is piling on paint strokes until some stopping point is reached. If the artistic goal is simply to build up an artistic representation by working with colored paint strokes that clone the original source image and completely fill the canvas then a user can ignore more advanced strategies for building tone and just let the paint synthesizer pile on paint strokes until a successful stopping point is reached.

But assume you are building a black and white sketch.  If one blindly piles on black paint strokes eventually the entire canvas will be filled with solid black. So for this kind of artistic image a more intelligent paint strategy must be programmed within the paint synthesizer so that the paint preset understands tone and tries to reproduce it within the black and white sketch.

There are various visual intelligence features in the paint synthesizer that can be turned on and will then work as an aid in reproducing tone in a sketch. This includes features in the path start, path end, and path application control panels. Some of these features are probabilistic in nature. Others are more intelligent and visually compare the original source mage with the existing working canvas to determine if adding a new paint stroke in a particular area of the canvas will lead to better or worse tone reproduction. The path start controls work to intelligently position paint paths to best reproduce source tone. The path end and path application controls work to stop drawing a particular paint path if it’s going to lead to bad source tone reproduction in a rendition.

Different attributes of a paint preset can also be modulated based on the original source luminance as an aid in reproducing tone or mass in a sketch. For example, the brush size can be modulated based on the source luminance so that larger brush sizes are used to reproduce dark tones while smaller brush sizes are used to reproduce lighter source tones. A similar kind of modulation can be used for the path length of paint strokes. These kinds of paint synthesizer programing can lead to paint presets that intelligently try to adjust how they draw to reproduce tone in an artistic rendition.

Path Start Control Panel

Setting the Inv Gray Prob Weight option for the Probability parameter will lead to paths starting more frequently on darker areas of the image and less frequently in lighter areas of the image.

The Lum Range parameters can also be used to restrict path start locations to a specific range of source luminance values.

Path End Control Panel

The Visual Error parameter provides a number of intelligent options that can be used to build tone rendition in a paint preset. White and black canvas lum error assume you will be starting with a white or black canvas. The last 2 options make no assumptions about the initial canvas state.  All of these options will end a path from drawing if they feel that continuing drawing would lead to incorrect tone reproduction.

There are also path end luminance range  options like those discussed for path start. The difference is that they end a path if the path leaves the retricted luminance range.

Path Application

The Probability parameter in the Path Application control panel includes a number of different options that can be used to intelligently build tone. These include WhiteCanvasProbErr, WhiteCanvasErr, BlackCanvasProbErr, BlackCanvasErr, Paint Lum Err, and Source Image Lum Err. This probability control acts to drop out or restrict individual paint nibs from drawing based on it’s specific setting while an overall path is being painted.

The tone mapped sketch image above was generated by auto-drawing with the Default : Auto Sketch : Visual Model2 paint preset. This preset uses a WhiteCanvasErr option for the Probability parameter in the Path Application control panel. It also uses a WhiteCanvasLumErr option for the Visual Error parameter in the Path End control panel.

Preprocessing the Source Image

An alternative approach to building tone with the paint synthesizer is to preprocess the source image and then paint the modified source. The processing can act to accent certain tonal areas while suppressing others.

The example above shows a modified source image that was processed with the Threshold ip op using the Soft LA Threshold algorithm. Generating a paint sketch off of this modified source image would enhance hatching in the dark areas of the image while suppress hatching in the lighter areas of the image. So the final sketch tone map will not accurately reflect the actual source luminance values but will reflect the distorted modified source tone map.

Movie Pixel Index Background Textures

The background texture control panel of the paint synthesizer includes a very useful tool for building hatching patterns that are used to reproduce tone in a sketch. A movie background texture generates the background texture based on individual frames in a movie file. How the individual movie frames are indexed or accessed during drawing can be modulated based on various visual attributes.  These include the source image rgb or luminance values, which would mean that the movie frame that best reproduces a particular area of the source image would be used when drawing in that associated area of the canvas.

The normal movie background texture re-indexes the movie frame every time the paint brush moves through the complete local area associated with the background texture. So if the movie background texture was 64 by 64 pixels then after the paint brush moved 64 pixels the movie frame used for the background texture would be re-indexed.

Movie pixel index background textures re-index the movie frame for each pixel when drawing.  This has the potential to run slower, especially if the movie is drawing from disk rather than from a memory preload.  But the movie frame is adjusted for each pixel to accurately track whatever frame index modulation has been chosen.

So, you can use movie pixel index background textures as a sophisticated technique to build customizable hatching patterns that accurately reproduce source luminance or tone for each pixel.  To do this, you build a stack of hatching patterns that reproduce tone for different luminance levels. The patterns should additively build to reproduce tone. Source luminance is then used to modulation the movie pixel indexing when drawing takes place.

Two examples of this approach to building intelligent hatching for tone reproduction in a sketch is shown above and below. This image was generated with the Circle Hatcher paint synthesizer preset which is located in the Default : General category.

For more information on working with movie pixel indexed background textures check out this tutorial.

Sketch Mass Ip Op

The Sketch Mass ip op is a new image operation effect in Studio Artist 4. This effect can be configured to create  a wide variety of different tone or mass sketch effects. There’s another Sketch Edge ip op that focuses of generating edge and feature sketches. Using the 2 effects together in a paint action sequence (PASeq) gives you the power to create an infinite variety of different sketch effects.

Both Sketch image operations can generate raster sketch images or they can generate a set of bezier paths to define a sketch. The generated bezier paths can then be used to drive the paint synthesizer leading to an even greater range of potential sketch styles.

The tone sketch examples at the beginning of this section and at the very top of this tip were created using the Sketch Mass ip op.

The Sketch Mass ip op is described in greater detail in this tip.

Generating bezier paths is described in greater detail in the tip.

The image below shows a simple painting generated from a single paint preset (Default:Utility:Bristle Paint-Flat”. Note the appearance is very abstract.

Now suppose you wanted to generate a painting like this that better reproduces the source image. One common approach is to work with multiple passes of the paint preset where you reduce the brush size and path length for each pass. This simulates what an artist would do in a real painting, where they would typically start with a large brush size to rough in the image and then use progressively smaller brush sizes to bring in more detail.

The image below shows off how this approach can generate additional detail. However, the painting still has a fairly abstract appearance.

Another approach to accenting edge detail in a painting is to superimpose an edge sketch on top of the abstract painting. There are a number of different ways to create this kind of effect. You could select a specific paint preset that generates an edge sketch effect. Many of these presets use path start and path end settings that focus the drawing onto the locations of edges in the source image. The example below shows off one way to add a black sketch effect to an abstract painting.

I generated a set of bezier paths from the source image edges using the ‘Path : Generate Paths : from Source Edges’ menu.

and then painted in the paths using a dark pencil preset by selecting a black pencil preset and running the ‘Path : Paint Paths : Full Layer’ menu command.

I then followed that with the Color Edge Ip Op setup to render a black edge min composited into the existing painted canvas. Note that i’m using the Source Image for the IP Source (as opposed to the current layer). This means the black edges are generated from the original source image as opposed to the painted canvas.

The 2 techniques described above work together in this example to create a nice tight edge sketch that helps the painting become more representational of the original source image. There are many different Ip Ops that can be used to create Edge renditions, including Color Edge, Edge, and Sketch as the first thing to check out. Smart Contrast, Gradient, Color Simplify, and Watershed Regionize also have parameter configurations that will generate black sketch effects. There are also many MSG presets that can create a wide range of different sketch effects.

Tightening up a Painting

A different approach to tightening up the edge rendition of a paint preset is to turn on ‘Local Image Range’ Nib Masking in the Paint Fill Apply control panel in the paint synthesizer Editor.

The various nib masking settings all work to automatically generate a on-the-fly selection mask for each individual paint nib before it is applied to the canvas. What the ‘Local Image Range’ setting does is to mask the paint nib if it spills into areas of the associated source image that are different colors the the current paint color. You can see the effect of turning on local image range nib masking below for the simple paintpreset we’ve been using in this example.

Note that the edges associated with the original source image are reproduced in the painting. However, the masking effect is very pronounced. Often a better approach is to start off by generating a more abstracted painting, and then turn on the local image range nib masking and overpaint. The example below shows off how this can give a more natural effect in the resulting painting.

There are 2 Paint Synthesizer Macroedits shown below available for you as active commands in the Help palette.

The actual specification of the size of the local image range is based on the Path End control panel ‘Local Color Range’ parameter value.

You can also use Local Color Range Path End masking to generate a tighter paint rendition. The difference between the Path End and the Nib masking approach described above is that the nib mask leads to a very tight rendition. Path End masking based on turning on Local Color Range will mask your paint paths but the complete nib will still draw, leading to a subtler, more natural paint effect, but with a subtler edge rendition due to the full paint nib being drawn.

There are some Paint Synth Macroedits available that can provide a quick way to modify a paint preset to focus on edge drawing or preserving local color areas.

Accenting Positive and Negative Space

Real painters work with their shading to help accent positive and negative space in the image they are painting. There are some different approaches you can use in Studio Artist to create this same kind of effect. One approach is to generate a region selection that breaks the image into positive and negative space, and then use that as either a selection mask for painting or as a probability modulator for things like path start and path end locations and path application probability.

The Smart Contrast Ip Op is a great tool for generating a selection mask that defines the positive and negative space in a source image. An example of what i mean is shown below.

This image was generated with the following Smart Contrast settings.

To generate a hard black and white mask image all three Smoothness controls need to be set to the same value. When they are different you will get additional gray levels in the resulting output, which can be useful if your goal is to use the output image as a probability mask in the paint synthesizer for things like path application probability based on the current selection region modulator.

To route the Smart Contrast output to the current region selection as opposed to placing it in the canvas, use the ‘Replace Region Selection’ Composite option shown below.

If you turn on Masking and then run the original ‘Bristle Paint-Flat’ paint preset located in Default:Utility used throughout this tip you will get the following kind of effect if Paint Masking is turned on.

You can turn on Masking by either checking the Mask checkbox in the main operation toolbar

or by using the ‘Canvas : Layer : Mask Status’ menu.

There are 3 different kind of masking you can use when painting. You choose them in the ‘Canvas : Layer : Paint Mask Type’ menu as shown below.

The ‘Paint’ mask type give a hard edge. The ‘Path’ mask type masks the paint paths as opposed to the paint nib, so it can be used to give a more natural appearance to a masked painting since the complete brush nib is drawn at the edge of a masked path area. Litho mask type masks the non wet part of a spreading wet preset, so it allows the selection mask to act like a lithography press where paint is applied in the non masked area by the wet spreading can move into the masked areas via drip or wiping effects.

I used ‘Path’ masking in the example below with a different preset that generates an airbrush effect. Note the smoother more natural appearance at the edges of the masked areas created by the Paint masking type.

You can invert the mask using the ‘Canvas : Selection : Invert’ menu command.

The next example was created using the same airbrush preset and the inverted selection mask with Path masking.

You aren’t restricted to dropping out the masked area in a painting as shown in the 2 examples above. The AutoPaint1 Paint Action Sequence preset located in Default : General shows off a more sophesticated way to use this kind of selection masking.

This PASeq starts off by setting the canvas to white. It then uses the Smart COntrast Ip Op as described above to generate a selection mask. The non selected area is then painted in.  Then the mask is flipped, and a paint preset with a different orientation and a luminance bias is then used to paint in the other side. So one side of the mask is painted in with darker paint that has a different orientation than the pain in the other side of the mask.

The final result for this PASeq is shown below. Note that the PASeq uses the black edge outline technique discussed earlier in this tutorial to improve the edge rendition of the painting.

So, we’ve covered a lot of ground in this tutorial tip, focusing on different approaches to improving edge rendition and representation of the source image in paintings created using the Studio Artist paint synthesizer. We’ve talked about working with different brush sizes to add detail over time to a painting, different approaches to masking, and the use of edge sketches to add additional detail rendition to painting. 

Keep in mind that you can also always mix in some of the original source image to de-abstract an abstracted painting. This could be done in a locally selective way using something like the ‘Default : Utility : Clone Source-Soft’ paint preset. Or you could use the Fixed Image Ip Op with the IP Source set to Source Image and something like a 30% Mix setting if you want to mix the source into the entire painting.