Path Start Regionization

The key to recreating Art Mapper style effects in Studio Artist v4 is to utilize the paint synthesizer’s path start regionization features. All of the regionization path start generators intelligently analyze the source image and break it up into component regions. Each of these regions is then processed by the paint synthesizer one at a time.

At the time this tip was written the regionization path start generators include the following options:

• Watershed Regionize
• Adaptive Block Regionize
• Cellular Regionize
• Cell Partition Regionize
• Color Simplify Regionize
• Selection as Regions (all, full on only, all except full on)
• Flat Region Canvas as Regions

The first 4 regionization options include normal and DM options. The DM options include an additional intelligent split of each generated region, and will better represent the features of the source image by preserving edge transitions.

There are several different approaches to programing the paint synthesizer when working with path start regionization.  The individual regions could for example be painted in with normal paint strokes. The Path Start Regionize and Path Start Regionize use RegionDraw Pen Gen Path Type options in the Path Shape control panel can be used with path start regionize generators to create a wide variety of different fill paths for painting in the generated regions.

Selection Derived Regionization

There are a number of different path start regionization options derived from the current selection. The selection as regions – all levels generator option is especially useful for expanding the range of potential region generation effects. Both the Vectorizer and Image Operation effects can be configured to output to the current selection as opposed to the raster draw canvas. So you could use the vectorizer to transform a source image into regions and route the vectorizer output to the current selection. You could then run the selection as regions – all levels path start generator and the paint synthesizer will act as if the vectorizer derived the path start regions it will be filling in.

Working with the selection buffer as an intermediate is the way to infinitely expand the options for generating different kinds of path start regionization shapes. You can record the component vectorizer and paint synthesizer steps in a PASeq to generate a single preset that generates the overall regionization effect. Whatever processing effect you use in this way to drive path start regionization via the selection buffer, you want to make sure the effect generates a series of flat colored regions as it’s output.

Generating the Region Brush Fill Nib

To duplicate the Art Mapper we want to either map images and or MSG preset processing to each generated region. So we really want to configure the paint synthesizer to preform image processing effects on each generated region area. The best way to do this is to use Region Fill as Brush pen mode.

Normal region fill as brush pen mode auto drawing takes the path generated by the paint synthesizer and converts it into a region that is then used as a substitute for the normal paint synthesizer source brush for a paint fill. However, since the path start regionization generators are directly generating raster image regions we can use them directly to create the region fill as brush paint nibs. The way to do this is to use the new v4 Selection Override options for the Generator parameter in the Region as Brush Mode control panel.

Keep in mind that you need to have the Path Type option in the Path Shape control panel set to Path Start Regionize if you want the Selection Override Generator setting to pass through the individual path start regionize region shapes directly to the region fill as brush fill operation. Otherwise it will just use the existing selection buffer for all of the fills.

Photo Mosaic Effects

MSG Evolver generated photo mosaic effects by working with an image database generated from a folder of images (or movies). Each image database (call a Poster Index in Evolver terminology), consisted of 3 different files (poster, index FV or feature vector).

Studio Artist does not currently work with Evolver image database files. It uses movie files as a way to access collections of images. You can load a movie file as a movie source brush or a movie background texture. Once a movie brush is loaded into the paint synthesizer, it is analyzed and feature vectors are available internally for the paint synthesizer to make intelligent decisions about choosing specific image frames in the movie that best represent spatial areas in a source image when constructing a photo mosaic.

The region fill as brush pen mode works to override the normal source brush settings, replacing the source brush with the generated region that will be used for a fill instead of the normal brush. So if you want to use a movie source brush in region fill as brush pen mode you need to configure the paint synthesizer a certain way. You want the region fill as brush generate region to be used as the alpha channel for the temporarily generated source brush while still using the loaded movie source brush for the RGB channels of the generated brush nib. The way to do this is to use the Alpha Only Brush Option setting in the Region Fill as Brush control panel.

And to use the Source Alpha Brush Brush Type option in the Brush Type control panel.

Colorization

The Art Mapper also offered a number of different colorization options for the sub images that make up a photo mosaic effect. Colorization processing can be useful to help generate a photo mosaic that better represents the source image. The way to build colorization processing into a photo mosaic paint preset in Studio Artist is to use the image processing brush load features of the paint synthesizer.

There’s a large number of different image processing brush load Algorithm settings that can be used for colorization, including all of the Paint Color Mean Shift and Paint Color Hue algorithms.

MSG Art Mapper Effects

MSG Evolver’s ArtMapper offered the ability to construct composite art images derived from multiple applications of MSG presets in spatially local subareas of a large art canvas. You can think of this as being like a photo mosaic, except that a MSG derived procedural image is generated for each sub area of the larger automatically constructed art canvas. The MSG generated images could be totally procedural, or source processing effects that processed images in the image database with an MSG effect for each sub image of the larger art canvas.

The way to duplicate this MSG derived art mapped image in Studio Artist 4 is to use the MSG Brush Load features of the paint synthesizer in conjunction with the path start regionization features described above.

MSG Brush Load will use the current MSG preset to generate the brush load paint nib. When used in conjunction with the path start regionization settings described above the MSG preset will be used to generate the fill image for each region generated by the path start regionization algorithm.

More Information

Some of the information covered in this tip has been described in other tip or news tutorial posts.

How to make a custom movie brush and use it in building photo mosaic paint presets is described in this tip.

Paint Regionization is discussed in this tip.

The following Studio Artist News blog posts discuss photo mosaic effects, photo mosaic presets,vectorizer derived paint regionization.

The Studio Artist User Guide pdf includes a detailed explanation of all of the paint synthesizer controls, including explanations of all of the path start regionization generators.

Converting a Folder of Images into a Movie File

The first thing you need to do is put together a collection of the individual images you want to use in your movie brush and put them in a folder.

There are lots of different approaches to choosing a set of component images for a photo mosaic. You might be choosing imagery related to a common theme. Or you could choose imagery that has some specific color or textural characteristics. Depending on your ultimate goal for the photo mosaic you are going to create you should choose the individual images accordingly.

For the movie brush i created for the example shown above i was choosing component images based on a biohazard theme. I used google’s image search to pull up various biohazard signs, and downloaded the appropriate ones to a folder.

Let’s assume you have put together a collection of component images for your photo mosaic. You now have to convert the folder of images into a movie file. This is easy to do in Studio Artist. You start by recording a simple one step Paint Action Sequence (PASeq) that sets the canvas to the source image. You then run an Action menu that processes a folder of images with a PASeq into a movie file.

Recording the PASeq

To record the PASeq, bring up the Paint Action Sequence palette if it is not already up in your workspace. You can use the appropriate Window menu if necessary to open any hidden palette.

Then use the erase button in the toolbar at the top of the PASeq palette to erase the existing PASeq. Then press the Rec checkbox in the same PASeq toolbar to turn on recording.

Once PASeq recording is active anything you do in the Studio Artist interface associated with manually or automatic actions in the canvas will be recorded as individual action steps in the PASeq. To set the canvas to the source image, you can run the Canvas : Default Color : Source Image menu shown below.

When you run this menu a new action step labeled ‘SetCanvas-source image’ will be recorded in the PASeq list, as shown below.

You could have also just dragged and dropped the source image into the canvas to do the same action.

Now that you have recorded the single action needed for this PASeq, press the Rec checkbox at the top of the PASeq palette to turn off PASeq recording.

Processing with a PASeq to Convert a Folder of Images Into a Movie

To process a folder of images with the PASeq you just made, run the Action : Process with Paint Action Sequence : Image to Movie… menu shown below.

After you run this menu you will be presented with a series of 3 different dialogs. The first is a standard file dialog that asks you to choose a folder to process. Use it to select the folder of images you want to use in your movie brush.

After you choose the folder you want to process, you are then asked to specify a size for the output movie as shown below. Choose an appropriate size for the movie frames. Note that all of the images in the folder you will be processing will be resized accordingly to fit the aspect ratio and size you are specifying in this dialog.  

After you specify the size of the output movie, you are then presented with a standard file dialog that lets you specify the name and location of the movie file you will be generating.  

When you press ok Studio Artist will scan the contents of the input folder you specified. Each image file it finds in that folder will be loaded into the source are one by one. The PASeq will be run with that source image. After running the PASeq then a new movie frame will be output. This process repeats over and over again until all of the images in the input folder have been processed.

If you press cancel in the Create Movie Stream dialog the processing will still take place but no output movie file will be generated. This can be useful for previewing what a PASeq will do prior to commiting to a full render run.

If you need to stop a PASeq processing Action midstream you can do so. Simply press the spacebar key.  When you do so you will get a dialog asking you if you want to stop or continue processing. Press the appropriate button to stop or continue processing.

After all of the images in your input folder have been processed with the PASeq you will then have a new movie file containing all of the individual images in the input folder. Depending on your movie preference settings the finished move file may open in the Quicktime Player when processing is finished.

This movie file can then be used as a movie source brush or movie background texture in the paint synthesizer.

Building a Movie Brush Paint Preset

To import your new movie brush file as a movie source brush in the paint synthesizer, just run the File Paint Synthesizer : New Movie Brush… menu shown below.

You will get a standard file dialog that asks you to choose the movie file you want to use as the new movie source brush. Select the appropriate movie file.

If you then go to the paint synthesizer Brush Source control panel in the Editor you will see that it has been configured to use your movie file as a movie source brush.

In this particular example i used the bioHazardA1 movie file that contained the individual biohazard sign images i wanted in my custom movie brush.

A movie source brush can be used in many different ways in the paint synthesizer. So depending on what you want to do with it you still have some additional paint synthesizer editing to do to complete your custom preset.

A good approach when getting started is to use an existing movie brush preset as a starting point before you run the New Movie Brush menu command. This way you can just load your custom movie into the existing preset and then you are ready to use it.

If you are a beginning user then you are now ready to create your own custom movie brushes to use with existing photo mosaic paint preset. The section below is for more advanced users and gets into the nitty gritty of how to use advanced features in the paint synthesizer to create custom photo mosaic presets from scratch.

Advanced Topic – Creating Photo Mosaic Presets from Scratch

Creating a photo mosaic preset totally from scratch requires a good understanding of how the paint synthesizer functions as a whole. Below we will walk through a fairly sophisticated preset so you can see how it was put together.  

As mentioned above, the easy way to get started is to choose an existing photo mosaic preset, add your own custom movie brush, and then do some minimal editing of that already working preset if you want to customize it further.

A common editing strategy for building photo mosaic presets in the paint synthesizer is to use its auto-regionization functionality. You don’t have to do this, you could build a more conventional paint preset that uses the movie source brush directly as a paint nib. Auto-Regionize features are more difficult to edit from scratch but allow for a much wider range of shape and positioning effects for the individual elements of the photo mosaic. You can duplicate most of the MSG Evolver Art Mapper functionality directly in the paint synthesizer by clever programming. 

The example image i created displayed at the top of this tip is using a new version 4 adaptive block regionization feature. This was setup in the Path Start control panel by choosing the Adaptive Block Regionize DM generator.

There are several different Regionize path start generators. They include the Adaptive Block, Watershed, and Color Simplify regionize generator options. They all look different, and have various controls that can also affect how they work, so it’s worth some experimentation to choose which one is right for the particular effect you are trying to achieve. The regionize options with the DM addition to the name will intelligently split each generated region to better represent the edge structure in the source image.

The Max Stroke setting is used to determine the number of generated regions. For the Watershed Regionize option this will be the exact number of regions generated. Increasing the Max Stroke setting for Adaptive Block options will increase the number of generated regions, but the actual number generated will not be the same as the Max Stroke Setting. The Max Stroke setting is working like the complexity control, increasing it generates a more complex mosaic with more regions. 40 to 400 range would be typical settings for an Adaptive Block path start generator.

For Simplify regionize, the Max Stroke Setting works like the # of colors option in the Simplify Ip Op.  So you want a fairly small number, say 3 to 12 for normal operation.

The screen snap below shows the Path Shape parameter settings i used for this example. I’m using a specific Path Shape option that works in conjunction with the Regionize path start generators. It is called Path Start Regionize.  

I’m using the Outline region pattern type, which creates a path that is the outline of the individual region generated by the path start generator.  

For this particular example i’m using the Auto Region Fill as Brush pen mode. Region Fill as Brush is a special pen mode that generates regions that are then filled in as on the fly generated paint nibs. So they paint very differently than the typical Studio Artist paint model of individual paint nibs drawn along a vector path. In the Region Fill as Brush case the path is used to define the shape of the nib region that is then rendered into the canvas. This works well for building a photo mosaic because we want each generated region to correspond to one component image of the generated photo mosaic.

The traditional approach to building a photo mosaic is to lay down a regular rectangular grid of equal sized individual images. You can certainly do that using the paint synthesizer. But you also have the ability to create much more interesting spatial arrangements that use adaptive block size and positioning as well as irregular freely drawn shapes for the individual components of the overall mosaic. The regionize path start generators work to automatically position the generated regions to intelligently represent the source image.

What gets drawn inside the individual generated regions is a function of how the controls in the Paint Fill Setup control panel are configured. As shown below, i’m using the Paint Brush Load Fill From option, and the Canvas Image Fill To option.

The Paint Brush Load controls show off a good way to work with a movie source brush. The brush is used as the Source for an Image Processing Brush Load effect. A Paint Color Mean Algorithm is used to colorize the brush image on the fly to match the current source color when painting.

As mentioned before, this preset is using Auto Region Fill as Brush pen mode. So, we need to setup the Region as Brush Mode control panel parameters for this preset, as shown below. I turned on the PathStart Regionize functionality for this pen mode with the first parameter below.

To create the individual photo mosaic regions, we want the generated path shape to define the shape of the region and we want the source brush to be filled in inside the region. Now the way Region as Brush mode normally works it overrides the Brush Source control panel settings since it can be thought of as generating a dynamic brush shape on the fly. But we don’t want this to happen since we want to use the movie source brush we have specified to fill in the generated region shape. We can get around this problem by setting the Brush Option parameter in this control panel to Alpha Only and then use a Source Alpha Brush Type.

What this does is to take the output of the Region as Brush Mode controls and route that to the source brush alpha only instead of the entire source brush. So we end up with a dynamic brush alpha channel that masks the brush shape and the movie source brush we originally specified for the RGB channels of the brush that are filled inside the dynamic shape.

We’ve covered a lot of ground in this preset, so it’s worth a recap. We configured the paint synthesizer to create an auto-regionize photo mosaic effect using a movie source brush. We used an adaptive block path start generator. This generator analyzes the source image and breaks it up into a collection of adaptive blocks positioned to intelligently represent the source image. The path shape controls are set to generate paths that track the outlines of each generated region. These outline paths are then converted by the Region as Brush Fill pen mode controls into a dynamic alpha channel for the source brush. The RGB channels of the source brush are generated from the custom movie brush we loaded. The image processing brush load module is used to colorize the individual regions to match the part of the source image they correspond to when they are rendered into the canvas.

Filling in Cracks Due to Incomplete Coverage

Depending on which path start generator you are using for this kind of effect you may not get complete coverage of the canvas. The approach to deal with this is to use something like the Geodesic Interpolator Image Operation to fill in the cracks. So you would start off by setting the canvas to a default white background. Then you would run the paint synthesizer action that generates the photo mosaic. Note that there are white cracks in the mosaic below at the edges of the individual regions.  

You would then run the geodesic interpolator ip op to intelligently fill in the white cracks. Note the Geodesic Interpolation parameters are setup to fill in a white background. This ip op effect extrapolates the adjacent colors to intelligently fill in the specified color background, in this case all the pure white parts of the canvas.

After running the Geodesic Interpolation ip op note that the white cracks have all been intelligently filled in with color. 

You now have your finished photo mosaic.

And you’ve gotten an intensive overview of how the paint synthesizer works, and how it can be configured to create auto-regionize effects.

An alternative approach to working with Region Fill as Brush pen mode for crating photo mosaic images is discussed below that is useful for avoiding any crack artifacts. It uses a new version 4 feature and is somewhat simpler than the full path generation process discussed above.

Directly Using the Path Start Regions 

There’s an alternative approach to using Region Fill as Brush pen mode for generating photo mosaic effects. The previous section discussed how to generate outline paths for the region.  The individual path start regions were converted into paths and then back into regions for the fill process. But since the path start regionization process generates regions itself you could just use those regions directly for fills in Region Fill as Brush pen mode.

The trick to do ing this is to use a new v4 Generator option in the Region Fill as Brush control panel called Selection Override. What this option does is use the current region selection as the region to be filled.  

Behind the scenes when you run a path start regionization process the paint synthesizer sets the current region selection is set to the currently drawn path start region. This is useful because you can mask your painting based on the original path start region boundaries. But it also provides a way to bypass the path generation conversion process discussed above and any potential shape variation or cracking artifacts associated with the region fill.

Keep in mind that you need to have the Path Type option in the Path Shape control panel set to Path Start Regionize if you want the Selection Override Generator setting to pass through the individual path start regionize region shapes directly to the region fill as brush fill operation. Otherwise it will just use the existing selection buffer for all of the fills.

Directly using the original path start regions as the region fill as brush fill regions avoids the potential for cracks or other artifacts created by the region to path to region conversions discussed above.

Additional Information

There’s a Studio Artist News Blog post entitled Creating Photo Mosaic Effects in Studio Artist that provides an overview of different approaches to building different kinds of photo mosaic effects.