Battle of the Matchmovers - An in-depth Review

Battle of the Matchmovers - An in-depth Review

By Christopher Nichols (cpnichols@attbi.com)

Christopher Nichols is an architectually trained digital artist in the Los Angeles area.


The concept of perspective matching may not be uncommon to many architectural illustrators. You have a photograph of your site and you want to insert your building or space into that photograph.

Some 3D packages try to offer you some tools to help you with this. 3ds max (and Autodesk VIZ) for example offer a “camera match” utility to help you. In order for that to work, you need to define points in your 3D space that are common with other points in your 2D photograph. 3ds max (and Autodesk VIZ) then use the data to reconstruct a camera in your 3D space that matches the location, rotation, and focal length of the camera. It has been a long time since I used this tool. While I do a lot of camera matching, I can “eyeball” a better result much faster then the utility can. It seems to never really give me what I need.

With the price drop of technology, Architects are now ready to enter the next dimension in camera matching (pun intended): match moving. As with camera matching, match moving tries to allow the user to derive 3D information from 2D data. However, we add an extra dimension to the equation: time. With camera matching, eyeballing a still image may be a good solution, but trying to do it for a movie is impossible. A camera’s location, rotation, and zoom factor may be different with every frame. It would be impossible to match up every frame by hand. On the other hand, the movie offers something that the still camera does not: movement. Computationally, this is an advantage. When the camera moves through a space, objects that are near the camera move faster through the frame than objects that are further away. Because of this, a new breed of tools have developed that can reconstruct the movement of a camera without defining matching points in 3D space.

In this article we will be talking about three of these tools. Considering the range of price of these tools, you may be asking yourself what one has that the other doesn’t. We will try to answer those questions, but there is one thing to keep in mind. The art of match moving (and it is an art,) is a skill that people have developed an entire career in doing. It has primarily been developed for the movie industry, which uses it all the time. When you see a “behind the scene” shot from a movie where the actors are in green room, with little white balls, someone took the film and reconstructed a 3D space to match the movement of the camera. It is not an easy task. Architects, or even architectural illustrators, may use this tool once or twice a year, if they are lucky. It would be arrogant to say that it is a simple skill that we could master in a few days. On the other hand, these new tools have evolved so that user intervention can be minimal and the results are fairly spectacular. That luxury comes at a price. With that said, lets introduce the tools that we are going to review.

Boujou version 2.0

Boujou, from 2d3, is now in version 2. It is considered by many to be one of the leading match mover packages. In fact, it even won an Emmy. At its current price $10,000, it may not find itself in many architects’ desks. On the other hand it is a very powerful tool that will give you amazing results under many conditions. Boujou is stand-alone software that will analyze your shot and export the data to a number of 3D packages.

Much of what I will discuss here will apply to the other packages that I will be talking about. The idea of match moving is process oriented. Input data: the initial 2D footage. Output data: the motion of the 3D camera (or object). Hence the name of the Company: 2d3. The beauty of that concept, and the interface of the software, is that is all centered on the steps of the process. It outlines all the steps that you need to take on the left of the screen, and shows your progress along the way. An interface of that nature should be used more often for general software tasks that are so process oriented. It makes learning very easy.

I will not outline all the process for you, since you can cover it in the tutorials. However, I will give you the general idea. The first step is to input your video. While that seems simple, understanding what type of video you have is not trivial. Questions such as aspect ratio, pixel ratio, and field order, all need to be understood. The next step is to start the automated tracking. This is where Boujou finds the best data in your 2D sequence to track. If you have ever done this manually, this is a blessing. Not only do you not need to define any points in 3D space to match up to points in your 2D sequence, but Boujou will help you find the best 2D points to track. It also happens to do amazing job at that. The next step is to get up and grab a drink. While Boujou is doing all this work for you, the process can take some time. Boujou 2.0 seems to be relatively fast at doing this. A 187 frame shot tool only about 15 minutes to find and track data. By the way, if you think you are going to easily take your 45 second (or 1350 frame) shot driving through your project and track all the data, think again. This would be a very complex task.

The next step that Boujou does is to interpret that data into 3D data. Again, this process can take some time as well, or around 10 minutes for my 187 frame shot. Keep in mind that the processing time is not just depended on the length of the shot but also on its complexity. However, it should also be noted that Boujou is actually pretty fast at finding a solution for an automated process of this nature.

The next step is to calibrate (or scale) your data. While Bujou has generated 3D data, it has no idea what is up or down, and has not idea how big something is. Lots of simple tools can be used to give Boujou the appropriate data. You can select a bunch of points and tell Boujou that they all lie on the XY plane. You can also pick two points and tell it, the distance between those points. Keep in mind, as I stated, Boujou has been developed for the film industry that is used to have Y as up, rather then Z as up as we are in the architecture. Thankfully, we are able to make it understand Z up under the preferences.
[img Boujou-Screen-02.jpg]

Once this happens, you output your data to your 3D application of choice. I would choose, 3ds max, which, I am assuming would be the most common one to choose for most architectural animators. Boujou doesn’t actually make a “.max” file. It generates a MAXScript file (or “.ms”). When this file is run inside MAX, the data is generated in MAX. The question you may have is what is the data? The data is NOT a model as you may hope. The data is actually a cloud of helper points with a camera. The camera has all of its key frames, and the points are named according to the names that you have in Boujou. This data can be overwhelming. Boujou is so good at finding points to track, that you can easily have a few thousand points or more. For this reason, you may want to rename groups of point that match areas or objects in your shot. You may also want to do this in Boujou before bringing it over into MAX.

This is a very simplistic look at what Boujou does. There are a lot of other “intermediate” steps that you can take advantage of. You can calibrate the shot for the curvature of your camera lens. You can add masks to mask objects that you don’t want to consider in your match move. For example, the car that drove in front of your shot is something that you want Boujou to ignore. “Gold Tracks” has been added which allows the user to hand pick additional tracks that s/he would want to track. You can also add some 3D objects to your scene in Boujou to test and see if the data moves correctly.

MatchMover Pro 2.5

MatchMover Pro 2.5 from REALVIZ offers a set of tools that is very close to Boujou at half the price ($4,999). How close? So close that it is hard to distinguish their interface at first glance. MatchMover Pro has adopted the same interface structure as Boujou has. An almost wizard like process that takes you to the next task step my step.

The steps are very similar. The first step is to import your footage. The same issues exist when importing a sequence. The user needs to know the frame rate, the pixel aspect ratio, etc… You can then simply hit the automatic tracking button and MatchMover Pro will go to work. Once again, it is time to go get a drink, and watch a little TV. The process can take about a half hour for my 187 frame shot.

Once it is done, you will be left with a large dataset of points, similar to Boujou. However, I have found that Boujou seems to create a lot more data than MatchMover Pro. On one sequence for example, Boujou created 1282 and MatchMover Pro on created 367. You may be asking yourself; I really don’t need more then a few dozen, so why would I care if Boujou found more than MatchMover Pro. The key to that is that Boujou may be able to track things that other packages may find to difficult to track. These data points may be an important part of the scene, so that extra data may be important to you. Also, if Boujou can find that data it may give you a better result in the end since it can find areas that may be important for the solution. In essence, it is always good to start with more data. On top of that, it is able to achieve it in approximately the same amount of time. This may be the key factor in the price difference between the two packages.

With that said, you are back on the same track as you were with Boujou. MatchMover Pro can help you clean up your data. It can clean up your data to leave you with fewer points in your final dataset. The self cleaning process is useful, since even 367 data points is a lot to work with.

You are now left to define your coordinate system. Similar to Boujou, you can define your axis, planes, and define a distance. Personally I found Boujou’s process of defining coordinates a little cleaner, but this is a simply a matter of getting used to. The results are the same.

Once that is done, you are ready for export. Here once again, I exported my data to 3ds max, MatchMover Pro writes a MAXScript file out which you can run inside MAX to generate your data.

Again, the steps I outlined are not the limit of what is possible in MatchMover Pro. MatchMover Pro also has the ability to add image matts. It can also manually track areas that you are interested in.

One feature that I thought was really nice with MatchMover Pro is that it has the ability to derive a solution for Nodal Pans, or tripod shots in a unique way. “What is that?” You may be asking. As you know by now, the way that match-moving software works is based on the fact that as the camera moves through space, the foreground moves faster then the background. But what is your camera was not moving in space and you simply rotated it around a node, similar to keeping it on a tripod (or in your hand) and simply rotate around. You get no 3D information. In a sense it is similar to a QuicktimeVR (or other panoramic file format) where there is no 3D information, you are looking at an image inside a sphere. In order to derive 3D information from a nodal pan, MatchMover Pro uses a trick derived from its sister image-modeling software. The key is to collect “helper” frames that are not part of the shot. For example, you could collect one from the left and one from the right. Once you have that you can derive the 3D information since now you have the data you need to understand the parallax.

SceneGenie 1.5

SceneGenie from Autonomous Effects works as a plugin inside 3ds max, and is priced at a more comfortable $995. SceneGenie does not offer the automatic tracking capabilities that Boujou and MatchMover Pro do, but its capabilities as a match-moving package are still very sophisticated. I remember first seeing SceneGenie at Siggraph 1999 in Los Angeles. It was the first match-mover I had seen that was able to give you a solution without referring to data points. For that reason, SceneGenie was far superior to 3ds max’s internal match-mover. To this day, SceneGenie is still impressive.

Lets outline the steps of SceneGenie. In essence follows the same logic that the others do. The first thing that you do is to import your sequence. Once that is done you can use a dialogue called the Master Tracker to take you through the steps of tracking.

The next step is to get your drink and forget about watching the Simpsons. This time you will be doing all the work. So bring your drink over to the computer and start tracking. The major task at hand is to find areas to track and to create Trackers. The advantage of the manual method is that you can select the exact areas that you want to track. You can select the corner of the building, the point on the sidewalk, etc… The more experienced you are, the better you are at finding good spots to track. Here lies some of the art in match moving. SceneGenie can then track the sensors that you have selected. You can manually tweak each sensor using the Fast Tracker dialogue that is fairly sophisticated. While I’m sure this task can become monotonous, I actually found it enjoyable to find spots to track.

Once you have tracked all your sensors, you can set up a coordinate system. Once again, a manual match-mover has an advantage at this point, since you can tracks areas that you want specifically to define axis and distance. These are known as constraints. In SceneGenie, setting up constraints is fairly clean.

Once you have set up constraints, it is time to find a solution. In essence, you simply click on “go” and it will find your solution. The solution may not always be very good. The main reason is that the user may not be very good at finding and tracking spots. This does not mean that you have to start over. SceneGenie can help you find problematic trackers and improve their tracking. You can also add more tracks. This all comes with experience. So is manual tracking just too painful since there is a need for so much input from the user? The answer is no. There are some clear advantages to a manual solution. If an automated match mover gives you a poor solution, it is harder to control it and tweak it. SceneGenie gives the users control over what they are doing. It is also able to diagnose problematic tracks and help predicts for you where you may need to revise their paths. This is very helpful when readjusting your solution. Also with much fewer tracks compared to MatchMover Pro or Boujou, it is a lot easier to “keep track” of what is going on. This amount of control will start to be extremely advantageous with more experience.

Sequence comparison

For the next step I want to look at a one of the many tracks that I tried in all three packages. On my vacation through Arizona, I took my DV cam and shot a bunch of video with the idea to using it for match-moving. The easiest shots to track would be a smooth shot from the side window of my car while driving down the road. My wife was driving so I was not putting myself in danger. A slightly more challenging shot is to look forward. The following is that type of shot. I decided to choose a shot with some high and low contrast areas to compare how these zones get tracked.

Download Original Video Test Footage (884KB)

In order to be fair to some of the very pricey packages, I decided to give myself a time limit of 45 mins as to how long I would spend with each package tracking.

Once I did my track in each package, I tried to place objects in the scene and see how stable they would appear. I also looked at the data and try to derive how close they there to my predicted measurements. With that, lets look at some of the results:

Boujou:

Boujou did very well. The shot is the most stable of all three, and was the easiest to set up. On this shot, it gave me over 1099 data points, some of them in very low contrast areas. However, there is an issue that I found surprising. I defined the sequence as having a fixed but unknown field of view, I therefore expected all the match-movers to come up with an accurate reading of the cameras viewing angle. Boujou had the strangest result. With a much narrower field of view then the others, at 26 degrees. Because of this, the dataset actually looked narrow in the X axis, and long on the Y axis. It was not enough of an error to make the match-move look wrong, but it was enough an error to make it hard to locate areas in the dataset, or scale things accurately.

Download Boujou Matchmoved Video Footage (884KB)

MatchMover Pro:

MatchMover Pro also did fairly well. Having only tracked a scary 666 points automatically, I was forced to add a few hand tracked points to help derive the constraints I wanted to use, since MatchMover Pro did not pick up those areas. Luckily, it was not very hard to do so, and it did not eat up too much time on my time constraint. I also noticed that some areas that MatchMover Pro did not pick up compared to Boujou, were in the areas with lower contrast. I then brought in the data into 3ds max. I was please to see that the data actually looked a lot closer to what I would expect. The field of view was at a more logical 35 degrees. After rendering the sequence, you will notice a frame or two that jump radically out of position. From my experience in MatchMover Pro, this type of “error” was fairly common. Luckily it is also easily fixable. When you display the path of the camera, or look at it in Track View, you can quickly see where the erroneous frame exists, and manually adjust it.

Download MatchMover Matchmoved Video Footage (884KB)

 

SceneGenie:

SceneGenie suffered the most from my test. The biggest problem was the time constraint. I was actually able to get a result very quickly. It was not a very good one. I had time to adjust my solution, which gave me a better result, but it was still off from where it needed to be. I then was forced to stop due to my deadline of 45 mins. The result you will see is based on how far I was able to get in that allotted time. I suspect that there are a few things that would have helped me get a better result. The first being more experience at match-moving. Without it, it may have taken me up to a day to fix, tweak, add all the different trackers in the sequence. Hopefully, the next time, I would be able to cut that time down. However, it is fairly unlikely that, as an architect, I spend a great deal of time training myself in the fine art of match-moving.

Download SceneGenie Matchmoved Video Footage (884KB)

 

In concluding, I would probably tell you what you already suspect. If you have a big project with a very big budget, Boujou may be the tool for you. It has been well trusted by the Movie industry and is indeed a powerful tool. I was a little surprised when it gave me a strange FOV angle on my data set, but this could easily be user error. On the other hand MatchMover Pro is an extremely competitive product at half the price. While it would be nice to see it track areas of lower contrast, it still can give you some really nice results. SceneGenie involves the most work, the most training, and the most experience. If you don’t spend the money on software, you will spend it on the amount of time it takes. The one big advantage I see over the other packages, is that while Boujou and MatchMover Pro offer manual tracking options, they are not as sophisticated as SceneGenie. If tweaking and control is your game, SceneGenie may be the best tool for you.

Product Websites:

Autonomous Effects - Scene Genie

REALVIZ - Matchmover

2d3 - Boujou

 

If you would like to post comments or questions about this review, please visit our forum