Motion Capture & Virtual Production

    Finally after weeks of hard work we closed on a rough but finished product. The motion capture I performed and processed were utlized throughout the whole production. And I was very satisfied with the end result.

    This sprint was focused on finishing out the rest of our animations. I assisted in recording both sets of our motion capture data as our actor. I processed the new cowboy animations and the characterized and processed the female animations.

    Processing the data can be the most time consuming part but that is simply due to the sheer number of recordings done. Otherwise with the pipeline I found for doing them it takes no time at all. It's the same pipeline I gave to my fellow animator while I was away. It ensures the most success and has two fail safes to avoid issues.

    There hasn't been a change in technique. Processed 10 animations as of this sprint. With 8 left for the robot character. My home computer hasn't been working well with AcuRig so I can't make a useable rig myself. This may result in making some cuts unless my colleague was able to finish that part.

    With the processing being taken care of I look to my colleague to counter animate fingers and such on the single take our team chooses to lead with. Due to scheduling conflicts I relied on my colleague to close out the few animations left to process, counter animation, and sequencing. I notice my new team is mainly focused with vertical slices. Given our numbers, equipment, and skill I believe that we do fantastically well despite our limitations. 

    Doing work like this and developing a pipeline has been very fulfilling. Its a great feeling to come in to a team and to solve issues right off the bat. It ignites a great sense of confidence in terms of working in an industry setting where the job is solving problems and providing results. 

    Likely our focus is aimed at refining our animations and sequences for a final, final product.

    Above is a clip of the sequencer I put together with the motion capture done prior to my introduction to the team. The data we have jumps from position to position but tweaking the visibility makes it easy to cut the chaff and use what is needed in a shot. If needed the sequencer can be adjusted.

    So a major issue that my team was having was getting motion capture data from the Xsens to work in Unreal. I immediately took to what I know and went through Motion Builder to hopefully get it work. Luckily the pipeline I found worked beautifully and expedited the process.

    So the pipeline is as follows.

Desired Rig into Motion Builder

    ↳Characterize the Rig

        ↳Save out Characterized Rig

            ↳Merge Motion Data With Characterized Rig

                ↳Characterize Motion Data

                    ↳Change Characterized Rig Source to Motion Data

                        ↳Bake Motion Data to Characterized Rig Skeleton

                            ↳Bake Motion Data to Characterized Rig Control Rig 

                                ↳Select Characterized Rig Skeleton

                                    ↳Export Characterized Rig Skeleton

Take Characterized Rig into Unreal

    ↳Import the Characterized Rig

        ↳Import Characterized Rig Skeleton(s)

    The pipeline appears to be long but truly it isn't. The 35 individual motion data files given to me took little to no time at all each: roughly 6 minutes per file. There are around 18 new files to be processed but with this pipeline myself or my fellow animator will take no time at all.

    For the new 18 files I acted in the Xsens suit. This was a fun reminder of my time in the motion capture course. This time around I believe we tried to take into account the environment rather than what I believe to have been done prior.

    The production has all but closed and we have our final product. Working out the last of the technical issues in live compositing consumed a large portion of the last sprint but this sprint has yielded us as close to a finished product as we can get given our experience.

    This time around a large portion of focus was getting sounds recorded and getting our footage compiled.

    First I took to recording our victim and our monster. I tried to avoid over editing the monster sounds which came out as good as it could have given my limited experience in sound design. The monster was surprisingly easy. A few takes to find the right pitch, but fairly easy. The human sounds was way more difficult. It became a matter of matching up with the video. After spending around an hour and a half I settled on a recording that was as close as I could get.

    So following the arduous recording process I sought sounds on Freesound.com for the helicopter and thunder. From there I made the edits in Premier Pro.

    Getting the sounds to align with the video took some time. I played with some of the filters and effects in Premier, but felt that it made the whole production feel far too cheesy: far more than it already was. I wasn't overall satisfied with the final product, but given the limitations was content. I recognize the amount of work that it takes to make virtual production look good. Overall I feel as though I learned a lot from this and look forward to trying my hand at more within the field.

              

    This sprint has drawn to a close. A great deal of the sprint was spent troubleshooting live compositing. Yet I remained productive. Our production is set in a post apocalypse. So I sought to create a look indicative of meek soldier, likely stationed to the store room during his service. 

    Our scene lacks context aside from dank, dark, and dirty. I took to applying a tattered, rough look to all clothing. 

                    

    I started by heading to the Army Navy Surplus Store in town. I picked up a cheap work shirt I felt resembled that of a low ranking soldier. I used art charcoal, a razor, and tea bags to survival damage the shirt. Charcoal used heavily to imply crawling through and surviving amidst rubble. The tears were to show that this individual has been through a lot. I wanted to singe some areas of the clothes but felt that it would be overkill.

                      

    Given the simple nature of our production I opted to implement some props to give the actor business. So I brought a crucifix to give some humanity to the character. Taking it out of his shirt to kiss it or hold it to his head. All he has left is a sliver of hope that the audience knows isn't going to get him far. The polaroid is one of my wife. For context its a photo of her in front of her parents house before we moved, just blurred for privacy. This is to add another human layer to our character. He's looking at a photo of a loved one, we can only assume is deceased as he soon will be.

                         

    The under shirt was given the same treatment as the work shirt. Applied to the areas where visible rather than just a total weathering.

      

    The pants were so dark they didn't benefit from the charcoal as much but it was still used for continuity. I put the pants on and slid around on my knees on the concrete to get some initial weathering and took a razor at angle to fray it. Some cuts and tears here and there and viola. Aged pants. I made sure to really tear at the bottom for more of that tattered look.

  

    A large portion of the sprint was spent trouble shooting live compositing. The first week spent in the studio room and the next in the class. After a great deal of tweaking and adjusting the camera, green screen, plug-ins, and project file I'm confident we can get a recording done before the deadline. The set up and break down was time consuming as opening a project file. Nonetheless, we are close to a rough finished product. 

This sprint comes to a close with a fair amount of progress made. I was tasked with getting fog in our scene, chromakeying the test green screen footage, and running a test recording.

    Getting the fog into the scene was just a matter of adjust the sliders between Ultra Dynamic Sky and Ultra Dynamic Weather. We first thought that one overrode the other, but actually it just makes one more prominent than the other. The fog came from the Ultra Dynamic Sky and was turned up to an appropriate amount. I used a directional light with low intensity to add more density to the fog. There was an option for volumetric fog but it reduced visibility to zero no matter how much I adjusted the slider. I believe it was being influenced by the Ultra Dynamic Weather. 

  The chromakeying had a fair share of complications. It took a lot of plug-ins enabled and a lot more of troubleshooting. I had the issue of the footage on the plane not playing unless I opened up the media player window and played the video in the background. Otherwise the material and instance wouldn't run. I had trouble setting up the video at first. After I got it working I just couldn't figure out how to change the brightness.

    The last part was getting a blocked out animation. The mesh we initially worked with had issues that stunted progress so a substitute was employed. A basic walk animation was used from mixamo to flush out the scene a touch. It's rough but I can see the value that can come from virtual production like this.

  This sprint draws to a close. This time around I was assigned with basic lighting and sequencer shots. 

I was to find where the camera would be placed within the scene. A difficult task compositionally yet I settled on the final placement as this is to be a short reaction shot. I experimented with various lighting options, but I believe that for the atmosphere we look to create simple spotlights with a yellowy tinge would suffice at least for the block out. 

Once I had the basic layout I used the sequencer to keyframe location and rotation information on the two meshes and a simple toggle of the spotlight intensity. I learned quickly to save often to avoid loss of work from frequent crashes. Yet after expanding the timeline and playing with the timing I reached a point of satisfaction with the product. I had to account for the actor's business on screen so the final shot may be shorter or longer give or take.