Hi everybody!
As of version 1.1.7, you can now integrate video files with alpha channel straight from the workflow tab and capture a video snapshot. This amazing new addition means that you can now pitch augmented reality concepts and will be able to execute them easily.
In this tutorial, we will show you how to accomplish a Multi Presets experience with AR animated figures that joins the final photo.
TABLE OF CONTENTS
1. Creating Content:
a. Green Screen Production:
Let's start with the essentials stuff:
- Green Screen Backdrop
- Lighting Kit (at least three lights)
- Video capture device (Webcam, DSLR, Arri Alexa)
- Measuring Tape
The first thing is planning, of course, you want to know how the experience will look. From there we will work backward.
In our case, we had to produce, post, and integrate with only a few days on our hands. We invested in shooting exactly what we need.
To capture the video we used a 4K Webcam: the Logitech Brio. to support the illusion of the AR figure being “really there” with you. We captured our Actors in the same measured space of the final experience.
Step one: Set
We used a 3X5m Green Screen backdrop, we built the backdrop about 3.5 meters from the camera so we will have a substantial “stage” with the backdrop filling the frame. The camera-backdrop distance differs with different cameras. Next set the angle and zoom (if available) and mark the exact camera position and lock it. In our case, we mounted the Webcam about eye level. Finally, test with the Actor at the perfect point for the end-user to be standing (framing wise) and mark it.
Step Two: Lights
Now we are getting to the good part, as the illusion will take place in “Live View” conditions we will have to carefully think and plan for the same light conditions when we are producing the content and also running the experience if the light will not look the same the illusion might not work and the final result will not look believable. As we were filming in our offices and also shot the promotional video in the same spot we were in full control of how it will look. We used separate light kits for the backdrop and the subject.
Lighting the backdrop – we want to make sure the backdrop is evenly lit if you are using a fabric backdrop check to see that there are no wrinkles that might cast shadows. We used a small continuous LED Lights kit with two lights. We positioned the two lights one from each side of the Green Screen backdrop between the subject and the backdrop, we set the height to the middle of the backdrop so it will be evenly lit.
Lighting the Subject - Naturally, you will want to use a three-point light system. Under the circumstances, we decided to use the fixed office lights.
Step Three: Action:
Just before you will be ready to shout “Action!”-
We captured the video using Arcsoft Webcam Companion software. With that said, you can use any webcam software that you seem fit, as long as the software is capable of controlling the camera settings such as Exposure, White balance, Focus, Etc...
Use the best quality and resolution you can get. This will be important as in post-production we used 4K resolution and since it’s a webcam we had to use compressed video files. We recorded single video files, we did 3 options for each Actor so we can later add more options for the experience.
An important note here is timing- since you will be cutting the video files to the exact length you will use in the experience, think of the last frame of the video. In our case we filmed the Actors entering the frame, positioning themselves for the photo taking and posing. We counted two seconds in pose and then leaving the frame. In this single video we cut into two video files so for each actor/player we have “in” and an “out” video, but not just yet, let's jump into the second part of this tutorial.
b. Post-Production:
This is the main part of creating your experience, but first, you will need some tools:
- Powerful PC (Windows/Mac)
- Adobe After Effects CC
Before we start, a quick note- this part is usually done by professionals, but it is totally doable even if you are not yet a professional. I recommend jumping to Youtube if you need help understanding some of what we are about to do.
We made a list of step-by-step instructions for you to follow.
Make sure to watch the tutorial video alongside this form:
[Can't see the video? Click here to watch on YouTube]
Step 1: After Effects:
1. Import the selected footage into After Effects.
2. Right-click the footage file and select “New comp from selection”.
3. In the top menu, go to Composition → Composition settings.
4. Switch between the width and height parameters to change the video ratio from Horizontal to Vertical. The parameters are:
- For 11" iPad: 1688 x 2388
- For 12.9" iPad: 2048 x 2732
5. In the composition timeline, press on the little arrow on the left of the selected footage and go to: Transform → Rotation, and set the rotation to 90°.
6. When the footage is selected on the timeline, select the rectangle tool or the pen tool from the top toolbar, and create a mask around all the parts of the footage that don’t contain a green screen. To mask them out, change the mask mode from “Add” to “Subtract”.
7. In the effects pannel's search bar, type “key” and drag the “keylight + key cleaner + advanced spill suppressor” Preset on your footage layer.
8. On the Keylight 1.2 effect (in the effect control), use the Eyedropper tool to select a green area of the footage.
9. On the Keylight 1.2 effect (in the effect control), change the “View” mode to “Screen Matte” and open the “Screen Matte” tab.
10. Refine the effect using the parameters Below Such as: clip black, clip white, etc.
11. On the Keylight 1.2 effect (in the effect control)Change the “view” mode to “intermediate result”.
12. To eliminate green reflection on the subject, in the effect control, turn on “Advanced Spill Suppressor”
13. Find your subject’s “In-Point” and adjust your clip-layer to the start point of your timeline.
14. Duplicate your composition by right click → duplicate (or CTRL + D).
15. Rename the two compositions “subject in” and “subject out”.
16. Inside your “subject out” composition, find your out point and adjust your clip layer. Your start\in point should be the moment where the subject starts moving again. Your finish\out point should be when your subject exits the frame completely.
17. To adjust and trim the workspace area, go to Composition → Trim comp to work area in the top menu.
18. Select “File → Export → Add to Render Queue. Go to your first and composition and repeat.
19. Under the “Render Queue” tab, press the “Output Module” and select the “Lossless with Alpha” option for both compositions:
20. Click on "Format options" and choose Apple ProRes 4444:
21. Under the “Render Queue” tab, press “Output To”, select your rendered video file’s name and destination.
22. Press the “Render” button (on the upper right’s “Render Queue” tab) and proceed to Part C.
c. Animation Creation:
This video tutorial will teach you how to create your own animations using Adobe Animate:
[Can't see the video? Click here to watch on YouTube]
* Make sure that the size fits your iPad- For 11" iPad: 1688 x 2388 and for 12.9" iPad: 2048 x 2732
2. Converting The File For iPad:
* This part can be done only on a Mac
The first thing we need to do is to export the file to a .mov file, so we will open Adobe Animate software and choose the file we want to convert.
1. Once we have that click on File -> Export -> Export Video/Media:
2. Choose the Format is 'QuickTime' and the Preset is 'Apple ProRes 4444 with alpha'. Make sure that the 'Start Adobe Media Encoder Render Queue immediately' option is checked and then click Export:
3. The media will be exported and the .mov file will be created. An Adobe Media Encoder will open and save the file.
Once we have the exported file we can convert it to be HEVC .mov H265 file - this action will also reduce the file size as it is a large file:
1. Right click on your chosen file - > select 'services' -> Choose 'Encode Selected Video Files'
2. In Settings, choose the maximum HEVC resolution -> check the 'Preserve Transparency' box and click continue.
3. Once you clicked on Continue, the file will be processed and once it's done, you can see the new file and its reduced size.
3. Integration With The Software:
Here we have an example of the workflow we made, which is available in the Preset pack (1.1.7 and onwards):
We will add an Animation state and load the “in“ animation of the subject - make sure to add “Live View start” Feature state beforehand.
In the Layout tab the subject of the animation is already part of the overlay (and can be found in the Preset pack -> Assets -> Overlays -> AR_Figures).
We added the “Out” animation after the “Photo” feature to keep the experience feeling more natural.
That's it! All done!