NatCorder and NatDevice: A Crash Course

Recording Gameplay with the Microphone

This article will serve as a tutorial on integrating NatDevice and NatCorder, and an overview of what’s new in NatCorder 1.3f2 that makes integration possible. Let’s jump in.

Let’s Talk About Time

You might have tried to start recording a video at an arbitrary framerate:

But then you realize that the generated video has something else:

What happened to the 10 FPS we passed into the MP4Recorder ? The interesting thing is that the framerate passed into the recorder is merely a hint to the encoder (and even I am not entirely clear how the encoder uses this value). What determines the actual framerate of the recorded video is the spacing between timestamps of committed frames.

Timestamps are an extremely crucial part of recording. Another good example of this occurs when recording with audio. When you record with audio, your video and audio timestamps need to be properly synchronized so that one track doesn’t drift — or isn’t offset — from the other. As you’ll see below, this becomes very important when recording microphone audio with NatMic.

Because of all this, we introduced the concept of Clocks (IClock) in NatCorder 1.3f2. A clock is simply a lightweight object that tells you what the time is:

When recording, a media recorder’s CommitFrame and CommitSamples functions take in a video or audio frame with a corresponding timestamp. The IClock.Timestamp property will give you the timestamp for the exact moment when the property is accessed. Now, let’s see how this fits in with NatMic:

NatCorder + NatDevice

You will likely use NatCorder and NatDevice in one of two cases: recording video with microphone audio; or recording video with microphone audio and game audio (think gameplay commentary). In either case, the usage pattern for using NatDevice with NatCorder is pretty much always the same:

The NatDevice audio device will stream audio sample buffers as they become available from the microphone. We simply forward these sample buffers to the NatCorder recorder for encoding using the IMediaRecorder.CommitSamples function. Note that we use the timestamps from the recording clock when committing audio samples, instead of using the timestamps provided by the audio device. This way, the video and audio tracks of the recording will be properly synchronized.

Pausing Recording

There is one more thing that clocks let us do: manipulate time itself. With clocks, we can enforce a constant frame rate in our video, speed up the video, and so on. In NatCorder 1.3f2, we included one implementation of the IClock interface: the RealtimeClock . As the name implies, the realtime clock reports timestamps with respect to realtime. With the RealtimeClock , you can ‘pause’ recording like so:

It is worth noting that this seems like a weird pattern, as compared to say Everyplay which provides a single pause function. The reason why pausing and resuming is slightly more involved with NatCorder is that we are exposing more of the inner workings of encoding to the front-end. If you notice, when we pause and resume, we actually don’t call any functions on the media recorder. Instead, we simply suspend sending frames to the media recorder for encoding over the period where recording is paused.

That’s about it for integrating NatDevice with NatCorder. If you have any questions, please post them in comments. Also make sure to check out the full NatMicCorder-Demo project. Till next time!

— Lanre Olokoba.