DavidRelo

Posted on Mar 11, 2022 • Edited on Jun 1, 2023

Built an App to Watch Movies Together Online in a Few Hours (source code inside)

#android #java #video #twilio

How did this project get started

My girlfriend and I live apart, so we don't get to see each other in person very often. One day, when we were on video chat, she said: "Hey, it has been quite a while since the last time we went to the cinema together. When can we watch a movie together again... "

She was right. It has been a while...

Though we still couldn't get to see each other immediately, maybe we could watch a movie together online. I am aware there are many apps out there that people can use to watch movies together online. Especially since the pandemic started, more and more such apps and services have been launched to meet people's needs for staying connected. But, a thought suddenly came to my mind: wouldn't it be a cool thing if I could build one by myself?

As I am a developer working for ZEGOCLOUD, a cloud-based real-time engagement API platform that helps developers quickly build real-time audio/video and in-app chat capabilities into any apps, I have all the stuff I need to build such a watch together app.

Then, I couldn't resist the thought and started building.

...

A few hours later, I got something looks like this:

As a developer, I love building things, and I love sharing how things are built. So, in the following sections, I am going to outline some important steps of the implementation, and at the end of the article, I also provide a link to the source code of the entire project.

The implementation

Set up the development environment

Android Studio (version 2.1 or later)
Android SDK 25, Android SDK Build-Tools 25.0.2, Android SDK Platform-Tools 25.x.x or later
An Android device (Android 4.1 or later) with a microphone and camera
Android is connected to the internet.

Prerequisites

Here are a few things that need to be done first before you can start building with ZEGOCLOUD.

Create an account with ZEGOCLOUD. Obviously, I already have an account, so I can skip this step.
Create a project in the ZEGOCLOUD Admin Console and obtain your AppID and AppSign, which are required for SDK authentication. For details, see ZEGOCLOUD Admin Console-Project management.

Now, we can start coding!

Feature overview

This is gonna be a simple app with the following features:

There will be two participants joining a room to watch a movie together. Let's call them viewer A and viewer B.
To ensure that the two participants can have a truly synchronized watching together experience, I decided to play and stream the movie to the room from a separate device. Let's call it the movie streamer.

The consideration in this decision is that if the movie is played and streamed to the room from the device of one of the participants, then the streaming latency may result in a perceivable out-of-sync issue between the two participants. But if the movie is played and streamed from a separate device, then both participants will be subscribing to and playing the movie stream from the cloud, and the probability of having a perceivable out-of-sync issue will be much lower.

Participants in the room can have video chat and text chat while watching the movie together.
To make it simple, in this project, I set the maximum number of participants in a room to 3.

The movie streamer side - Play a movie and stream it the room

Step 1: Create a ZegoExpressEngine instance.

/// Define the ZegoExpressEngine object
ZegoExpressEngine engine;
/// Specify the AppID and AppSign for SDK authentication
/// AppID Format：123456789L
long appID = ; 
/// AppSign Format："0123456789012345678901234567890123456789012345678901234567890123"
String appSign = "";
/// Create a ZegoExpressEngine instance
engine = ZegoExpressEngine.createEngine(appID, appSign, true, ZegoScenario.GENERAL, getApplication(), null);

Step 2: Enable custom video capture for the engine.

ZegoCustomVideoCaptureConfig videoCaptureConfig = new ZegoCustomVideoCaptureConfig();
// Use RAW_DATA as video buffer data type
videoCaptureConfig.bufferType = ZegoVideoBufferType.RAW_DATA;  engine.enableCustomVideoCapture(true, videoCaptureConfig, ZegoPublishChannel.MAIN);

Step 3: Set a callback handler for custom video capture and implement the callback methods.

// Set the engine itself as the callback handler object
engine.setCustomVideoCaptureHandler(new IZegoCustomVideoCaptureHandler() {    
     @Override     
     public void onStart(ZegoPublishChannel channel) {         
     // On receiving the onStart callback, start to capture video and send the captured video frame data to the ZegoExpressEngine.

        ...     
     }    
      @Override     
      public void onStop(ZegoPublishChannel channel) {         
      // On receiving the onStop callback, stop the video capture process.
        ...     
      }
  });

Step 4: Join a room.

/// Create a user
ZegoUser user = new ZegoUser("userA"); 
/// Join a room
engine.loginRoom("room", user);

Step 5: Set up a video event handler for the ZegoMediaPlayer. By setting up this handler, you can receive the video frame data of the movie being played through the callback onVideoFrame.

mZegoMediaPlayer.setVideoHandler(
    new IZegoMediaPlayerVideoHandler() {
        @Override
        public void onVideoFrame(ZegoMediaPlayer zegoMediaPlayer, ByteBuffer[] byteBuffers, int[] ints, ZegoVideoFrameParam zegoVideoFrameParam) { 
        if (RoomManager.getInstance().isCanSenRawData()) {
            int totalDataLength = byteBuffers[0].capacity();
            if (tempByteBuffer == null || tempByteBuffer.capacity() != totalDataLength) {
                tempByteBuffer = ByteBuffer.allocateDirect(byteBuffers[0].capacity()).put(byteBuffers[0]);
            } else {
                tempByteBuffer.clear();
                tempByteBuffer.put(byteBuffers[0]);
            }
            ZegoSDKManager.getInstance().getStreamService().sendCustomVideoCaptureRawData(tempByteBuffer, tempByteBuffer.capacity(), zegoVideoFrameParam);
            }
        }
    }, ZegoVideoFrameFormat.RGBA32);

Step 6: Start stream publishing. Load the movie from the specified file path (local file path or an URL that points to an internet media resource), and start playing the movie.

/// Start publishing the stream
engine.startPublishingStream("streamMovie");

mZegoMediaPlayer.loadResource(path, new IZegoMediaPlayerLoadResourceCallback() {
    @Override
    public void onLoadResourceCallback(int code) {
        if (code == 0) {
            mZegoMediaPlayer.start();
            if (callback != null) {
                callback.onLoadResourceCallback(code);
            }
        }
    }
});

Step 7: When there are no other participants in the room or the participant who is publishing stream leaves the room, stop publishing the stream, and the callback for stopping custom video capture will be triggered.

ZegoSDKManager.getInstance().getDeviceService().setCustomVideoCaptureHandler(new IZegoVideoCaptureCallback() {
    @Override
    public void onStart(ZegoPublishChannel channel) {
        canSenRawData = true;
    }

    @Override
    public void onStop(ZegoPublishChannel channel) {
        canSenRawData = false;
    }
});

The movie viewer side - subscribe to the movie stream to watch together, video chat, and text chat

To realize the watch together feature, we will need to implement the following:

The viewers subscribe to the video stream of the movie being played so they can watch it together.
Both viewers can have control over the movie playback. For example, either one of them can pause and resume the playback.
Stream publishing/subscribing and camera/microphone operations for video chat while watching the movie.
Sending and receiving real-time text messaging in the room.

The high-level program logic for movie playing:

The movie streamer will create a room first, and the viewers need to join the same room (by specifying the same Room ID when calling the loginRoom method).

If viewers attempt to join a room that doesn't exist or without a movie streamer in there, they will be promoted with a message Room doesn't exist.

If the specified room has already reached its full capacity, then the room login will fail, and the user will be notified accordingly.

When a viewer clicks the Start Movie button, call the setRoomExtraInfo() to send a notification to the movie streamer. Upon receiving such notification through the onRoomExtraInfoUpdate() callback, the movie streamer will execute the logic to start playing the movie and start streaming the movie to the room.
The movie stream will then be received by both viewers and played on their devices.

/// start playing stream
engine.startPlayingStream("stream1", new ZegoCanvas(play_view));

Both viewers can call the setRoomExtraInfo(String roomID,String key,String value,null) method to control the movie playback (Play or Pause).

The following table shows the key-value parameters you can pass to the setRoomExtraInfo method to send out different notifications:

key	value
roomInfo	0: movie being loaded... 1: movie playing 2: movie on pause 3: room has been closed

The high-level program logic for video chat:

After joining the room, viewer A (or B) can start publishing video from the camera with a unique Stream ID and also start the local preview.
When the other viewer joins the room and starts publishing his/her video to the room, the ZEGO Express SDK will send out an event notification accordingly. Viewer A (or B) then can start playing the other viewer's video using the Stream ID received from the callback notification.
Both viewers can control their camera (e.g., switch between front/rear camera, turn on/off the camera) and microphone (e.g., mute/unmute) as they wish.


/// Switch between the front/rear camera
expressEngine.useFrontCamera(front);

/// Turn on/off the camera
expressEngine.enableCamera(enable);

/// Mute or unmute the microphone
expressEngine.muteMicrophone(!enable);

The high-level program logic for text chat:

Both viewers can send and receive real-time text messages in the room.
Call sendBroadcastMessage method to send a Broadcast Message (no longer than 1024 bytes) to other participants in the room.

/// Send broadcast messages
engine.sendBroadcastMessage(roomID, msg, new IZegoIMSendBroadcastMessageCallback() {
   @Override     
   public void onIMSendBroadcastMessageResult(int errorCode, long messageID) {   
    /// Returned result of sending broadcast messages       
   }
 });

Implement the callback method onIMRecvBroadcastMessage defined in IZegoEventHandler to listen for and handle the Broadcast Messages sent by other participants. You can get the details of the message from this callback, including the message content, message ID, time message sent, and message sender.

  /// reveice message
public void onIMRecvBroadcastMessage(String roomID, ArrayList<ZegoBroadcastMessageInfo> messageList){          
}

How the finished app looks like (source code download link attached below)

The movie streamer side

The viewer side

You can install the following demo apps and try them out yourself if you wish to do so :

Download the demo

How to use the demos:

Start the movie streamer first, select a movie, and then set up the Room ID.
Start the viewer app and then join the room by specifying the same Room ID.

Download the source code

Conclusion

Through this sharing, I hope you will find that building an app for watching movies together is actually not as difficult as you might think. Ready to build one yourself?

DEV Community