Efficient Realtime Dashboard with Pusher WebHooks

Today we want to talk about a code update we have made on Panda, taking advantage of the Pusher web hooksreleased a few days ago.

This is a fantastic feature for us as it has considerably reduced the number of api calls made to Pusher, saving us bandwidth and money.

One of the great features of Panda is it’s real time dashboard. Panda can display the new encoding jobs, their progress, and your stats instantly with no effort. Thanks to Pusher, implementing this was really easy.

At such a scale the right design for your software is to separate your app into smaller apps, small apis and workers. In our case, the encoding dashboard is completely separated from the api but they share the same Pusher app. When an event occurs on Panda (like a job progress), the api triggers a Pusher Event and the web app reflect that change immediately.

Before the release, we were facing one problem: we were sending a lot of unnecessary api calls to Pusher.

Panda’s api has no interaction with the user. It only communicates through private and public apis. This means it doesn’t know whether someone is listening to the Pusher channel. We are then pushing thousands of messages even if nobody is looking at the dashboard. The api triggers messages, the web app listens. This is not a two way messaging.

Now with Pusher Web Hooks, our api is able to know when at least one person is looking at the dashboard. As each customer uses a different private channel, that solves our problem.

Our Pusher dashboard is self-explanatory.

Now we would like to show you how we implemented it. First we need to setup an endpoint for the web hooks. Let’s make a small Sinatra app and use the Pusher::WebHook object, which comes with the new gem.

class PusherWebHooks < Sinatra::Base

    post '/pusher' do
        wh = Pusher::WebHook.new(request)
        if wh.valid?
            wh.events.each do |event|
            case event["name"]
                when "channel_occupied"
                    channel_occupied(event["channel"])
                when "channel_vacated"
                    channel_vacated(event["channel"])
                end
            end
        else
            status 401
        end

        return
    end
end

We can setup our notification url inside the Pusher dashboard and we are ready to receive web hooks.

Because splitting your code is the right thing to do, Panda’s encoders are not communicating directly from the api neither. So we somehow need to send this information to them. To do that, we will use Redis Sets as we already use Redis to distribute messages across our farm of encoders.

def channel_occupied(channel_name)
  redis.sadd("pusher:channels", channel_name)
end

def channel_vacated(channel_name)
  redis.srem("pusher:channels", channel_name)
end

Now our Panda encoders can use this information to check whether the channel we are sending the message to is occupied.

 def push_websocket(channel_name, event_name, data)
    if redis.sismember("pusher:channels", channel_name)
      Pusher[channel_name].trigger!(event_name, data)
      true
    else
      false
    end
 end

Redis commands on sets are O(1) operations which makes the design very efficient.

How simple is that? More information is available inside the Pusher docs. Now you have no excuse not to implement Pusher WebHooks. 🙂