com.overset.dev» Blog Archive » Live Streaming of a Webcam Feed from CANON VB-C10/C50 From a Relay Proxy Server

1.22.07

Live Streaming of a Webcam Feed from CANON VB-C10/C50 From a Relay Proxy Server

UPDATE 10/01/2007 found out that if the input from the cam is 6FPS and the output to SWF via FFSERVER is 4FPS - there’s no lag in camera controls. The scripts below have been changed to pull the MJPEG images from the camera at 6FPS and keep the flash SWF output as 4FPS.

The controllable IP Network cameras form Canon are great. They helped start an odd yet somewhat accepted precedent on the web facilitating user-controlled cameras through the web.

Naturally all the IP Network cameras’ UI Module are written in Java - and have remained unchanged for a some time. This is the point of contention for this write-up and all the digging I have done to find an alternative to this Java UI module.

The crux of these cameras - in a high-availability scenario - is that they offer no proxy of sorts. There are several products that facilitate the replication of a single stream from said IP cameras and duplicate them as part of a streaming server. Again - all of these offerings only support re-use of the java UI Module from either Canon of Sony - a kludge of software and very “closed” in the sense that it’s not very reusable.

With the crux being known, I set out to find an alternative route to interface with the cameras. I must disclaim that this is a proof of concept - yet is not far from being a production-quality piece (similar to Perl-ish kludge-hackery fashion).

My first milestone was to be able to stream the live feed from a Canon VB-C10R Network Camera into Flash Video without relying on the custom Canon UI module so affectionately referred to as “LiveView”.

My second milestone was to be able to “proxy” the live stream as pulled from the camera. This means we’ll only be pulling the stream from the camera once and the proxy server handling all the external requests - naturally a large number of requests.

The first milestone was to figure out how to pull the live video stream from the Canon VB-C10R. To begin - I “sniffed” incoming requests from the LiveApplet example using tools such as tcpdump, ngrep, and snort.

More background information: The Canon VB-C10R (and the VB-C family) offer two methods of accessing the live stream from the camera, via a specialized port, or via HTTP. I didn’t want to setout to try and use the HTTP port because it is actually a slower protocol because it adds a HTTP header for every request. The only advantage of the HTTP protocol to access the stream is so that it’s more accessible over popular firewall configurations. In this case - it’s a direct unprotected link to the camera (private network). This also makes pulling the images as fast as possible. The proprietary port in question for the live stream is 65310. The format for the stream is simple and widely accepted MJPEG (Motion JPEG) video format.

In order to figure out how to communicate in this proprietary format, I used the following snort command (snort worked the best out of all the sniffers to figure out the communication method…) :
# snort -dvq host 10.0.0.1
With the following return set (10.0.0.1 = the server 10.0.0.20 = the camera):

Do not forget that 10.0.0.1 = the server 10.0.0.20 = the camera.

I dissected the protocol and highlighted the requests and acknowledgments in red. The outline of the pulling the stream is a simple process (pseudo code):

open socket to camera server via TCP
send MODE 0\0AVERSION\0ASTART\0A
wait for response: WebView/Livescope Video Capture...
first send\0A\0A\0A\0A
wait for header response
loop send \0A\0A\0A\0A
loop wait for response for next frame in MJPEG

Before the MJPEG “header” is send over this protocol - there is a send “\0A\0A\0A\0A” with a response of “100″ - which is meaningless as far as I can tell. I still write it to my MJPEG stream in the end without fail.

I wrote the following PHP script to pull the MJPEG stream directly from the CB-V10 camera on it’s native 65310 streaming port:

<?

// Set time limit to indefinite execution

set_time_limit(0);

// create TCP socket

$sock = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);

// connect to the camera's native stream protocol IP:Port

socket_connect($sock,"10.0.0.20", 65310);

// handshake with the camera

socket_write($sock,"START\r\n");

// ignore the handshake response

socket_read($sock, 12000);

// daemonize this script

while (true) {

    // simulate 4 escape sequences "\0A" with "\r\n" - or chr(0)

    socket_write($sock,"\r\n\r\n\r\n\r\n");

    // read the image from the stream (max 80k)

    print socket_read($sock, 80000);

    // limit to ~6 frames per second

    usleep(166666);

}

?>

Download this code: streaming_php-pull.phps

This writes the stream as pulled directly from the VB-C10 through its native video streaming port and outputs it directly to STDOUT. PHP was chosen because of simplicity - and is simple enough to port to a more robust C application. That’s on the list of things todo. I would not fear using this in a live production environment so long as there were protective measures in place to ensure this would be daemonized indefinitely.

To output the VB-C10 video stream directly to STDOUT, run this PHP script through PHP-CLI via:
# php -f pull_stream.php
This will flood your terminal if you do not direct STDOUT, but is good for testing to make sure all is well. Don’t forget to reset your terminal if it has been “flashed” - i.e. all the characters are now “funky”.

As per the code - you can change the socket_read() function to attempt to pull a specific amount of data as per the configuration of your camera. Depending on the quality of your images as setup in the camera settings - each frame pulled via this script will be that size. If you get corrupted images in your stream, increase this number. If you want a better framerate than 4 frames per second, as I have chosen, minimize the usleep() function as per how many micro-seconds between frames.

I must disclaim that this PHP script is merely a proof-of-concept and will not pull the exact FPS desired, nor will it be a completely reliable proxy to pull the camera’s stream. Not only that, it cannot gracefully handle disconnects of the camera or the like.

We now have a way to capture the live MJPEG stream from the camera - now what to do with it…? My second milestone was to take the MJPEG stream and convert into native flash streaming video format. FFMPEG and FFSERVER opensource programs are absolutely amazing pieces of work. Not only that, but they manifest my second milestone perfectly.

More detail on the milestone: I want to dynamically take the MJPEG stream and convert it to FLV on the fly. I also want to make the product FLV something that can be streamed from countless clients. FFMPEG does the on the fly conversion and FFSERVER provides streaming functionality.

In sum - I have tweaked and configured FFMPEG and FFSERVER to produce a replicated proxy server that takes the VB-C10 MJPEG stream and duplicates it in FLV real-time to be served to a much wider audience.

FFMPEG and FFSERVER must be configure to work in concert and here are the following commands and configuration I gave to produce the final product:

FFMPEG
# php -f pull_stream.php | ffmpeg -an -f mjpeg -maxrate 500 -r 6 -i - -r 6 http://localhost:8090/feed1.ffm
NOTES: I run the “daemonized” pull_stream.php script, pipe STDOUT directly to ffmpeg via the “… -i - …” arguments. The -i denotes an input file and - for a filename denotes STDIN . The -f denotes forcing of the MJPEG format for the input stream seeing as it will not be obvious to ffmpeg coming from STDIN. The -maxrate 500 limits the bitrate for the input file to 500kb/s. The -r 4 limits both the input and output frame-rate to 4fps (before the -i - affects the input stream and after affects the output stream). Then the output is the last argument is a URI which points to the FFSERVER’s configured default stream feed running on the same server.

FFSERVER - /etc/ffserver.conf snippet

Port 8090

# bind to all IPs aliased or not

BindAddress 0.0.0.0

# max number of simultaneous clients

MaxClients 1000

# max bandwidth per-client (kb/s)

MaxBandwidth 10000
<Feed feed1.ffm>

    File /tmp/feed1.ffm

    FileMaxSize 5M

</Feed>
# SWF output - great for testing

<Stream test.swf>

    # the source feed

    Feed feed1.ffm

    # the output stream format - SWF = flash

    Format swf

    # this must match the ffmpeg -r argument

    VideoFrameRate 4

    # generally leave this is a large number

    VideoBufferSize 80000

    # another quality tweak

    VideoBitRate 500

    # quality ranges - 1-31 (1 = best, 31 = worst)

    VideoQMin 1

    VideoQMax 5

    VideoSize 320x240

    # this sets how many seconds in past to start

    PreRoll 0

    # wecams don't have audio

    NoAudio

</Stream>
# FLV output - good for streaming

<Stream test.flv>

    # the source feed

    Feed feed1.ffm

    # the output stream format - FLV = FLash Video

    Format flv

    VideoCodev flv

    # this must match the ffmpeg -r argument

    VideoFrameRate 4

    # generally leave this is a large number

    VideoBufferSize 80000

    # another quality tweak

    VideoBitRate 500

    # quality ranges - 1-31 (1 = best, 31 = worst)

    VideoQMin 1

    VideoQMax 5

    VideoSize 320x240

    # this sets how many seconds in past to start

    PreRoll 0

    # wecams don't have audio

    NoAudio

</Stream>
<Stream stat.html>

    Format status

</Stream>
<Redirect index.html>

    # credits!

    URL http://ffmpeg.sourceforge.net/

</Redirect>

Download this code: streaming_ffserver_config.xml

This is a limited yet functional ffserver.conf file.

To prove Milestone 1 and Milestone 2 working in concert and to get a stream working, ffserver must be running before running the ffmpeg command.

The next two projects will be:

Encapsulating all this functionality to make it high-availability.
Reverse engineering the VB-C10/C50 camera controls via a Flash Remoting Application.
Started this work here: http://www.overset.com/2007/01/24/canon-network-camera-control-protocol-hack-vb-c50i-control-port-communication/

As always — More to come…

IvanV on 9.13.07 at 5am

Hi, thanks for this post, its helped me out a lot with a little project I’m working on.

Bj on 10.7.07 at 3pm

Very nice tut.. Is it possible to stream with full size like 640×480?
Thx. Bj

Jim Palmer on 10.8.07 at 10am

I think 640×480 would be fine. Even a faster framerate. Getting it just right would require tweaking all the settings just right. You need to make sure bandwidth is plentiful as well considering even 4FPS for 640×480 of MJPEG video even at quality 8 is a lot of bandwidth. As far as I can tell FFMPEG/FFSERVER would be fine supporting it in terms of CPU and load. If you wanted to keep the 4FPS I would only change the resolution that the camera can output and up the bitrate on the ffmpeg to something like 1200 (from ~500) at least.
I’ll give streaming 4FPS at 640×480 a try and see how it affects the CPU/load on the machine, the lag for the end user, the bandwidth consumption, etc.

Bj on 10.11.07 at 12pm

Thx mate.. I just tryed to find a good way to “record” the streams so I can leave out the flash part..
BTW. When I’m recording from this stream (Save the stream from the php script) the playback is very fast. Like X2 speed.
But I’ll try to check back since your the expert on this. :)

Jim Palmer on 10.12.07 at 11am

If all you want to do is save the stream to a file you don’t need FFSERVER. All you need is the PHP script pushing directly to FFMEG. The syntax would resemble something of the sort:

php -f pull_stream.php | ffmpeg -an -f mjpeg -maxrate 500 -r 6 -i - -r 6 -f avi output.avi

Check your FFMPEG installation to see what formats are supported using the:
ffmpeg -formats command. With the syntax above, the “-” character alone means that STDIN is the input “file” which is why we can pipe the MJPEG from the php script directly into ffmpeg. If the video is too fast - you absolutely have to match the sleep() command in the PHP script and the framerate in the ffmpeg command line. If matching doesn’t work very well - then lower the sleep() time in the PHP script so that it will attempt to pull more frames per second and then tell FFMPEG to expect it as a lower frame-rate. This tends to work with the streaming through FFSERVER, but it’ll take a little work to tweak it just right for saving to a flat file.

Bj on 10.15.07 at 4pm

oki, Did try to just direct stream from the cam server -> webserver -> to my pc and it did seem to work “ok”
Only thing that went wrong was that the the image size was the medium and the playback speed was VERY fast.
Reson for that I’m doing this is that the webserver read / store the stream better then just running the script local on my pc. Will try to figure how to set the Image size to max and get the correct speed on the stream for downloading the mjpeg stream.

Bj on 10.15.07 at 4pm

BTW. Can the bitrate/framrate be diffrence between CB-V10 and the VB-50iR ? Maby that the problem with the x2 speed playback?

Jim Palmer on 10.18.07 at 9am

The bitrate and quality setting for ffmpeg shouldn’t affect the speed of playback. The speed of playback is controlled solely by the framerate, i.e. the “-r 6″ setting should force the input and output video in ffmpeg to be 6 FPS. Ensure the the PHP script is pulling an image from the cam at that rate by setting the sleep function to usleep for 1666666 milliseconds. This also might be a problem with using the AVI output format. Have you tried different output formats? I’ve honestly still only been using the SWF output format through FFSERVER which doesn’t seem to have the problem you have. Sounds like a little tweak/testing is in order!

Bj on 10.28.07 at 11am

Hi,

Not tryed other outputs because I dont use the ffmpeg part. I just use the php script to “download” the stream (record) to my computer. But is there a way to stream to my webserver for like 5min and then it stops to stream? I tryed wget to stream to my webserver but only one problem. It never stops. :D lol… Sorry, I’m a noob when it comes to things like this. It seems my download speed is to low to get more then 1-2 fps at 320×256.

Nikolaj on 11.9.07 at 7pm

#!/usr/bin/python
# If you wanted to do it in python
import socket
from time import sleep

s=socket.socket(socket.AF_INET)
s.connect(("10.0.0.20", 65310))
s.send("START\\r\\n")
r=s.recv(12000)
while True:
    s.send("\\x0a"*3)
    print s.recv(80000)
    sleep(1.0/6)

Bj on 11.13.07 at 10am

Is there a way to make a cgi , perl or php script who makes a output file like stream.mjpeg or something like that? Also maby with a stop option or a timeout? Sorry, I’m no “real” programmer. :D

dk on 12.17.07 at 3am

hi everybody,

really surpriced to see simple code to capture video from webcam using PHP…friends cant it be run in internet also…
like if v have a website wr we have a fecility like video conferencing….so where we should do the FFSERVER configuration…it is only for linux OS right….but how it ‘ll be in windows….
when we uploads the site site on the server where we should do the configurations………..
PHP runs at the server side only…so how it ‘ll be capturing the video at the users side(whomever using the web site)?
seeeking for a solution………………

hope i’l get an immed’t reply

John on 12.28.07 at 7am

dk, to be honest, you don’t sound like you know enough about the subject/Linux to be able to create this kind of solution. That may sound harsh, but from reading your post that is the impression I get.

I recommend you try using either Skype or a Polycom video conferencing unit. If you are needing a Windows streaming solution, Windows media server may be more appropriate.

d_blessed_one on 2.5.08 at 4am

hello all,
i am not that much talented in PHP …i wanted to know the config of Windows server for Live webcam streaming…i got some code before but it performance is not that much good..may be because of any mis configuration…plz let me know abt the windows settings that we needed in this…

kate on 3.3.08 at 7pm

Is it possible to use Windows Media Server and pull a stream of video directly from the camera? I see on the Axis cameras there’s code out there to do it. I’m trying to pull in video from a VB-C50ir camera to a server then multi-cast it from there as not to eat up the small bandwidth the camera is connected to remotely.

Jim Palmer on 3.5.08 at 4pm

Lots of interest in the Windows Media Server - yes the FFMPEG and FFSERVER are more suited for Linux, but they can run on Windows.

I would honestly see if there was a way to replicate the PHP script in a native windows language, i.e. C#, VB or the like, and “pipe” the data to a Windows Media Server hook. I can only assume that there is a WMS SDK out there. It still comes down to the simple fact that the VB-C’s use Motion JPEG - which I know can be converted into WMS native encoding. Start by exploring FFMEG on windows.

Another thing - if you want to save the stream from the camera to a file - I don’t think there is anything wrong with routing the output of the PHP script to write to a .MJPEG file.

shekar on 3.12.08 at 6am

Hi,
can any body tell me how to find a video from anywhere is streaming or not using PERL

Steve on 3.25.08 at 6pm

Hi folks,
Does anyone know how to modify Nikolaj’s Python script above so it will work with XBMC ???

Also, has any more progress been made with this so that a Canon camera can be streamed to a web page?

Fred J.G. on 4.9.08 at 7pm

Hi Jim Palmer,

I read in the manual of the vb-c50 that it uses software based on GPL licenses as quoted here from the manual:

“This product uses software based on the GNU General Public License (GPL), GNU Lesser General
Public License (LGPL) and the BSD License. For each of the licenses, see GPL.txt, LGPL.txt and
COPYING.txt in the LICENSE folder on the CD-ROM.
If you require the source code for a program based on GPL or LGPL, please see the following web
site:
http://www.canon.com/webview/webview-tech/
Alternatively contact either the place of purchase or your nearest Canon Sales Company.”

Maybe by getting the sources you can get better control
over the features using your own open source software.
The URL that refers to Canon is not valid anymore but they should give access to the GPL software.

cheers,
Fred

Ebbe on 4.11.08 at 7pm

Just want to point out that you can probably use native linux commands to read the video feed before piping it into ffmpeg. for example curl or wget could probably do the job:

curl $stream | ffmpeg param…..
in this case:
curl 10.0.0.20:65310 | ffmpeg param….

correct me if im wrong.

dirk maas on 4.12.08 at 12pm

Maybe the “AXIS Video Server PTZ Drivers” will be helpful for reverse engeneering, as Canon seems to be a 213 PTZ OEM.
http://www.axis.com/techsup/cam_servers/ptz/driver.php?man=vcc4
VCC-2.04.ptz is for use of Canon PTZ cams with Axis video encoder hardware. I’m close to selling my 6 Canons in favour of Axis. They have much better SDK, documentation and support!

dirk maas on 4.12.08 at 1pm

This is a nice fake:
http://www.spaceneedle.com/webcam/
but how do these guys do it?
http://webcam.sanduskyregister.com/regcam_high2.html#

Krishnan on 7.14.08 at 11am

Great post! Reverse engineering the controls would be great!

Jim Palmer on 7.14.08 at 3pm

Ebbe
Good idea! Never thought of trying curl in this scenario but have used it before for similar integrations. It’s worth a try - I’ll comment again on if this is successful or not!

Krishnan
I started to hack around with the control protocol and posted about it here: http://www.overset.com/2007/01/24/canon-network-camera-control-protocol-hack-vb-c50i-control-port-communication/
What might make more sense to further this work would be to decompile the actual Canon java liveview. I might work further on this, but there are actually a bunch of other blogs with similar work. These cams can be controlled through the internal web server as well which is worth looking at in the formal Canon documentation.

Krishnan on 8.17.08 at 5am

Great, this is too good. No words enough to thank ye… :-)

Krishnan on 8.17.08 at 5am

I am working on video processing at a university. It would therefore be nice to control the camera as the target moves. it may sound terribly foolish but is it possible to control/get packets through a C/C++ program?
I have not seen the Canon documentation very strictly, also i am a complete novice at such hacking. I would certainly like to do it but i do not know even how to begin. any tips/hints would be great.

26 Comments