Wednesday, March 24, 2010

VLC HTTP Live Streaming module patch (iPhone streaming support)

As noted in the post iPhone HTTP Live Streaming - Between a rock (Microsoft) and a Hard Place (Apple) I created an access module for VLC to allow VLC to support the Apple/iPhone HTTP Live Streaming protocol. There is also a basic outline of how HTTP Live streaming works in that post.

Thanks go out to ioncannon.net's post on his FFMPEG based HTTP live streamer which gave me a tool to play around with/learn from before attempting this patch for VLC.  It is also a good place to look for more  information on HTTP live streaming.

Here is the patch:

This patch should no longer be needed, as livehttp is included in the latest VLC git trunk, and should also be included in the nightly builds.

vlc-livehttp.patch



Notes:
  • This has only been tested using H.264 w/MP3 or AAC audio using mux=ts, and raw MP3 using mux=raw
  • I've been mostly an FFMPEG guy till now, so forgive me if my VLC understanding/terminology is somewhat off. 
  • This plug-in should support both Live and non-live HTTP Live streaming feeds, depending on the options passed to the module.

Instructions:

The name of the module is livehttp, and is specified by specifying "access=livehttp"

Options:
splitanywhere= (default: false)
Tells livehttp to split the stream anywhere, not just on video keyframes.  Currently required to be set to true for audio-only streams and not recommended (probably won't work) for video streams.

seglen= (default: 10)  
How many seconds of audio/video each segment should contain.  Apple recommends 10, I have been using 5.

numsegs=(default: 0)
  The number of segments to keep in the index file.  The default of 0 keeps all segments in the index (which you would want for non-live streaming).  For live streaming the specification require at least 3.

delsegs= (default: true)
Delete segments as they are no longer needed.   If numsegs=0 this parameter is ignored (as all segments are assumed to be needed)

dst=
This is actually an option to the access std module.  The path of the segment files to write.  The # characters get replaced with the segment number.  So a path of "seg-###.ts" will end up with files called "seg-001.ts, seg-002.ts, seg-003.ts" etc.

index=
The path of the index file to write, which will contain the "playlist" of video/audio segments to stream.  Recommended to end in .m3u8 by specifications.  This is the file the <video> tag should be pointed to.

index-url=
This is the URL that corresponds to the dst above (how a browser would access the dst file).  The # characters get replaced same as in the dst parameter.  Note:  The filename portion of this URL will most likely need to be in the exact same format as the dst parameter.  So for example if dst=/www/seg-##.ts then the index-url should be something like index-url=http://mydomain.com/streams/seg-##.ts  (Note the same number of # characters)

rate-control=(default: false)
If set to false the there is no rate control (the muxer sends the data as fast/slow as it can to the streamer).  If set to true the muxer should do rate-control to control the speed to muxed audio/video is sent to the streamer.  I'm a little unsure what the "best" setting for this parameter is when re-streaming an existing video stream.  I've been leaving it at false...

Examples:
All examples assume the following:

  • The Web Server root directory is /var/www
  • The domain name of the web server is mydomain.com
  • The stream segments &amp; index files will be written into /var/www/streaming/ and will be accessed via http://mydomain.com/streaming/...
  • The destination stream name index file will be called "mystream.m3u8"
The following HTML will allow you to view the video based on the above on an iPhone:
<html>
  <head>
    <title>Video Test</title>
    <meta name="viewport" content="width=320; initial-scale=1.0; maximum-scale=1.0; user-scalable=0;"/>
  </head>
  <body style="background-color:#FFFFFF; ">
    Audio/Video Stream:<br/>
    <center>
      <video width='150' height='150' src="http://mydomain.com/streaming/mystream.m3u8" />
    </center>
  </body>
</html>

Re-stream a live video feed:
vlc -I dummy --mms-caching 0 http://www.nasa.gov/55644main_NASATV_Windows.asx vlc://quit --sout='#transcode{threads=2,width=320,height=240,fps=25,vcodec=h264,vb=256,venc=x264{aud,profile=baseline,level=30,keyint=30,bframes=0,ref=1,nocabac},acodec=mp3,ab=96}:duplicate{dst=std{access=livehttp{seglen=10,delsegs=true,numsegs=5,index=/var/www/streaming/mystream.m3u8,index-url=http://mydomain.com/streaming/mystream-########.ts},mux=ts{use-key-frames},dst=/var/www/streaming/mystream-########.ts}}'
Create a VOD stream:
(Non-live.  When this command finishes, all the segments should have been created and the index file contain pointers to all of them)
vlc -I dummy /var/myvideos/video.mpg vlc://quit --sout='#transcode{threads=2,width=320,height=240,fps=25,vcodec=h264,vb=256,venc=x264{aud,profile=baseline,level=30,keyint=30,bframes=0,ref=1,nocabac},acodec=mp3,ab=96}:duplicate{dst=std{access=livehttp{seglen=10,delsegs=false,numsegs=0,index=/var/www/streaming/mystream.m3u8,index-url=http://mydomain.com/streaming/mystream-########.ts},mux=ts{use-key-frames},dst=/var/www/streaming/mystream-########.ts}}'
Re-stream a live audio feed:
vlc -I dummy --mms-caching 0 http://www.nasa.gov/55644main_NASATV_Windows.asx vlc://quit --sout='#transcode{acodec=mp3,ab=96}:duplicate{dst=std{access=livehttp{seglen=10,delsegs=true,numsegs=5,index=/var/www/streaming/mystream.m3u8,index-url=http://mydomain.com/streaming/mystream-########.mp3},mux=raw,dst=/var/www/streaming/mystream-########.mp3},select=novideo}'



 Possible improvements/fixes:
  • Have the module auto-detect audio only streams, so the splitanywhere option is not required.
  • I'm not sure I am doing the right thing with the Win32 rename function.  Linux (I believe) allows me to rename a file over an existing file, even if the existing file is in use.  Win32 is not so friendly.  This ability is useful for updating the index file at same time it may be currently being read by the HTTP server serving the files.
  • Break the dst= and index= parameter into seperate filename/directory entries, so you only need to specify the filename format once. (instead of once for the dst= parameter, and once for the index-url= parameter)

66 comments:

  1. Do you know if there's a way to view an HTTP Live stream in VLC?

    ReplyDelete
  2. Bobby--

    Unfortunately I don't believe there is at the current time.

    ReplyDelete
  3. How do I add the module to VLC? I have applied the patch to VLC source, but when I compile I still get 'no sout access module matched "livehttp"' from vlc.

    ReplyDelete
  4. Sonix:
    You should be able to compile as normal (bootstrap, configure, make, make install), the exact instructions are going to depend on your OS. You can search the web for VLC compile instructions on various OSes.

    To make sure you are running the version of VLC you think you are running you can type "vlc --version" and if it is one of the newer versions from git it should be a version like "VLC media player 1.1.0-pre1 The Luggage" . You can also type vlc --list and make sure you see "access_output_livehttp HTTP Live streaming output" which is the livehttp modules.

    ReplyDelete
  5. Thank You!
    I am restreaming RTSP to Live HTTP and your module is doing great :D

    I love how it does all the indexing and removing of old files. With ffmpeg/live_segmenter I had to do it with php.

    ReplyDelete
  6. Where did you get your VLC code? Did you get it from their Download link, or did you get it from the git repository. I got mine from the Download link, and I'm having trouble getting it to run. I wonder if mine is not as update as yours.

    ReplyDelete
  7. It's based off of the vlc git repository:
    git clone git://git.videolan.org/vlc.git

    ReplyDelete
  8. Thanks. This is my first open-source experience. I managed to get "The Luggage" version installed. (Verified as you described above.)
    I think I applied the patch correctly. When I do vlc --list, I see
    access_output_livehttp File stream output
    The second part isn't exactly what you said above. But, I'm taking the first part to be the important one.

    Do you know anything about the following problems?
    Blocked: call to unsetenv("DBUS_ACTIVATION_ADDRESS")
    Blocked: call to unsetenv("DBUS_ACTIVATION_BUS_TYPE")

    I googled the first one and got 2 hits. :0
    One I did not understand at all, and the other was "not found".

    I'm workng on Fedora 10.
    These aren't all the problems. But, I'm starting from the top.

    This is a wonderful thing you have done. I so hope I can find a way to use it. Thank you.

    ReplyDelete
  9. Interesting work! thanks! Is it possible for this to stream a live pvr:// feed from my tv input card?

    ReplyDelete
  10. Estelle--

    Sorry, I'm not sure what those errors are from. It may make sense to try and compile VLC from git without the patches first and see if you can getting running that way. Once it is working apply the patches and re-compile. (I don't think those errors are livehttp related...)

    As an aside, I am hoping the VLC team will accept the patch into the main VLC branch so it will be included in the nightly builds... They are just finishing a "release" cycle and are starting to accept new features.

    ReplyDelete
  11. re: PVR card

    Shouldn't be a problem, I'm pretty sure I had feedback from another user that they successfully streamed from a TV card to the iPhone. (I'm not sure of the exact models/details).

    I have definitely tried it myself with a dshow:// webcam and it worked fine.

    ReplyDelete
  12. Hi,
    I had no luck with "The Luggage." I could not even get it to open a file and play it. So, I retrofitted the patch to 1.0.3. And used rpm to build and install it. It actually plays videos better than it did before.

    The LiveHttp preferences show up in the Preferences. (Good sign)

    I am still not able to run it from the commandline. Is there a way that I can activate the LiveHttp through the gui?

    I've tried the streaming option, using the sout parameters from your live streaming example above.
    For the output file, I used the /var/www/http/stream/mystream-###.ts

    Any thoughts?
    Thanks.
    -Estelle

    ReplyDelete
  13. Kerry,
    Hey. Just an update...
    I'm still trying with the gui because I can't seem to get it to take in a udp stream from the commandline and there's always a dummy error as well. I'm making some progress using the gui however.
    In the sout box, I use your whole command after --sout for the "restream a live video feed example."
    The error I'm getting is "no mux specfied or found by extension"
    If I leave out the transcode stuff, it generates the .ts and index files, that the http server serves up. But alas, the phone can't use it. Safari displays it however. I feel like I'm close.....

    ReplyDelete
  14. Hi Estelle--

    It does sound like you are getting close. You need to make sure that whatever options you are selecting in the GUI will create the following format file:
    1. MPEG-TS container
    2. H.264 video
    3. MP3 audio

    It sounds like maybe you are ending up with some other video format which livehttp can split, but the iPhone can't play.

    ReplyDelete
  15. Yes, that's it exactly. It seems that the rpm fusion x264 encoder has been disabled in 1.0.3 because it is out of date. I'm installing the version from the git repository now. Hopefully, this will be compatible.

    ReplyDelete
  16. Keary - got this working (with static files and with pvr://)! Thank you so much! This is great..

    John

    ReplyDelete
  17. Terry,
    If the video coming into VLC is already h264 MpegTS, is there a way to get around trancoding? The video I'm interested in doesn't even have audio.

    ReplyDelete
  18. You might be able to get around transcoding, but it's probable that at a minimum the ts-muxer (the code that put's the h264 into a mpeg-ts container) needs to be run, as it set's some flags during the muxing process that livehttp needs to look at. I'm not sure of the exact process to do this however. (Either from the gui or command line, but it's probably possible at a minimum from the command line)

    ReplyDelete
  19. Hey Terry,
    Turns out the problems were in Fedora 10. When I upgraded to Fedora 12 it just worked. It's great. Thank you so much.

    ReplyDelete
  20. Keary
    oops. Pardon the wrong name. I'm working with a Terry. Also, I'm really tired.
    Great work. Thanks.

    ReplyDelete
  21. Thanks for your great work!
    I can build it in win7 with msys, now I can watch dvb tv in my ipad!
    And it seems that MoveFileExW failed to return error, I have to write another script to move temp index file.

    ReplyDelete
  22. Hello,

    I added the patch to 1.0.6 and recompiled but when doing the make I got following error.
    Any idea what might be wrong?
    Thanks

    livehttp.c:42:20: error: vlc_fs.h: No such file or directory
    livehttp.c: In function Open:
    livehttp.c:193: error: implicit declaration of function vlc_unlink
    livehttp.c: In function updateIndexAndDel:
    livehttp.c:347: error: implicit declaration of function vlc_open

    ReplyDelete
  23. The VLC module API changed at some point (I haven't tried the patch against 1.0.6, but have tried it against 0.8.6 and had the same problems).

    You'll either have to use VLC 1.1 or try to backport the patch to the older VLC versions.

    The module should already be included in the latest VLC 1.2, so the patch is not needed.

    ReplyDelete
  24. This comment has been removed by the author.

    ReplyDelete
  25. KEARY, May I know which version of VLC you patched? Thank you

    ReplyDelete
  26. Hi, thanks for your code, it's really great and I used it to make my own TV streaming site (for personal use).

    Is there any way of making multiple quality streams for multiple connections so that iPhone detects the bandwith and read the TS file that will best play on the phone?

    As far as I know that is possible with ioncannon.net's development but I could never get it to work and your code works perfect...

    Thank you!

    ReplyDelete
  27. Mickey,

    The original patch was of VLC 1.1, but it has been merged into the official VLC 1.2 (available from their git repository or in the nightly builds), towards the end of May.

    ReplyDelete
  28. Hernan,

    Take a look at http://tools.ietf.org/html/draft-pantos-http-live-streaming-01, sections 6.1.3 & 7.4

    Basically you can (by hand) create a "super" index file that points to multiple index files created by vlc, and put that index file in the HTML instead of the one generated by VLC. For example, superindex.m3u8:
    #EXTM3U
    #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1280000
    http://example.com/low.m3u8
    #EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=2560000
    http://example.com/high.m3u8

    This says for bandwidths > 128k use low.m3u8, and for > 256k use high.m3u8

    Then just run VLC twice with the appropriate parameters to create low.m3u8 and high.m3u8 (make sure you use a different base name for the segments also...)

    VLC may be capable of creating both streams at the same time, but I'm not sure.

    ReplyDelete
  29. Thank you Keary, I'll try that...

    ReplyDelete
  30. I'm trying to re-stream a live feed, as shown in the post. I keep getting error messages:

    [0133104c] stream_out_standard stream out error: no mux specified or found by ex
    tension
    [0131faec] main stream output error: stream chain failed for `standard{mux="",ac
    cess="'#transcode{threads=2,width=320,height=240,fps=25,vcodec=h264,vb=256,venc=
    x264{aud,profile=baseline,level=30,keyint=30,bframes=0,ref=1,nocabac},acodec=mp3
    ,ab=96}",dst="duplicate{dst=std{access=livehttp{seglen=10,delsegs=true,numsegs=5
    ,index=mystream.m3u8,index-url=http://10.0.0.52/mystream-########.ts},mux=ts{use
    -key-frames},dst=mystream-########.ts}}'"}'
    [012e7d0c] main input error: cannot start stream output instance, aborting


    I'm not sure where/why the standard{mux="" part is coming from. I'm using a nightly build from 6/15, I've tried other builds with the same result. Not sure what I'm doing wrong here... anyone have suggestions?

    ReplyDelete
  31. Forgot to mention, I'm running this on Windows 7...

    ReplyDelete
  32. Kendall--

    My best guess is that it relates to how windows handles quoting on the command line. I can't really be of more help, I had trouble myself getting the command line just right for windows (during testing).

    The command lines posted were tested on Linux.

    ReplyDelete
  33. VERY important note - "x264 aud" option is mandatory, otherwise iphone wont be able to play the stream. I spent hours of testing before I read this post.
    THANK YOU!

    ReplyDelete
  34. Apples own mediavalidator complains about the playlist not beeing completely valid.
    There is missing a "," in the EXTINF line.

    Here is a diff patch against the VLC gode in git (1.2.0)

    --- modules/access_output/livehttp.c.ORG 2010-07-15 11:53:49.000000000 +0200
    +++ modules/access_output/livehttp.c 2010-07-15 11:54:30.000000000 +0200
    @@ -275,7 +275,7 @@
    fclose( fp );
    return -1;
    }
    - val = fprintf( fp, "#EXTINF:%zu\n%s\n", p_sys->i_seglen, psz_name );
    + val = fprintf( fp, "#EXTINF:%zu,\n%s\n", p_sys->i_seglen, psz_name );
    free( psz_name );
    if ( val < 0 )
    {

    ReplyDelete
  35. One other note.
    sometimes Apples mediavalidator says it can't find a *.ts segment. I guess it is because the code deletes the *.ts segments a little too agressive. Could the code be tweeked so it leaves the old files for a greater time.

    I guess a formular could be
    delete files older than number_of_files_in_playlist * segment_length.

    for example: playlist with 5 segments of 10 seconds - then delete files older that 50 seconds.

    ReplyDelete
  36. Bjarne:

    Thanks for the EXTINF patch-- I missed that in the spec (and the iPhone didn't seem to care). If you could submit the patch to the vlc-devel@videolan.org group so you could get credit for it in the VLC git-log that would be great. Here are VLC's instructions for submitting a patch:
    http://wiki.videolan.org/Git#Submitting_patches_to_the_vlc-devel

    If you'd rather I do it just let me know.

    ReplyDelete
  37. Bjarne:

    As far as the problem with deleting TS files too early, if I'm thinking about it correctly, the formula you propose is basically what I use...

    I delete the TS file when it falls out of the playlist. Assuming 5 segments of 10 seconds each this means it will get deleted only after there are 5 newer segments in the playlist (i.e. 50 seconds of newer video in the playlist).

    I guess I could always keep 1 additional TS file around (so 60 seconds in the above example...)

    ReplyDelete
  38. Could you provide the ./configure parameters you provided to compile VLC?

    I'm having issues with the following error: stream_out_transcode stream out error: cannot find video encoder (module:x264 fourcc:h264).

    (The livehttp is showing when I issue a vlc -list command and my version is VLC media player 1.2.0-git Twoflower (revision 1.1.0-pre1-1270-g5ca3019))

    ReplyDelete
  39. JW:

    It looks like it is missing libx264 support (--enable-x264). However this parameter should be enabled by default if it can find the libx264 library so my guess is that it's not installed. (Or it's installed but configure still can't find it...)

    ReplyDelete
  40. Got it working... x264 was missing...

    Any recommendations on only inputting one stream to multiple bit rate outputs?

    ReplyDelete
  41. Hi,

    i'm using VLC nightly build v1.2.0
    My m3u8 playlist file isn't updating itself as the segments get created. Only a temporary file is getting the updates properly (playlist.m3u8.temp).
    Any chance to get VLC to play back the segments without intrruptions between them ?
    Would it be possible to implement a multi-thread download feature into VLC so that it can download each segment as download managers / accelerators do ?

    ReplyDelete
  42. Hi,
    Is there a VLC build (installable) available with livehttp module included for Fedora 10? If Fedora 10 is a problem ( I notice someone had issues), let me know which Fedora release may have a build available. ( Other OSes?)

    FYI - I downloaded the latest VLC ( 1.1.4) source to compile but it does not have livehttp in it.

    Thanks for your help.

    ReplyDelete
  43. @Kerry.

    News problems:
    iPhone Developer Program now refuses to accept my streaming solution based on your excellent module.

    The reason it gets rejected is that the (stupid) person at Apple runs the mediastreamvalidator for 60 seconds. So if we run VLC with let say 10 seconds clips - the module deleted the *.ts clips before the Apple mediastreamvalidator get to play them. Resulting in a HTTP 401. And then iPhone Developer Program rejects the application.

    I suggest a setting with like a "clipstokeep" where you can specify how many old clips should be stored backwards. Ie the possibillity to keep the latest 6 x 10 sec. clips in order to make Apple happy.

    I have commented out the unlink that deletes the *.ts files and is running a cronjob which cleans up old files. But I guess that it would be cool to have this function inside the livehttp module?

    Regards,
    Bjarne

    ReplyDelete
  44. Bjarne, I had the same issue with them.

    The other issue is that the stream needs a lower than 64 kb audio stream with a valid format. Did you run in to this issue also?

    ReplyDelete
  45. @JW

    http://developer.apple.com/library/ios/#technotes/tn2010/tn2224.html#//apple_ref/doc/uid/DTS40009745

    Not lower than 64 kbps. But a least 1 stream must be 64 kbps if you send a multi-rated stream.

    And yes - I have included an 64kbit audio-only stream to satisfy iPhone Developer Program.

    Regards,
    Bjarne

    ReplyDelete
  46. Yeah, I just meant 64 kbps or lower. My App just got approved yesterday.

    ReplyDelete
  47. Hi! I am trying to use this amazing VLC module, but I'm having some trouble. I tried looking in the forums, since there seems to be more people with the same problem, but I can't solve it. The main error that appears is the following:

    access_output_file access out error: cannot open 'D:/someexistingfolder/somenewfolder/stream-####.ts' (No such file or directory)

    It seems to be some permissions problem. However, you can read it detailed here: http://forum.videolan.org/viewtopic.php?f=4&t=80302#p276325

    Could you please give me some hint?

    Thanks!

    ReplyDelete
  48. hi Keary,

    I'm trying this on winXP, apache and iphone emulator, trying to stream a live video feed.
    I just used the vlc command line you mentioned above for "Re-stream a live video feed:".

    On the emulator the video plays for 2 seconds and hangs there, and audio continues to play fine.

    Any thought on why the video hangs after 2 seconds?

    ReplyDelete
  49. Hi Keary,
    I am trying your module using "VLC media player 1.2.0-git-20110128-1731 Twoflower" nightly build on WinXP SP3, but I do not get .m3u8 file updated with created ts segments locations, here is the command line and output http://pastebin.com/EKgmaFT7

    created live.m3u8 has this content which never changes:
    #EXTM3U
    #EXT-X-TARGETDURATION:10
    #EXT-X-MEDIA-SEQUENCE:1
    #EXTINF:10,
    http://localhost/iphone/live-00000001.ts

    Any idea what is the problem?

    ReplyDelete
  50. Hi Soroush,

    On windows machines livehttp module doesnot update ".m3u8" file with "m3u8.tmp" file. The module is just writing the latest data to the "m3u8.tmp" file. You can write a powershell script which copies "m3u8.tmp" to ".m3u8" for every 5 seconds(which is smaller than your .ts file).

    Regards,
    Bhupathi

    ReplyDelete
  51. Any idea why the latest vlc version doesn't update the .m3u8 file under Windows? The original patch didn't have this problem.

    ReplyDelete
  52. Hi, I'm using the latest binary installer and having a terrible time trying to get livehttp working. Is this included in the official release now? If I do vlc -l I don't see anything called livehttp in there...

    ReplyDelete
  53. which version of VLC "doesn't need this" 1.1 or 1.2?

    ReplyDelete
  54. In case people Google this post:

    The problem experienced by Kendall TregoJun 17, 2010 09:16 AM is indeed a quoting problem. Just replace the single quotes ' by double quotes " and things will work.

    I just successfully set this up on Windows 2008 R2 with IIS and VLC 2.0.1. In addition to fixing the quotes, you need to add .m3u8 file extension as MIME type application/x-mpegURL into the IIS MIME map.

    ReplyDelete
  55. Please give me a solution, I am trying to run this VOD stream code using php exec function. It just create only one segment and playlist. When we run it, it has only a beep sound, Nothing will display. When i execute the code using mac terminal it works fine. Please help, i am hanging my head for few hours with this problem.

    ReplyDelete
  56. This post on the subject of how to embed a YouTube video code is really huge planned for new web access visitors. Stunning work, keep it up.

    Tablet repairs

    ReplyDelete

Note: Only a member of this blog may post a comment.