Overlay Images on Video Using MPlayer and MEncoder

 in

In the ongoing saga of What doesn't work after the upgrade? I tried to run my logo-overlay script the other day to add the Linux Journal logo to a video and, as you may have guessed, it didn't work. In this case I'd been forewarned...

When I say I'd been forewarned, what I mean is that a reader/commentor on one of my posts had alerted me to the fact that the feature in ffmpeg that I was using (the "vhook" feature) was being deprecated and had already been removed from the development trunk. Of course I didn't do anything about it at the time, cuz hey, it's working for me. Well, now it's not!

The latest versions of ffmpeg have replaced the "vhook" mechanism with "avfilters". These are not the actual overlay feature, rather they are the way that "filters" were introduced into the processing chain. The new "avfilters" supposedly contain an "overlay" filter, but the avfilter capability does not appear to be compiled into the openSUSE version of ffmpeg. I downloaded the latest ffmpeg from development repository and configured it and built it with the avfilters enabled but that didn't produce the overlay filter either. Furthermore, I couldn't find anything in the downloaded source that appeared to be an overlay filter. I didn't feel like spending too much time trying to figure that out so I started looking for another solution.

I started looking at mencoder (part of the mplayer project) and noticed the -vf bmovl option, which is the bitmap overlay (bmovl) video filter (-vf). My first attempts with it didn't seem to be working but fortunately someone else had already figured it out and documented it here. I thought it would be useful to reproduce the guts of it here and add a bit of my own explanation as well as show a bit more of what can be done with this feature. The following is in essence the same as what you'll find at the link above, it overlays a bitmap onto a video and outputs a new video.

logo=logo.png
logo_width=600
logo_height=500
input_video=input.ogv
output_video=output.raw

rm -f $output_video
rm -f tfifo
rm -f tlogo.rgba
mkfifo tfifo

# Convert logo to RGBA.
convert $logo tlogo.rgba

# Copy logo to fifo.
(echo "RGBA32 $logo_width $logo_height 0 0 0 1" ; cat tlogo.rgba ) >tfifo &

# Convert input video with overlay from fifo.
mencoder -oac pcm -ovc raw -vf bmovl=0:0:tfifo -o $output_video $input_video

rm -f tfifo
rm -f tlogo.rgba

Mencoder does video encoding/converting much the same as ffmpeg does. One of its features is the ability to overlay data coming from a fifo on top of the video. In this case I'm taking the logo that I want to overlay and converting it to RGBA32 format (which is essentially just the uncompressed/unencode image data, 32 bits for each pixel). Then a mencoder fifo command and the RGBA32 data are fed into the fifo:

(echo "RGBA32 $logo_width $logo_height 0 0 0 1" ; cat tlogo.rgba ) >tfifo &

Mencoder runs in the foreground and converts the input video while simultaneously overlaying the data coming from the fifo.

mencoder -oac pcm -ovc raw -vf bmovl=0:0:tfifo -o $output_video $input_video

Note that for this to be of any value your logo (overlay bitmap) needs to have an alpha channel, i.e. it needs to be transparent everywhere except where the logo is.

The option of interest here is the -vf bmovl=0:0:tfifo option. This tells mencoder to apply a video filter (-vf) to the conversion/encoding, in this case the bitmap overlay filter (bmovl). The bmovl options are: hidden:opaque:fifo, where:

  • hidden specifies whether the overlaid bitmap should hidden or visible, here 0 means visible (not hidden).
  • opaque specifies whether the overlaid bitmap should allow the video to show through or should block it, here 0 means transparent (not opaque) so that the transparent part of the logo bitmap does not block the rest of the video.
  • fifo is just the name of the fifo file, here it is tfifo.

Also of interest is the fifo command that precedes the bitmap data in the fifo: RGBA32 $logo_width $logo_height 0 0 0 1. The RGBA32 specifies that the data is in RGBA32 format, its parameters are "width height xpos ypos alpha clear", where:

  • width / height specify the width and height of the bitmap data in pixels (not bytes). The bitmap file (the .rgba file) should be width * height * 4 bytes in size. Here the size of the logo overlay image is specified.
  • xpos / ypos specify the X and Y position in the video at which the bitmap data is to be overlaid. Here the position is simply 0,0.
  • alpha allows you to modify the alpha-ness of your bitmap (see below). Here 0 means don't change the alpha-ness of the bitmap.
  • clear specifies if the existing bitmap data in the fifo is cleared before applying the new bitmap data or if the new bitmap data is overwritten on top of the existing bitmap data. Here 1 means clear the data, this may be moot since we only have one bitmap image but just in case there's garbage by default we specify 1 to clear it.

The alpha option probably requires a bit more explanation. The alpha option takes a value between -255 and 255. Values from -1 down to -255 make your bitmap ever more transparent allowing more of the video to show through, at -255 your bitmap is completely transparent and only the video will be visible. Values from 1 up to 255 make your bitmap ever more opaque allowing less of the video to show through, at 255 your bitmap is completely opaque and none of the video will be visibile in the area where the bitmap is.

To back up a bit, the mencoder/mplayer fifo is not a single command fifo, it can accept multiple commands and bitmaps (if the command requires bitmap data). So by passing increasing or decreasing alpha values we can fade in and fade out bitmap overlays on a video. For example, we can fade in and then fade out a bitmap on top of a playing video with the following sequence of commands:

rm -f tfifo
rm -f overlay.rgba
mkfifo tfifo
convert overlay.png overlay.rgba

(
	alpha=-255
	fade=20
	while [[ $alpha -le 0 ]]
	do
		(echo "RGBA32 200 200 0 0 $alpha 1"; cat overlay.rgba) >tfifo
		sleep 0.1
		let t=alpha+fade
		if [[ $alpha -ne 0  &&  $t -gt 0 ]]; then t=0; fi
		alpha=$t
	done
	sleep 0.5
	while [[ $alpha -ge -255 ]]
	do
		(echo "RGBA32 200 200 0 0 $alpha 1"; cat overlay.rgba) >tfifo
		sleep 0.1
		let t=alpha-fade
		if [[ $alpha -ne 0  &&  $t -le -255 ]]; then t=-255; fi
		alpha=$t
	done
) &
pid=$!

mplayer -vf bmovl=0:0:tfifo video.ogv

rm -f tfifo
rm -f overlay.rgba
if kill -0 $pid; then kill -9 $pid; fi

You'll notice that this script uses mplayer and not mencoder. My initial thought was that I could do this with mencoder to overlay the sequence of bitmaps as the video was encoded by mencoder and save the video in a new file, but so far I haven't been able to make that happen. Mencoder just encodes and so it doesn't sequence things at the rate that data appears in the fifo: it just goes as fast as it can, which means most of the fifo commands are missed. There may be a way to make it work, but I haven't found it yet. See the attached video for a sample of run of the script.

Back to the original problem though for just a second, that of overlaying the LJ logo on a video. Using mencoder turned out to be a less than useful solution since in addition to my overlay script breaking, my video recorder solution (xvidcap) also is broken after the upgrade. It captures video but it's jerky on playback. So that required me to change recorder programs and start using recordmydesktop for recording video. It works reasonably well (it did not the last time I'd tried it) although xvidcap has a better UI.

However, recordmydesktop only captures .ogv video and mencoder won't output .ogv files. So I had to overlay the logo onto the captured .ogv file and output raw video. Then I had to use ffmpeg to convert the raw video (with the logo) back to .ogv (and to .mp4) so that I had both .ogv and .mp4 files. Unfortunately, ffmpeg screws up the audio/video sync in the converted .ogv file, so using that sequence of operations wasn't going to work very well.

My final solution will wait till next time. And my apologies for this twisted tail.

AttachmentSize
fade_sh.txt631 bytes
fade.ogv62.82 KB
______________________

Mitch Frazier is an Associate Editor for Linux Journal.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Rewriting it in PHP

XAOWEB's picture

Hi everyone,

I encoutered some problems with the fading in / fading out script part.
It appeared to deal with the background child process.
Almost all of the time, it did not terminate correctly (defunct process).
I needed watermarking in a PHP website project, so I tried to rewrite it in PHP and it's now fully working without any trouble (tested on a high trafic production server).
I thought it could help some people who are maybe experiencing the same problems, so here is the link to my solution: http://blog.xaoweb.fr/2010/09/16/php-mencoder-des-filigranes-watermark-d...

It's using the pcntl PHP extension and this class (for threading): http://blog.motane.lu/2009/01/02/multithreading-in-php/

Hope this helps.

Thanks Mitch Frazier for this excellent article (the only one about watermarking with mencoder I could found).

Using melt

ddennedy's picture

Here is how to do it with melt, a command line utility that comes with MLT, the editing framework used by Kdenlive and OpenShot.

$ melt $input_video -profile square_ntsc -filter watermark:$logo \
composite.geometry="0=0,0:200x200:0%;30=0,0:200x200:100%;60=0,0:200x200:100%;90=0,0:200x200:0%" \
-consumer avformat:$output_video b=2000k acodec=libvorbis

and $output_video will be Theora by default if the extension is .ogv. So much easier.
-profile is important and sets the project resolution and framerate - similar to page size in a word processor. Look in /usr/share/mlt/profiles for one to use or make your own.

composite.geometry syntax is frameNum=X,Y:WxH:alpha[;]

It will automatically scale down whatever $logo is (including another video) to fit within WxH while maintaining the logo's original aspect ratio. X, Y, W, H can be in pixel values or have '%' suffix (% of project resolution).

-consumer avformat:... is the way to melt to encode. Leave it off to preview your command in a window.

Something that might help for

salsaman's picture

Something that might help for the last part. You can get mencoder to output in yuv4mpeg format, using -vo yuv4mpeg [it will write to a fifo called stream.yuv by default]. This output can then be read by encoder_example (theora_encoder_example in ubuntu) along with a wav file to produce an ogg/theora/vorbis file.

s/mencoder/mplayer above.

salsaman's picture

s/mencoder/mplayer above.

Now it makes sense

Mitch Frazier's picture

I'd been pondering that but it wasn't making any sense before...

Mitch Frazier is an Associate Editor for Linux Journal.

Seems like a lot of work

salsaman's picture

Blimey Mitch, what a lot of work for a simple task ! Why didn't you just use LiVES (http://lives.sourceforge.net) or another of the ever improving video editors designed specifically for tasks like this ?

LiVES even has a specialised plugin just for this purpose (Insert Image plugin @ http://lives.sourceforge.net/index.php?do=addons).

Automation

Mitch Frazier's picture

The main reason is that I want to automate the task. I don't want to have to start a program, load a video, load an image, etc. and go through 20 steps every time I want to do this. I just want it to happen as part of the script that I use to upload the video.

I actually tried one video editor, kdenlive, and couldn't figure it out how to do it. I suspect kdenlive can do it but I couldn't seem to figure it out.

Beyond that, this capability is interesting in and of itself, but you're right it was way too much work.

Mitch Frazier is an Associate Editor for Linux Journal.

Commandline solutions are

salsaman's picture

Commandline solutions are very cool, and your method is very interesting. Yes, for automating such stuff this is a good way to do it.

GUI Version

Mitch Frazier's picture

Since you *are* the LiVES man, maybe you'd like to whip together a little article, or better yet, a little video of doing that with LiVES?

Mitch Frazier is an Associate Editor for Linux Journal.

OK, I will see what I can do

salsaman's picture

OK, I will see what I can do !

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState