Creating a shared library that statically includes ffmpeg

Question

I'm having hard time trying to create a shared library that has ffmpeg libraries "baked in" as static ones.

Consider the following directory schema:

include/
  my own .h files
  ext/
    ffmpeg .h files
lib/
  libav*.a archive files (softlinks to the actual .a files)
  libValkka.so (my shared library)
test/
  mytest.cpp
bin/
  (binaries appear here)

I've come a long way (see Including objects to a shared library from a C++ archive (.a) ) and the library compiles ok with this: ([STUFF] has been omitted for brevity)

/usr/bin/c++ -fPIC -std=c++14 -pthread -Iinclude/ext -I/usr/include/libdrm -g -shared -Wl,-soname,libValkka.so -o lib/libValkka.so CMakeFiles/Valkka.dir/src/avthread.cpp.o CMakeFiles/Valkka.dir/src/opengl.cpp.o CMakeFiles/Valkka.dir/src/openglthread.cpp.o [STUFF] CMakeFiles/Valkka.dir/src/filters.cpp.o -lX11 -lGLEW -lGLU -lGL -Wl,--allow-multiple-definition -Wl,-Bsymbolic -Wl,--whole-archive -Wreorder lib/libavdevice.a lib/libavfilter.a lib/libavformat.a lib/libavcodec.a lib/libavutil.a lib/libswscale.a lib/libswresample.a -Wl,--no-whole-archive

However, when creating executables - their source code does not use any ffmpeg api (just my own api) - with:

c++ -std=c++14 -pthread -Iinclude -Iinclude/ext -Llib test/mytest.cpp -lValkka -g -o bin/mytest

I get a hoard of errors about missing ffmpeg dependencies. Not everything is missing, just some weird stuff:

lib/libValkka.so: undefined reference to `pa_stream_get_index'
lib/libValkka.so: undefined reference to `deflateInit_'
lib/libValkka.so: undefined reference to `pa_stream_get_state'
lib/libValkka.so: undefined reference to `lzma_stream_decoder'
lib/libValkka.so: undefined reference to `BZ2_bzDecompress'
lib/libValkka.so: undefined reference to `vaInitialize'
lib/libValkka.so: undefined reference to `pa_stream_unref'
lib/libValkka.so: undefined reference to `deflateInit2_'
lib/libValkka.so: undefined reference to `snd_pcm_close'
...
lib/libValkka.so: undefined reference to `vaGetDisplayDRM'
lib/libValkka.so: undefined reference to `vaMaxNumEntrypoints'
lib/libValkka.so: undefined reference to `uncompress'
lib/libValkka.so: undefined reference to `pa_stream_drop'
lib/libValkka.so: undefined reference to `pa_context_connect'
lib/libValkka.so: undefined reference to `FT_Get_Kerning'
lib/libValkka.so: undefined reference to `ass_free_track'
lib/libValkka.so: undefined reference to `pa_operation_unref'
lib/libValkka.so: undefined reference to `FT_Stroker_Done'
lib/libValkka.so: undefined reference to `vaTerminate'
lib/libValkka.so: undefined reference to `ass_new_track'
lib/libValkka.so: undefined reference to `jack_client_close'
...
lib/libValkka.so: undefined reference to `xcb_xfixes_query_version'
lib/libValkka.so: undefined reference to `xcb_shape_rectangles'
lib/libValkka.so: undefined reference to `pa_mainloop_free'
lib/libValkka.so: undefined reference to `snd_device_name_hint'
lib/libValkka.so: undefined reference to `vaCreateImage'
lib/libValkka.so: undefined reference to `vaBeginPicture'
lib/libValkka.so: undefined reference to `DtsSetColorSpace'
lib/libValkka.so: undefined reference to `vaDestroyConfig'
lib/libValkka.so: undefined reference to `pa_stream_writable_size'
lib/libValkka.so: undefined reference to `snd_pcm_hw_params_get_buffer_size_max'
lib/libValkka.so: undefined reference to `ass_read_file'

This is pretty frustrating, especially when I can see that those names are included in the shared library..!

nm lib/libValkka.so | grep "vaBeginPicture"

gives

U vaBeginPicture

etc. I thought it might be a problem regarding the dependency order the archive .a files, and also tried with:

..... -Wl,--allow-multiple-definition -Wl,-Bsymbolic -Wl,--start-group -Wl,--whole-archive -Wreorder lib/libavdevice.a lib/libavfilter.a lib/libavformat.a lib/libavcodec.a lib/libavutil.a lib/libswscale.a lib/libswresample.a -Wl,--no-whole-archive -Wl,--end-group

But the problem persists.

I have succesfully created a shared library that does not "bake in" those .a archives, i.e. that just depends dynamically on ffmpeg libraries, and there are no such problems.

I am baffled.. Have I misunderstood something fundamental, forgot some annoying linked option, or both? Help appreciated!

Answer 1

You need to link your shared library with 3rd party/system libraries required by ffmpeg: libbz2, libva, libxcb, libass, freetype2 etc. Actual list should be somewhere in ffmpeg distribution/build artifacts (automake's .pc files)

ignore-all is not a good idea; your application might run OK but these unresolved items are still there; it will crash as soon as it hits any of them. My guess is that most of them won't be hit, ever, since they are for libavdevice which you might not be even using, but still a bad idea. Also, check if you actually need that libavdevice library - you might trim a list of required libraries quite a bit if you get rid of that one.

Answer 2

Telling linker to ignore unresolved symbols when creating the executable does the trick:

c++ -std=c++14 -pthread -Iinclude -Iinclude/ext -Llib test/mytest.cpp -lValkka -g -o bin/mytest -Wl,--unresolved-symbols=ignore-all

The resulting executable also runs OK.

However.. using such a linker flag rubs me the wrong way. Maybe there is a better option? And why those symbols are not found in the first place?

EDIT

Following Andrey's suggestion, I stripped all external libraries from ffmpeg, with the aid of ffmpeg's configure script. That was a bit awkward process, so I created a nice python script that does it automagically. It might be a bit of an overkill, but here it goes:

#!/usr/bin/python3
"""
* Creates script "run_configure.bash" that launches ffmpeg's "configure" script with correct parameters (enabling/disabling stuff)
* Run in the same directory where you have ffmpeg's configure script
"""
import subprocess
import os
import re

def features(switch, adstring="", remove=[]):
  p=subprocess.Popen(["./configure",switch],stdout=subprocess.PIPE)
  st=p.stdout.read()
  fst=""
  for s in st.split():
    ss=s.decode("utf-8")
    ok=True
    for rem in remove:
      if (ss.find(rem)!=-1):
        ok=False
    if ok: fst+=adstring+ss+" "
  return fst


def disable_external():  
  p=subprocess.Popen(["./configure","-h"],stdout=subprocess.PIPE)
  st=p.stdout.read().decode("utf-8")

  # find some text tags from the configure output:
  # i1=st.find("External library support:")
  i1=st.find("themselves, not all their features will necessarily be usable by FFmpeg.")
  i2=st.find("Toolchain options:")
  st=st[i1:i2]
  """ # debugging ..
  print(st)
  stop
  """
  p=re.compile('--(enable|disable)-(\S*)')
  switches=[]
  for sw in p.findall(st):
    if (sw[1] not in switches):
      # print(sw[1]) # debugging
      switches.append(sw[1])
  fst=""
  for sw in switches:
    fst+="--disable-"+sw+" "
  return fst

st ="./configure "
st+="--disable-everything --disable-doc --disable-gpl --disable-pthreads --enable-static --enable-shared "
st+= disable_external()
st+= features("--list-decoders",adstring="--enable-decoder=", remove=["vdpau","crystalhd","zlib"])
st+= features("--list-muxers",  adstring="--enable-muxer=")
st+= features("--list-demuxers",adstring="--enable-demuxer=")
st+= features("--list-parsers", adstring="--enable-parser=")

f=open("run_configure.bash","w")
f.write("#!/bin/bash\n")
f.write(st+"\n")
f.close()
os.system("chmod a+x run_configure.bash")
print("\nNext run ./run_configure.bash\n")

"""
For cleaning up .a and .so files, use
find -name *.a -exec ls {} \;
find -name *.so* -exec ls {} \;
"""

Hopefully someone finds it useful. Run with python3.