Loading mod_ilbc in FreeSwitch failed

Question

I've successfully installed other FS modules before, but this one I cannot fix. I'll produce as many details as possible and this question may not be a FreeSwitch-specific question. Any general troubleshooting suggestions would be appreciated.

Environment is like below, and I'm using freeswitch-1.6.19 source.

$ cat /etc/redhat-release 
CentOS Linux release 7.2.1511 (Core)

$ ls /usr/lib64/ilbc2/ -lh
total 64K
lrwxrwxrwx. 1 root root  16 Oct 24 20:18 libilbc.so -> libilbc.so.0.0.2
lrwxrwxrwx. 1 root root  16 Sep 18 18:03 libilbc.so.0 -> libilbc.so.0.0.2
-rwxr-xr-x. 1 root root 64K Jun 23  2015 libilbc.so.0.0.2

$ nm -D /usr/lib64/ilbc2/libilbc.so
--- omitted many lines ----
0000000000007590 T ilbc_decode
0000000000007790 T ilbc_decode_init
0000000000007c30 T ilbc_encode
0000000000008f80 T ilbc_encode_init
--- omitted many lines ----

I took these steps to compile mod_ilbc:

1. Uncomment the line codecs/mod_ilbc in modules.conf
2. ./configure --prefix=/opt/freeswitch/
3. make
4. make install
5. Modify /opt/freeswitch/etc/freeswitch/vars.xml to add iLBC@30i to global_codec_prefs.
6. Start freeswitch.

Here are the evidences that I see things are going smoothly:

1. make install successfully installs mod_ilbc. The make process successfully detects libilbc and uses libtool to link. Outputs of make install, note the -L/usr/lib64/ilbc2 -lilbc part.

making install mod_ilbc
make[4]: Entering directory `/home/cenzhe.zhu/freeswitch-1.6.19/src/mod/codecs/mod_ilbc'
make[5]: Entering directory `/home/cenzhe.zhu/freeswitch-1.6.19/src/mod/codecs/mod_ilbc'
make[5]: Nothing to be done for `install-exec-am'.
test -z "/opt/freeswitch/lib/freeswitch/mod" || /usr/bin/mkdir -p "/opt/freeswitch/lib/freeswitch/mod"
 /bin/sh /home/cenzhe.zhu/freeswitch-1.6.19/libtool   --mode=install /usr/bin/install -c   mod_ilbc.la '/opt/freeswitch/lib/freeswitch/mod'
libtool: install: (cd /home/cenzhe.zhu/freeswitch-1.6.19/src/mod/codecs/mod_ilbc; /bin/sh /home/cenzhe.zhu/freeswitch-1.6.19/libtool  --silent --tag CC --mode=relink gcc -I/usr/include/uuid -I/home/cenzhe.zhu/freeswitch-1.6.19/src/include -I/home/cenzhe.zhu/freeswitch-1.6.19/src/include -I/home/cenzhe.zhu/freeswitch-1.6.19/libs/libteletone/src -fPIC -Werror -Wno-unused-result -fvisibility=hidden -DSWITCH_API_VISIBILITY=1 -DHAVE_VISIBILITY=1 -g -ggdb -DHAVE_OPENSSL -Wall -std=c99 -pedantic -Wdeclaration-after-statement -I/usr/include/ilbc2 -g -O2 -avoid-version -module -no-undefined -shared -o mod_ilbc.la -rpath /opt/freeswitch/lib/freeswitch/mod mod_ilbc_la-mod_ilbc.lo /home/cenzhe.zhu/freeswitch-1.6.19/libfreeswitch.la -L/usr/lib64/ilbc2 -lilbc -lssl -lcrypto )
libtool: install: /usr/bin/install -c .libs/mod_ilbc.soT /opt/freeswitch/lib/freeswitch/mod/mod_ilbc.so
libtool: install: /usr/bin/install -c .libs/mod_ilbc.lai /opt/freeswitch/lib/freeswitch/mod/mod_ilbc.la
libtool: finish: PATH="/sbin:/bin:/usr/sbin:/usr/bin:/sbin" ldconfig -n /opt/freeswitch/lib/freeswitch/mod
----------------------------------------------------------------------
Libraries have been installed in:
   /opt/freeswitch/lib/freeswitch/mod

2. mod_ilbc is compiled properly. It expects symbols like ilbc_decode from libilbc. Analyzing mod_ilbc.so:

$ nm -D /opt/freeswitch/lib/freeswitch/mod/mod_ilbc.so
--- omitted many lines ----
0000000000001060 T _fini
                 w __gmon_start__
                 U ilbc_decode
                 U ilbc_decode_init
                 U ilbc_encode
                 U ilbc_encode_init
--- omitted many lines ----

3. Upon starting freeswitch, it tries to load the module using dlfcn but failed. Output of freeswitch.log:

2017-10-25 13:54:24.243208 [CRIT] switch_loadable_module.c:1528 Error Loading module /opt/freeswitch/lib/freeswitch/mod/mod_ilbc.so
**/opt/freeswitch/lib/freeswitch/mod/mod_ilbc.so: undefined symbol: ilbc_decode**

It appears to me that this is because freeswitch failed to link /usr/lib64/ilbc2/libilbc.so, but why? How to further troubleshoot the problem?

Answer 1

It seems formulating the problem and writing it up do help solving it. Problem solved now.

In the beginning, I don't know which libilbc I would need, so I installed both by sudo yum install ilbc-devel ilbc2-devel. Later when things don't work, I came across this issue so I removed version 1 of it by sudo yum remove ilbc-devel. BUT THAT DIDN'T REMOVE THE LIBRARY FILE!

There still lies /usr/lib64/libilbc.so.0.0.1 and /usr/lib64/libilbc.so.0, and freeswitch seeks libilbc from them. BANG!