SphinxSearch - indexer config option problem

Question

I using sphinx search to create indexes and search data in my PostgreSQL database.

I have 2 questions about it.

If I run command

/usr/bin/indexer --config /etc/sphinxsearch/sphinx.conf --rotate --all

I get output from 'show tables;'

Index	Type
dist_title_de	distributed
word_title_de	local
word_titlestemmed_de	local
rt_title_de	rt

But If I run command

/usr/bin/indexer --config /etc/sphinxsearch/sphinx_another_conf_file.conf --rotate --all

Then I get the same output on terminal, but I dont see new indexes on 'show tables;'. It seems like '--config' option in indexer not working and only properly name is sphinx.conf. It's problematic, because if I want reindex sphinxsearch I have to changing file sphinx.conf.

Second question is it possible to 'add' new index without delete old ones? Currently I using sphinx like (everyday):

Get new data (datasource1, datasource2, ..., datasource8)
Index --rotate --all (index data from 8 datasources)
Search some info on indexes
Write it to db

But now, I want sth like:

Get new data from datasource1
Index datasource1
Get new data from datasource2
Index datasource2 (without delete index datasource1)
Search something in index datasource1
....
Get new data form datasource8 (without deleteing indexes)
Index datasource8 etc

On 'without delete index' I mean, now if I use command from top of topic, I 'lost' my indexes and get only new ones (from sphinx.conf).

My sphinx.conf (only 1 datasource):

source src_title_de
{
    type            = pgsql
    sql_host        = #######
    sql_user        = #######
    sql_pass        = #######
    sql_db          = #######
    sql_port        = 3306  # optional, default is 3306
    sql_query       = \
        SELECT id, group_id, (date_extraction::TIMESTAMP) AS date_extraction, title \
        FROM sphinx_test
    sql_ranged_throttle = 0
}

index word_title_de
{
source          = src_title_de
path            = /var/lib/sphinxsearch/data/word_title_de
docinfo         = extern
dict            = keywords
mlock           = 0
morphology      = none
stopwords       = /var/lib/sphinxsearch/data/stopwords.txt
wordforms       = /var/lib/sphinxsearch/data/wordforms_de.txt
min_word_len        = 1
}

index word_titlestemmed_de : word_title_de
{
    path            = /var/lib/sphinxsearch/data/word_titlestemmed_de
    morphology      = stem_en
}

index dist_title_de
{
type            = distributed
local           = word_title_de
local           = word_titlestemmed_de
agent           = localhost:9313:remote1
agent           = localhost:9314:remote2,remote3
agent_connect_timeout   = 1000
agent_query_timeout     = 3000
}

index rt_title_de
{
type            = rt
path            = /var/lib/sphinxsearch/data/rt_title_de
rt_field        = title
rt_field        = content
rt_attr_uint        = gid
}


indexer
{
mem_limit       = 128M
}

searchd
{
listen          = 9312:sphinx
listen          = 9306:mysql41
log         = /var/log/sphinxsearch/searchd.log
query_log       = /var/log/sphinxsearch/query.log
read_timeout        = 5
client_timeout      = 300
max_children        = 30
persistent_connections_limit    = 30
pid_file        = /var/run/sphinxsearch/searchd.pid
seamless_rotate     = 1
preopen_indexes     = 1
unlink_old      = 1
mva_updates_pool    = 1M
max_packet_size     = 8M
max_filters     = 256
max_filter_values   = 4096
max_batch_queries   = 32
workers         = threads # for RT to work
}

My second file for 8 datasources like the same like above with CTRL+C CTRL+V on 'source src_title_de', 'index word_title_de', 'index word_titlestemmed_de', 'index rt_title_de' with another countries and change table with data in 'sql_query'.

barryhunter barryhunter · Accepted Answer · 2020-12-22T11:58:11

On your first question, the --config option only applies to that indexer run. Ie the --all should cause it index (or try to ) index all the plain indexes mentioned in that file.

... but when it sends the signal to reload (what the --rotate) does, searchd just reloads its CURRENT config file, NOT the one you told indexer about.

To get serachd to use a new config file would have to stop searchd, and start it again with new config file.

So change sphinx.conf directly, rather than a 'second' file.

Acully the second question is the same answer...

So change sphinx.conf directly, rather than a 'second' file.

Ie add your new index to sphinx.conf, use indexer to 'build' it. When indexer has finished, it will tell 'reload' whcih will cause searchd to load the new config file AND the new index just built.

SphinxSearch - indexer config option problem

1 Answers