Setting up a development Sphinx search engine in Mac OS

Sphinx is one of the high performance, free open-source full-text search engine with rich feature list. Recently I was experimenting, on this tool for developing a search platform, for big and complex databases. It is very interesting, and easy to setup a developing environment in Mac OS. Sometimes the compiling using the method specified in the documents of sphinx will not work that easily. I found it is super easy, installation using mac ports. Here is how I done it.

Installing Sphinx using Mac Ports

1. If you don’t have mac-ports installed, Install mac-ports downloading from their site.
2. If you not installed X code tools from Mac OS X, DVD install it – need this for compiling sources in a UNIX development way.
3. Now install sphinx using following command. [sourcecode language=”plain”]sudo port install sphinx[/sourcecode] This will compile and install everything required for making the sphinx work.

Now sphinx is installed and we have ‘indexer’, ‘search’ utilities and a ‘searchd’ daemon.

Basic way to use Sphinx

Sphinx comes with a sample sql file to setup a sql-query based search (Simplest configuration). This SQL and a sample Sphinx sample configuration is located at [sourcecode language=”plain”]/opt/local/etc/sphinx[/sourcecode]. Try following steps to make it running with your configuration
1. Create a directory some where in your system. In my case I created at desktop named ‘sphinx’. [sourcecode language=”plain”] cd ~/Desktop/sphinx [/sourcecode]
2. Create ‘test’ database in your mysql server and import the Sample SQL.
2. copy sphinx.conf from sample /opt/local/etc/sphinx to this folder.[sourcecode language=”plain”]cp /opt/local/etc/sphinx/sphinx.conf.dist sphinx.conf[/sourcecode]
3. Now edit sphinx.conf and change the database Params and paths. In my case it looks like following. Edit the queries related to your custom databases to make it yours.
[sourcecode language=”plain”]
source src1
type = mysql

sql_host = localhost
sql_user = test
sql_pass =
sql_db = test
sql_port = 3306 # optional, default is 3306

sql_query = \
SELECT id, group_id, UNIX_TIMESTAMP(date_added) AS date_added, title, content \
FROM documents

sql_attr_uint = group_id
sql_attr_timestamp = date_added

sql_query_info = SELECT * FROM documents WHERE id=$id

index test1
source = src1
path = /Users/lijeeshms/Desktop/sphinx/data/test1
docinfo = extern
charset_type = sbcs

mem_limit = 32M

port = 3312
log = /Users/lijeeshms/Desktop/sphinx/log/searchd.log
query_log = /Users/lijeeshms/Desktop/sphinx/log/query.log
read_timeout = 5
max_children = 30
pid_file = /Users/lijeeshms/Desktop/sphinx/log/
max_matches = 1000
seamless_rotate = 1
preopen_indexes = 0
unlink_old = 1
4. Create 2 folders, ‘data’ and ‘log’ inside the ‘sphinx’ directory. [sourcecode language=”plain”]mkdir data log[/sourcecode]
5. Now run the indexer with the configuration we created from the test directory. [sourcecode language=”plain”]indexer –config /<path-to>/sphinx.conf –all[/sourcecode] This will create the indexes based the confutation.
6. Now run the Sphinx search daemon using the same command. [sourcecode language=”plain”]searchd –config /<path-to>/sphinx.conf [/sourcecode] If everything is fine, sphinx search engine will start with the above index configurations. If any error happens you need to check the configuration for any errors.
7. You can now test a search with ‘search’ command from terminal. [sourcecode language=”plain”]search <query> [/sourcecode] The results will be shown.

Once all the above is working, it will be easy to use any of the client libraries for PHP (or PECL), Python, Ruby or Java to search from a web or desktop based environments. Sphinx provides options to run multiple indexes and search on selected indexes, thus allowing one daemon to use with multiple type of searches.
Sphinx can index data from an XMLPipe, this is best for creating configurable search indexes from very dynamic datasources like a CMS or CRM.

Easy open source software for the Macs

Mac ports is the way to use most of the opens source software from the community on Macs. Recently I found out new tool named Porticus, that makes the life so easy.

Porticus for Macports

Porticus for Macports

Setting up Porticus for the first time – snow leapard

1) Install macports, downloading the dmg – checkout Macports website for updates

2) Install Porticus downloading fromĀ  Porticus website

That is it. It is realy usefull tool to try free software on mac. Feels like one of the software package management GUIs Linux users enjoy

Two must watch Linux documentary films

Recent days I was encountered with some fundamental questions and criticism on concepts of open source, and energy behind Linux operating system. The answers are already explained by the experts in many ways through different mediums. I like to share these two good documentaries made about Linux on this occasion


To see more on this video see

The Code

More information is on

You can see a lot more videos of linux by doing a Google search – How Linux powers some of the hollywood films studios, high end mission critical servers and supper computers, almost any type of microprocessor based electronic devices etc. Linux is nothing but the open knowledge of a “Computer system”, and how it works.

Updates from UAE Linux user meetup

UAE Linux user meetup (@uaelug on twitter), was very interesting yesterday. About 20 Linux Enthusiasts were present on the meetup. It was an open discussion session after an Ubuntu linux installation workshop for newbies. Some interesting talks with new linux users and experts gave lot of amazing information about linux.

Meetup started with number of new guys introducing themselves. It was fun and some “jokes are thrown on the table” . @tarekkassar was live tweeting from the meetup. ” If you love open-source, join #uaelug, all people here will be your friends ” – he said on twitter.

After the meetup it was very clear to everyone that why we use linux and what is the importance of this Operating system. Some topics we discussed was about how you use a computer in its maximum possible ways, that cannot be achieved using propitiatory systems. It explained how to make computer to do more stuff than, just following some ready made “products”. Experts explained how we take challenges and do understand how software really works.

Here is some photos from the meetup taken by @smshuja

Guys at UAE Linux meetup

Guys at UAE Linux meetup

UAE Linux meetup

UAE Linux meetup

Guys at linux UAE linux meetup

Guys at linux UAE linux meetup

Hacking Wireless at UAE linux meetup

Hacking Wireless at UAE linux meetup

Tarek kassar live internet update from uaelug meetup

Tarek kassar live internet update from uaelug meetup

I am in uaelug

I am in uaelug

You can see more photos of uaelug at meetup group.

First edition of UAE Linux user meetup experiance

United Arab Emirates Linux Meetup Group

More than 20 Linux minded people met on venue at Flip Media meeting rooms, yesterday evening. The Meetup went for about 2 hours. It was an amazing experience that people with various level of experience in Linux and computer systems in common, talked about themselves and the importance of Linux everywhere.

There was people with experience of more than 10 years, hackers, desktop users, programmers and people never used Linux in the group. We discussed mainly on how to proceed to make this group work for everyone And shared some of interesting experiances.Here is some pictures.

UAE linux meetupUAE linux meetupUAE linux meetup

See more pictures at the meetup group here . I recommend all Linux users in this region to join on this group

You can visit and register at to know more. Follow @uaelug on twitter for updates.