From 2.3s to 22ms in 1 minute. Mysql index optimisation

In my little free time i developed for Tfm Group Software a small php application that read from a mysql table , made some calculation and presented a small table as result. In a cron job at every 5 minutes some data were inserted in the database. So far so good. The application was ok  and did it’s job . Another projects came and i forgot about it. However the data kept being inserted in the table. Several months later  i tried to load the result page . And for my surprise it loaded in 2.3 seconds. For a moment i was thinking that i did something wrong or the application was modified. Well it didn’t . In the database i had 375k rows but no indexes. The most used column was `insert_date` . So instantly i checked if i used index on it . Well i didn’t . So the quick fix came naturally:

create index a1 on tfm (insert_date);

And the page load time decreased magically from 2.3s to 22ms . This is a reminder to power of myql indexes in tfm linux or any other linux of your choice.

 

Adding Oracle support to PHP

If you want to connect to an Oracle database with PHP you will need the pecl module named oci8

First in order to compile it you will need Oracle Instant Client ( both basic & sdk ) . You can download them from here:

http://www.oracle.com/technology/software/tech/oci/instantclient/htdocs/linuxsoft.html

If you don’t have an Oracle account you will need to create one.

At the time of this post the following files are available: instantclient-basic-linux32-11.2.0.1.zip , instantclient-sdk-linux32-11.2.0.1.zip .

Create a directory in /opt  mkdir /opt/oracle/instantclient and copy those files there . Then unzip them . You will need to create a symbolic link ln -s libclntsh.so.11.1 libclntsh.so

Then you will need to install libaio if you don’t have it already . Don’t forget to add /opt/oracle/instantclient/instantclient_11_2 to /etc/ld.so.conf .

At this point you are ready to install the pecl extension to php.Create a temporary directory ( /tmp/1 ) and cd there.

mkdir /tmp/1

cd /tmp/1

pecl download oci8

tar xf oci8-1.3.5.tar

cd oci8-1.3.5

phpize

./configure –with-oci8=shared,instantclient,/opt/oracle/instantclient/instantclient_11_2

make

make install

edit /etc/php.ini and add

extension=oci8.so

And enjoy oracle extension for php. Note: if you have apache running restart it.


New CMS needed for tfm.ro

tfm.ro needs a new face. in the beginning we started with an inhouse mini CMS. Then we switched to mediawiki and focused on end user documentation and howto. At this point we use wordpress with Structure theme .

What is not ok:

  • Community options are limited
  • User generated content is limited to comments
  • Integration with the development site is done only via an RSS feed
  • Looks old and dusty
  • We focus on development and by doing this the site is most of informations are old and sometime obsolete

So ,  i’m begining to search for an alternative that will have to provide:

  • community options ( registration , posibility to upload pictures , patches , docs etc )
  • bug tracker integrated with the main site
  • online manuals creation / gathering
  • forums

Do you know what options are out there ?

Working with svn

When you manage a project with svn there are a lot of things to consider: how you create the repository, what external resources will be imported in the repository. Basically  it’s a constant job on how to organize things better to keep developers happy.

After lots of try/fail cycles i came to the conclusion that for linux distribution the best approach would be a tree like this

server
-> bzip2
* trunk
* tags
* branches
-> atk
* trunk
* tags
* branches
-> tfm-filesystem32
* trunk
* tags
* branches
server64
-> bzip2
* trunk
* tags
* branches
-> atk
* trunk
* tags
* branches
-> tfm-filesystem64
* trunk
* tags
* branches

What is wrong with this structure? In time a developer will update bzip2 from server 32 tree but will forget or won’t have the time to update server64 tree. So the trees will not stay in sync. And this is bad because it might happen that the projects will have different sources.

What can be done? Use svn:external declarations. First idea was to create a common tree and create externals from both server and server64. But this way we will have 3 trees and lot of work to do. And it will become hard to manage. Another approach was needed. We decided that server is the main tree. In the server64 tree we declared bzip2 to be external and pulled from 32 bit tree.

This way if a developer commits a change will go in both trees simultaneously.

How to use svn: external? Let’s take for example bzip2 from server64

svn del bzip2; svn commit ; svn pedit svn:externals . ; svn up ; svn commit

and when editing svn external you should add a like like this:

bzip2 https://svn.tfm.ro/tfm/server/bzip2