Customizing Redis pubsub for message persistence

Redis Logo

Redis comes packed with a simple yet powerful PubSub API.  It provides low latency and scales well.  A message published on a channel is received by subscriber(s) at the other end.  However, if no active subscriber is found the message is simply lost.  This drawback puts Redis out of the probables list for several use cases where message persistence of unprocessed published messages is desired.  It’s also probably a reason why several open source projects that support Redis as a broker are based upon it’s list push / pop API.  In this post I will demonstrate how to modify Redis PubSub API to support message persistence, opening possibilities for several interesting use cases.

Last Published Message

Ability to fetch the last published message on a particular channel without subscribing to the channel opens doors for several interesting use cases.  src/pubsub.c:publishCommand is where Redis handles publish command.  Let’s add a line of code to persist the most recently published message on a channel:

void publishCommand(redisClient *c) {
    ....

    /* Persist last published message in channel specific key */
    setKey(c->db, c->argv[1], c->argv[2]);

    addReplyLongLong(....);
}

Above, we added a call to src/db.c:setKey function that sets the value of key c->argv[1] (channel name) to c->argv[2] (published message).

Run make from the project root directory and start ./src/redis-server. Now we can do something like:

127.0.0.1:6379> publish channel1 c1m1
(integer) 0
127.0.0.1:6379> get channel1
"c1m1"
127.0.0.1:6379> publish channel1 c1m2
(integer) 0
127.0.0.1:6379> get channel1
"c1m2"

Voila. We published a message with no subscriber. However, an incoming user can still be served with the last published message on the channel by fetching the value of key channel1 without explicitly subscribing to the channel.

Let’s take this idea one step ahead. XMPP Publish-Subscribe (XEP-0060) defines a specification for receiving the last published item. It says,

When a subscription request is successfully processed, the service MAY send the last published item to the new subscriber.

Let’s add this idea to Redis PubSub mechanism. src/pubsub.c:subscribeCommand function is where Redis processes channel subscription requests. Add the following lines of code at the end of this function.

void subscribeCommand(redisClient *c) {
    ....

    /* Send last received message on the subscribed channel(s) */
    robj *o;
    for (j = 1; j < c->argc; j++) {
    	o = lookupKeyRead(c->db, c->argv[j]);
    	if(o != NULL) {
		addReply(c,shared.mbulkhdr[3]);
		addReply(c,shared.messagebulk);
		addReplyBulk(c,c->argv[j]);
		addReplyBulk(c,o);
    	}
    }
}

Here, post subscription, we fetch and send the last published message for all channels that the client just subscribed to. make and restart ./src/redis-server. Now on a new ./src/redis-cli terminal subscribe to channel1:

127.0.0.1:6379> subscribe channel1
Reading messages... (press Ctrl-C to quit)
1) "subscribe"
2) "channel1"
3) (integer) 1
1) "message"
2) "channel1"
3) "c1m2"

Voila! Now Redis server will send the last published message upon subscription. But what about PSUBSCRIBE use case?

src/pubsub.c:psubscribeCommand handles pattern based channel subscription logic. Add following lines of code at the end of this function:

void psubscribeCommand(redisClient *c) {
    ....

    /* Send last received message on the channel(s) matching subscribed patterns */
    for (j = 1; j < c->argc; j++) {
    	robj *pat = c->argv[j];
    	dictIterator *di = dictGetIterator(server.pubsub_channels);
    	dictEntry *de;
    	while((de = dictNext(di)) != NULL) {
		robj *cobj = dictGetKey(de);
		sds channel = cobj->ptr;
		if (stringmatchlen((char*)pat->ptr,
				sdslen(pat->ptr),
				(char*)channel,
				sdslen(channel), 0)) {
			robj *o = lookupKeyRead(c->db, cobj);
			if(o != NULL) {
	                addReply(c,shared.mbulkhdr[4]);
	                addReply(c,shared.pmessagebulk);
	                addReplyBulk(c,pat);
	                addReplyBulk(c,cobj);
	                addReplyBulk(c,o);
			}
		}
	}
	dictReleaseIterator(di);
    }
}

Above, for every subscribed pattern, we iterate over active server.pubsub_channels and check if the active channel matches the subscription pattern. On match we fetch and send the last published message on the channel to the client.

With previous redis-cli subscribe terminal running, open a new terminal and try:

127.0.0.1:6379> psubscribe channel*
Reading messages... (press Ctrl-C to quit)
1) "psubscribe"
2) "channel*"
3) (integer) 1
1) "pmessage"
2) "channel*"
3) "channel1"
4) "c1m2"

Next

You can checkout my Redis fork and commits under pubsub-persistence branch. Enhancements described above can also be found on this github commit.

Currently it is unclear what Antirez (Sanfilippo Salvatore) plans to do further with PubSub in Redis. It stands on a solid base and recent efforts are rightly put behind Redis cluster. However, I see some interesting enhancements that can be made to Redis PubSub mainline. In the next post I will take the current idea one step ahead and add persistence support for all or only unprocessed published messages in a Redis list (possibly with a cap or expiration on persisted messages).

Announcing Jaxl v3.x – asynchronous, non-blocking I/O, event based PHP client/server library

Jaxl v3.x is a successor of v2.x (and is NOT backward compatible), carrying a lot of code from v2.x while throwing away the ugly parts. A lot of components have been re-written keeping in mind the feedback from the developer community over the last 4 years. Also Jaxl shares a few philosophies from my experience with erlang and python languages.

Jaxl is an asynchronous, non-blocking I/O, event based PHP library for writing custom TCP/IP client and server implementations. From it’s previous versions, library inherits a full blown stable support for XMPP protocol stack. In v3.0, support for HTTP protocol stack was also added.

At the heart of every protocol stack sits a Core stack. It contains all the building blocks for everything that we aim to do with Jaxl library. Both XMPP and HTTP protocol stacks are written on top of the Core stack. Infact the source code of protocol implementations knows nothing about the standard (inbuilt) PHP socket and stream methods.

Source code on GitHub

Examples

Documentation

Group and Mailing List

Create a bug/issue

Read why v3.x was written and what traffic it has served in the past.

Releasing Jaxl 2.0 – Object oriented XMPP framework in PHP

After months of restructuring the Jaxl library, I am pleased to announce Jaxl 2.0, an object oriented XMPP framework in PHP for developing real time applications for browsers, desktops and hand held devices.

What’s new in Jaxl 2.0?

  • A lot of structural changes has been done from the previous version to make it more scalable, robust, flexible and easy to use
  • Library now provides an event mechanism, allowing developers to register callbacks for various xmpp events in their application code
  • Use integrated BOSH support to write real time web applications in minutes
  • More than 10 new implemented XMPP extensions (XEP’s) added
  • Development hosting moves to github, stable releases available at google code

Documentation for Jaxl users
Below is a list of getting started documentation for XMPP app developers:

Implemented XEP’s
A lot of new XEP’s has been implemented and packaged with Jaxl 2.0. Developers can use Jaxl event mechanism to implement new XEP’s without knowing the working of other core parts of the library.

Below is a list of released implemented XEP with Jaxl 2.0:

Documentation for project contributors
For developers interested in contributing to the Jaxl project, here is a list of insight documentation to get you started:

  • Jaxl core workflow and architecture (coming soon)
  • How to implement new XMPP extensions using Jaxl (coming soon)

Useful Links
For live help and discussion join jaxl@conference.psi-im.org chat room

WordPress Toolbar v 2.2 : Custom toolbar url, Support for WPMU and bug fixes

download-wordpress-plugin

WordPress toolbar plugin provide a facebook, digg style toolbar for all outgoing links from your blog posts. The toolbar url defaults to http://yourblog/wp-content/plugins/wordpress-toolbar/toolbar.php. However with version 2.2, blog admin can customize toolbar url to http://yourblog/wordpress-toolbar/ through the admin panel. A lot of other enhancements have been added like cross-plugin compatibility and support for WPMU hosted blogs. Check full feature list below.

What’s New?
Listed below is list of new features and bug fixes released with v 2.2:

  1. Support for customizing toolbar url through admin panel
  2. Support for WPMU hosted blogs
  3. Support for removing “Get this Plugin” widget from the toolbar through admin panel
  4. Security fix for possible XSS attack. Fix done by passing encoded hash string instead of plain text parameters. Also added various security checks on toolbar page to avoid possible XSS attacks.
  5. Bug fix where plugin didn’t work as expected because of cross plugin compatibility issues. Fix done by replacing server side toolbar logic with client side (using jquery) logic.
  6. Bug fix to show sociable share icons and tinyurl share link only for single posts and pages
  7. Bug fix for unrecognizable code in the toolbar when the encoding of hosted blog is different from utf-8. Fix done by using hosted blog settings instead of hardcoded utf-8.

Also core plugin code has been restructured (OOPS oriented now) so that maintainability and support becomes easier and quicker.

Steps to customize the default toolbar URL
Enable WordPress Toolbar v 2.2 plugin. Assuming you want to change default toolbar url from /wp-content/plugins/wordpress-toolbar/toolbar.php to /wordpress-toolbar, follow these steps:

  1. Enable apache mod_rewrite
  2. Add AllowOverride All in your blog virtual host config file and restart apache
  3. Add following apache rewrite rule by editing your blog .htaccess file
    RewriteRule ^wordpress-toolbar$ wp-content/plugins/wordpress-toolbar/toolbar.php
    RewriteRule ^wordpress-toolbar/$ wp-content/plugins/wordpress-toolbar/toolbar.php
  4. If you have blogs hosted using WPMU, add following apache rewrite rules in .htaccess file
    RewriteRule ^wordpress-toolbar$ wp-content/plugins/wordpress-toolbar/toolbar.php
    RewriteRule ^wordpress-toolbar/$ wp-content/plugins/wordpress-toolbar/toolbar.php
    RewriteRule ^([0-9a-zA-Z-]+)/wordpress-toolbar$ $1/wp-content/plugins/wordpress-toolbar/toolbar.php
    RewriteRule ^([0-9a-zA-Z-]+)/wordpress-toolbar/$ $1/wp-content/plugins/wordpress-toolbar/toolbar.php
  5. Manually check if rewrite rules are working. Open your custom toolbar url and you should see a result similar to this http://abhinavsingh.com/blog/wordpress-toolbar
  6. If for some reasons you DO NOT see “Working! Though required parameters are missing.” on toolbar page, it means rewrite rules didn’t worked as expected. Before you proceed with the setup, you SHOULD fix rewrite rules
  7. Go to wordpress admin and click "Wordpress Toolbar" under Settings tab
  8. Update your new custom toolbar url as shown: wordpress-toolbar-v-2.2-custom-toolbar-url-demo
  9. Clear cache and verify your toolbar

Enjoy and kindly let me know if you have issues installing plugin on your host.

5 exciting (gaming) bots you can create using Jaxl (Jabber XMPP Library) in PHP

Jaxl is an open source XMPP client library written in PHP. The object oriented structure of JAXL allow developers to build various extensions using Jaxl library as their base. If used intelligently, JAXL client library is capable of doing more than just chat message transfers. Here are a few applications where developers have tried using JAXL for delivering more than just chat messages:

“I used your library to develop a prototype that connects dynamically some users to a XMPP server if an external event is detected. The script runs like a daemon. Because of your object-oriented class design it was very easy to set up dynamic number of parallel XMPP sessions. I would like to use your library as part of a software that integrates telephony and XMPP functionality. The software will also be licenced unter GPL. Thanks again for your great work.”

“I’m thinking on creating a symfony plugin for jaxl library. I’ve worked before with the Jabber php library. This one of yours is much nicer! And it is working really nice, good job! “

The possibilities are endless. In this blog post I will discuss various possible use cases of JAXL client library starting from, creating an 24×7 online chat bot, broadcasting messages to gtalk friend list, rss feed aggregator, custom out of office email bot for gmail and google apps user, and finally a simple game using Jaxl client library. (similar to anagram gaming bot games@gtalkbots.com)

Setting up the environment
Jaxl is hosted on Google Code. Checkout the latest version of JAXL client library:

svn checkout http://jaxl.googlecode.com/svn/trunk/ jaxl-read-only

Alternately you can download the latest version of Jaxl from here:
http://jaxl.googlecode.com/files/jaxl-1.0.4.rar

From here on I will assume you have all the library files in a folder called jaxl. You should see the following set of php files inside the jaxl folder:

  • config.ini.php : Holds your jabber account and mysql connection information
  • mysql.class.php : Basic MySQL connection class used to insert received messages and presence into MySQL database
  • logger.class.php : A very basic logger class which allows you to log all XML stanza’s send and received from jabber server
  • xmpp.class.php : Base XMPP class library which implements the XMPP protocol
  • jaxl.class.php : JAXL class which extends XMPP class library. Should be the starting point for your application
  • index.php : After building your application in jaxl.class.php, you finally initialize and call your methods here

You will also see a bunch of other php files: jaxl4broadcast.class.php, jaxl4gmail.class.php, jaxl4dzone.class.php, which are extensions written using Jaxl library as their base. We will discuss them all as we proceed on the blog.

Another point I would like to discuss before we go ahead, is the structure of Jaxl client library. XMPP class is written in xmpp.class.php php class file which implements the XMPP protocol. It takes care of user authentication, user presence (available, busy, idle), user status, sending and receiving messages and everything which we will see as we proceed on the blog. JAXL class extends XMPP class in jaxl.class.php php file. We will develop all our bots/applications in jaxl.class.php php file and will never require to touch the base xmpp.class.php file. Finally, index.php is the file which invokes our application written in jaxl.class.php. As a convention, we rename jaxl.class.php to jaxl4app.class.php where app is the name of our application.

XMPP class defined in xmpp.class.php passes program handle to various methods whenever an event occur. Following 4 methods are of our use, while developing an application inside jaxl4app.class.php file:

  • eventMessage($fromJid, $content, $offline = FALSE) : This is the method where XMPP class passes the handle, when it receives a message. The message can be either online or offline, which is indicated by the $offline parameter being passed to this method. Other two parameters received are $fromJid which is the jabber id of the user sending the message. e.g. jabberxmpplibrary@gmail.com, and $content which is the actual message sent by the user identified by $fromJid
  • eventPresence($fromJid, $status, $photo) : This is the method where XMPP class passes the handle, when it receives a presence i.e. notification about status change of a user. A change in status event is triggered by either user changing his status text or by user changing his online presence i.e. available, idle, busy. $fromJid is the parameter passed to this method which is the jabber id of the user who changed his status. $status is the new status set by the user.
  • eventNewEMail($total, $thread, $url, $participation, $messages, $date, $senders, $labels, $subject, $snippet) : XMPP library also implements the Gmail Notification extension for XMPP protocol, and passes the handle to eventNewEMail() method when ever a new email is received on Gmail or Google Apps mail. One of the real life example of this protocol called Gmail Notification can be seen using Gtalk. Gtalk will pop up a window when ever you receive a new email on Gmail. For more detail about various parameters passed to this method refer the Gmail Notification documentation.
  • setStatus() : XMPP class passes the handle to this method before setting the status of the bot/application we intent to run using Jaxl library. Customize this method for setting custom status messages on logon or anytime during the execution of the application.

In 99.99% of the applications which we will intent to build, will not require handle to other events which happen in the background and handled by the base XMPP class. For more information about all the events refer this blog post. Behind the scenes – How and What XML’s are exchanged by JAXL

Jaxl also provide provisions to switch your bot between production and development environment by a simple change in the config.ini.php file. $env parameter (allowed values are “prod” or “devel”) decides what environment do you want to run this bot on. $env is set to “devel”, when you are developing your bot/application and don’t want to connect to a production jabber server repeatedly during development. Setting $env to “prod”, will configure the bot/application to connect to the production jabber server.

1. My first bot: Creating an 24×7 online status aggregation bot
This was how I started working on JAXL. I wanted to collect status messages of all my gtalk friends. I also wanted to plot on a graph, when and which of my gtalk friends come online or go offline. A basic example of this graph can be seen on my timeline at Gtalkbots.

The good thing in Jaxl library is that, by default it comes with a built-in bot capable of doing the above tasks for us. (because this is how i started working on Jaxl, hence is the default behaviour). Here is how you can configure Jaxl for the same:

  • Choosing an environment: Open config.ini.php and choose an environment. Since we want to collect information about all our friends on gtalk, we will set $env="prod". This will allow our bot to connect to the jabber servers hosted at talk.google.com (see config file). Also set $logDB = TRUE;, which will enable logging of user information to the MySQL database.
  • Updating user credentials: Register a username at Gmail (if you don’t have one already) and update the username and password of this user in the config file. Our bot will use these credentials to authenticate with the google talk servers. Add a few friends using gtalk to start with. Also update your MySQL database hostname, username and password. Leave the database name as jaxl.
  • Creating jaxl database: Run the database.sql file against your MySQL database. It will create a database called jaxl with two tables called message and presence. Our application will use these tables for storing information about our gtalk friends.
  • Creating the bot: jaxl.class.php by default is ready to work as we want it to. It will log all your gtalk friend’s information in the MySQL database if $logDB is set to TRUE inside the config.ini.php file. Also by default it replies back a welcome message for every message received (online or offline). You may want to un-comment that section of the code as of now.
  • Running the bot: Open command line (windows) or the terminal window (unix, mac) and migrate to the jaxl folder. (Remember you cannot run your application with something like http://localhost/jaxl in your browser. XMPP is a TCP-IP level protocol and not made for running directly using HTTP protocol i.e. browsers. However, we can use BOSH extension of XMPP protocol to make it run over HTTP. Jaxl currently doesn’t support BOSH extension. It’s currently under testing). Now simply run the following command.
    sudo php index.php

    on the terminal. You should see something like this on your terminal: jaxl-jabber-xmpp-library-demo-1

    To debug more while development, you should enabled logging in the config file. Jaxl library will start logging every xmpp stanza sent or recieved to the google talk servers. This is also a good way of learning more about the internals of xmpp protocol.

  • Running the bot 24×7: One of the most common query i get (specially from college enthusiasts) is how to run their applications 24×7. Just like the anagram gaming bot at gtalkbots (games@gtalkbots.com). Run the following command on your terminal to run your application as a background process (only possible on unix or osx, not on windows):
    sudo nohup php index.php > log/logger.log &

    . This will start the bot as a background process and hence the bot will not stop its execution even if you close the terminal window. When you want to kill your application, simply search for the process id of your application using:

    ps aux | grep index.php

    . Note the process id corresponding to your application and issue

    kill -9 process_id

    to kill the application.
    jaxl-jabber-xmpp-library-demo-3

2. Broadcasting messages to your gtalk friend list
You might want to broadcast a message to your gtalk buddy list for a number of reasons. Extension jaxl4broadcast.class.php will do exactly the same for us. To run this extension you should checkout the latest xmpp.class.php file from the repository. Jaxl v 1.0.4 doesn’t support Gmail Extension. Leave your configuration file as it is form the previous example. Simply include jaxl4broadcast.class.php in your index.php file, instead of jaxl.class.php. Or you might also want to create separate index file for each application you build using Jaxl.

Now simply type in

sudo php index.php

on terminal window. You will see the following action logs on the terminal window:
jaxl-jabber-xmpp-library-demo-4
The script broadcast the default message to everyone on the gtalk friend list. In the screen shot you can also see me receiving default message sent by our bot. If any of the friend(s) are offline, the bot sends an offline message to them.

3. RSS feed aggregator bot
Here we will try to make a bot which keep processing RSS feeds in the background. We will also make provisions in this application to retrieve aggregated RSS feed results just by sending simple text messages to the bot. One such application build using Jaxl library is RSS feed integrator for Dzone. You can find this application in the jaxl folder if you have checked out the code from repository. Otherwise download the application from here.

Leave your configuration file as it is from previous applications and instead of jaxl.class.php, include jaxl4dzone.class.php inside index.php. As before simply run the following command on the terminal window:

nohup php index.php > log/logger.log &

.

Read How to get dzone feeds as IM using JAXL? Add dzone@gtalkbots.com for pre-requisites required before running this application and a complete list of provisions made for retrieving rss feeds from this running bot. Here are is a response from the bot, when I send a message reading “php” to the bot:
jaxl-jabber-xmpp-library-demo-5
The application in the background, checks for the incoming message. Further it checks for a cached RSS feed in the cache folder. If the cache is stale or expired, it refetches the RSS feed from Dzone and throw back the results as seen in the screenshot above. If the bot finds a fresh cache of RSS feed in cache directory, it simply throw back the same RSS feed.

4. Custom out-of-office email bot for gmail
jaxl4gmail.class.php is an extension which shows the power of Gmail Extension integration into Jaxl library. This extension allows you to send custom out-of-office email’s to your contacts. I might want to send out a custom out-of-office mail to my colleagues in office and a custom mail to my friends and family.

Download the extension from here, if not already present in the jaxl directory. Include jaxl4gmail.class.php in index.php. Finally run the bot using:

nohup php index.php > log/logger.log &

For details information on how to customize this extension refer this blog post: Programatically control your google mails using JAXL v 1.0.4

5. Building an online multi-user gaming bot
By now we know how to run our bot 24×7 using Jaxl library. In this section we will develop a basic online multi-user gaming bot. Before we go ahead and code our bot, lets decide a few rules for our game:

Users worldwide can add our bot as buddy in gtalk (or using any other IM client). Below are the rules and actions a user can perform:

  • Send a message “start” to enter the multi-user gaming arena.
  • Send a message “stop” to exit the gaming arena.
  • Send a message “options” to view available options for the gaming arena.
  • Any other message sent by the user, will be considered as his answer to the previously broadcasted question. We will make sure “start”, “stop” and “options” are not an answer to any of the question being broadcasted.
  • Whom-so-ever sends a right answer to the broadcasted question receives 5 points. Bot immediately notify everyone in the arena about right answer being received. Thereafter, bot will broadcast the next question to all the users in the arena.
  • Bot reads a list of questions from a file or database as soon as the bot is started. Thereafter, it will keep reading questions from the list of questions randomly and keep broadcasting them to users in the arena.
  • Bot keeps a track of jabber id for incoming (identified by “start”) and outgoing (identified by “stop”) users.
  • Bot also maintains the index of current question being broadcasted

We will code our application in a file called jaxl4gaming.class.php. Here is how the final code will look like. See comments inside the code for more explanation:

jaxl4gaming.class.php (download)

  /* Include XMPP Class */
  include_once("xmpp.class.php");

  class JAXL extends XMPP {

    // List of question contained in an array
    var $questions = array();

    // List of answers corresponding to above questions
    var $answers = array();

    // list of answers which are not allowed for any question
    var $answers_not_allowed = array('start','stop','options');

    // last sent question key (basically index value of question in questions array)
    var $last_question_key = -1;

    // an associative array storing user scores
    var $user_scores = array();

    // stores jabber id of users currently in the arena
    var $user_jids = array();

    // game status
    var $game_status = FALSE;

    /* START OF STANDARD JAXL METHODS */
    function eventMessage($fromJid, $content, $offline = FALSE) {
      // Take action only if the message received is online
      if(!$offline) {
         // trim incoming content
         $content = trim($content);

         // get bare jid for the user
	 $fromJid = $this->getBareJid($fromJid);
	 switch($content) {
	   case 'start':
	     $this->add_user_to_arena($fromJid);
	     break;
	   case 'stop':
	     $this->remove_user_from_arena($fromJid);
	     break;
	   case 'options':
	     $this->display_options($fromJid);
	     break;
	   default:
	     $this->handle_user_message($fromJid, $content);
	     break;
	 }
      }
    }

    // not required for this gaming demo
    function eventPresence($fromJid, $status, $photo) {

    }

    // set the status for our gaming bot
    function setStatus() {
      // Set a custom status or use $this->status
      $this->sendStatus("Type *options* for getting started");
      print "Setting Status...n";
      print "Donen";

      // initialize game
      if(!$this->game_status) {
        $this->logger->logger('Initializing gaming arena....');
        $this->init();
        $this->game_status = TRUE;
      }
    }
    /* END OF STANDARD JAXL METHOD */

    /* START OF GAME CODE METHODS */
    function init() {
      // called when the bot starts
      // read the list of questions and their answers from a txt file
      // populate the $question and $answers array
      // HARDCODING arrays for DEMO purpose.
      $this->questions = array('q1','q2','q3','q4','q5');
      $this->answers = array('a1','a2','a3','a4','a5');
      return TRUE;
    }

    function broadcast_message($message, $except=array()) {
      foreach($this->user_jids as $jid => $info) {
	if(in_array($jid, $except)) continue;
   	else if($this->user_jids[$jid]['status'] == 'online') {
          $this->sendMessage($jid, $message);
        }
      }
      return TRUE;
    }

    function add_user_to_arena($jid) {
       // check if user visited the game before
       // you may want to send some custom welcome messages depending upon the user type
       if(!isset($this->user_jids[$jid])) {
         $this->logger->logger('Adding user_jids key for: '.$jid);
	 $this->user_jids[$jid] = array();
       }

       $this->user_jids[$jid]['status'] = 'online';
       $this->user_jids[$jid]['start_time'] = time();
       $this->logger->logger($jid.' joined the arena: '.json_encode($this->user_jids[$jid]));

       $this->send_current_question($jid);
       return TRUE;
    }

    function send_current_question($jid) {
      // is this the 1st user in the arena
      if($this->last_question_key == -1) $this->last_question_key++;
      $current_question = $this->questions[$this->last_question_key];

      $this->logger->logger('Sending current question at index: '.$this->last_question_key.', question: '.$current_question.' to: '.$jid);
      $this->sendMessage($jid, $current_question);
      return TRUE;
    }

    function broadcast_next_question($except=array()) {
      if($this->last_question_key == count($this->questions)-1) $this->last_question_key = 0;
      else $this->last_question_key++;

      $this->broadcast_message($this->questions[$this->last_question_key], $except);
      return TRUE;
    }

    function broadcast_right_answer($fromJid, $answer, $except) {
      $message = '*'.$fromJid.'* gave the right answer: '.$answer;
      $this->broadcast_message($message, array($fromJid));
      return TRUE;
    }

    function remove_user_from_arena($jid) {
       if(isset($this->user_jids[$jid])) {
	 $this->user_jids[$jid]['status'] = 'offline';
         $this->user_jids[$jid]['end_time'] = $this->user_jids[$jid]['start_time'];
       }
       return TRUE;
    }

    function display_options($jid) {
      $options = '*start* To join the arena, *stop* To quit the arena, *options* To display this help';
      $this->sendMessage($jid, $options);
      return TRUE;
    }

    function handle_user_message($jid, $message) {
      // check if user already exists in the arena
      if(!isset($this->user_jids[$jid]) || $this->user_jids[$jid]['status'] == 'offline') {
        $this->display_options($jid);
        return TRUE;
      }

      // we treat this message as an answer
      $current_answer = $this->answers[$this->last_question_key];
      if($message == $current_answer) {
        $this->increase_user_points($jid);
        $this->broadcast_right_answer($jid, $message, array($jid));
        $this->broadcast_next_question(array());
      }
      else {
        $message = $message.' is a wrong answer. Try again!';
        $this->sendMessage($jid, $message);
      }
      return TRUE;
    }

    function increase_user_points($jid) {
      if(!isset($this->user_jids[$jid]['points'])) $this->user_jids[$jid]['points']=0;
      $this->user_jids[$jid]['points'] += 1;
    }

  }

This is the basic game architecture which will generally be followed while you build games using Jaxl library.
In brief here is the explanation to above code:

  • Initializing game: setStatus() is the last method called during the whole initialization process of bot. Hence this is a right choice to call our game initialization method init(). You can do a number of things in this method. For this demo, I simply hard code the questions and their answers in respective arrays. Once the game is initialized, these $questions and $answers will reside in program memory for the life time.
  • Basic game flow: The flow is simple. I have customized the eventMessage() method provided by Jaxl class. I simply check for a number of cases and divert the flow of the game. If the incoming message is one of the available options, I simply do the respective action. (start triggers add_user_to_arena(), stop triggers remove_user_from_arena() and options trigger display_options() method). If incoming message doesn’t match any available options, I consider it as an attempt to answer the current question and redirect to handle_user_message(). However, if user is yet not a part of the gaming arena and he tries to answer a question, handle_user_message() function will simply redirect to display_options() method.
  • User stats: I also maintain basic user stats in a variable called $user_jids. For each user, I maintain the following fields: 'status' field value can be ‘online’ or ‘offline’ depending upon user availability in the arena. I also maintain a 'start_time' field which indicates when did the user last joined the arena. You might want to have this for a number of reasons. Every time user quits the arena, I also save a field called 'end_time' indicating when did the user last left the arena. Finally, I maintain user points in a field called 'points'. This field is incremented by 1 for every correct answer by the user.
  • Infinite questions: Every time a new question is broadcasted, I check the status of current question key. If it has reached the end of questions array, i simply reset it to 0. Hence the bot will keep serving the questions always. This logic is inside broadcast_next_question() method.
  • Broadcasting messages: broadcast_message() is the main method which broadcast all message from the bot to users playing in the arena. It takes two values as parameters: $message i.e. the message you want to broadcast and $except array which contains user jid’s which you want to skip while broadcasting.

Now how do we test our game. Simply follow the following steps:

  • Download jaxl4gaming.class.php and include it inside index.php
  • Update the config.ini.php file with your production username and password. We will run this bot using gtalk user credentials.
  • Run the bot using
    sudo php index.php
  • Add bot into your gtalk and try to send a message options to it. If everything is fine, you should be able to see the bot performing as we described in the game rules above.
  • Finally customize the methods inside jaxl4gaming.class.php and build your own games.

Debugging your Jaxl bot
In case you run into some error while trying to run Jaxl here are a few things you SHOULD do:

  • Checkout the open/closed issues here. I get lot of queries specially from college enthusiasts and in 99% of the cases solution can be found on the issue’s link above.
  • Another thing you should do is, enable error logging in your php.ini and check the error logs.
  • If you are still unable to find a solution, file a new issue with relevant information here.
  • Search the jaxl forum and discuss with other users who must have encountered similar errors before. Jjoin other users on jaxl’s google group here.
  • Finally, if nothing helps. Send a mail or IM me.

All the best with Jaxl. 😀

Introducing jSlider: A Content Slider plugin for JQuery

jSlider is a light weight JQuery plugin for content sliding. By content we mean everything: HTML code, Images, Advertisements etc etc. jSlider allows to put our content in simple <div>‘s, and then it automatically generates a content slider for you, which one can customize using various options provided.

Screenshot
Below is a screen shot of a content slider using jSlider:

jSlider

Download and Demo
jSlider is hosted at google code. Use the following links for demo and downloads:

Using jslider.jquery.js
Below is a sample html code which will be processed by jslider:

Sample input to jslider

    <div id="jslider">
      <div>
        <input type="hidden" value="Title for Content 1"/>
        <div>
          HTML Content # 1
        </div>
      </div>
      <div>
        <input type="hidden" value="Title for Content 2"/>
        <div>
          HTML Content # 2
        </div>
      </div>
      <div>
        <input type="hidden" value="Title for Content 3"/>
        <div>
          HTML Content # 3
        </div>
      </div>
      <div>
        <input type="hidden" value="Title for Content 4"/>
        <div>
          HTML Content # 4
        </div>
      </div>
      <div>
        <input type="hidden" value="Title for Content 5"/>
        <div>
          HTML Content # 5
        </div>
      </div>
    </div>

One must preserve the div structure as given in the example above. The hidden input values will be taken as heading for that block of content. If you want to have no heading or a common heading, fill this hidden input field appropriately.

Sample output from jslider

  <div id="jslider">
    <div class="slider">
      <h2>Title for Content 1</h2>
      <ul>
        <li class="selected">1</li>
        <li>2</li>
        <li>3</li>
        <li>4</li>
        <li>5</li>
      </ul>
    </div>
    <div class="content">
      <div class="block">
        <input type="hidden" value="Title for Content 1"/>
        HTML Content # 1
      </div>
      <div class="block">
        <input type="hidden" value="Title for Content 2"/>
        HTML Content # 2
      </div>
      <div class="block">
        <input type="hidden" value="Title for Content 3"/>
        HTML Content # 3
      </div>
      <div class="block">
        <input type="hidden" value="Title for Content 4"/>
        HTML Content # 4
      </div>
      <div class="block">
        <input type="hidden" value="Title for Content 5"/>
        HTML Content # 5
      </div>
    </div>
  </div>

Customizing jslider.jquery.js
jSlider provides following options for customization:

  1. width: Width of jslider div above, defaults to ‘610px’
  2. height: Height of jslider div above, defaults to ‘225px’
  3. slider_height: Height of slider div above (navigation bar), defaults to ’24px’
  4. content_height: Height of content div above, defaults to ‘180px’
  5. block_width: Width of block div inside content div’s above, defaults to ‘590px’
  6. block_padding: Padding of block div inside content div’s above, defaults to ’10px’
  7. animation_time: Time taken by 1 slide of content, defaults to 300 msec
  8. animation_type: Animation type, defaults to ‘linear’. Other option is ‘swing’

Rest of the options like various padding etc can be controlled using the css properties. View demo for more implementation details. This is my first jquery plugin and I am only 2 weeks old in jquery. If you find any bug or need any enhancement, you are most welcome.

Building a Custom PHP Framework with a custom template caching engine using Output Control functions

In past 1 year or so, I had opportunities of using a lot of php frameworks including zend, symfony, cakephp, codeigniter. All frameworks have their pros and cons, however that is out of scope of this blog post. You may want to checkout this comparison list of php frameworks here.

In this blog post I will build a custom PHP framework (MVC Architecture). Then go on to discuss in brief about the output control functions and finally show how to build a custom template caching engine using these functions for our framework.

Source Code
You may want to download the complete source code for this blog post from here.

Building a custom PHP Framework
We will choose a MVC architecture for our framework. Here is a basic directory structure for our custom framework:

/index.php
/.htaccess
/config.ini.php
/404.php
/view
    /view/test1.php
    /view/test2.php
/model
    /model/model.class.php
/controller
    /controller/controller.class.php
/log
    /log/logger.class.php
    /log/log.log
/cache
    /cache/cache.class.php
    /cache/template.cache.class.php
    /cache/template
        /cache/template/test1.php.tmp
        /cache/template/test2.php.tmp

The view, model, controller, log and cache directories contains the following framework modules respectively:

  1. view directory contains our view level files. i.e. files containing our HTML, js, css code.
  2. model directory contains the model class responsible for interacting with database and other storages
  3. controller directory contains our controller class. Each incoming request is first received by the controller class constructor, which thereafter controls the flow of request in the framework
  4. log directory contains our logger class. This class is auto loaded for every request providing a basic logger::log($log_message) logger method throughout the framework. This class logs all data in a file called log.log.
  5. cache directory contains our cache class. For this blog tutorial, we will only write the template caching engine class. In production systems, we might have individual classes for other types of cache systems e.g. memcached (Read Memcached and “N” things you can do with it – Part 1 to know more about memcached and MySQL Query Cache, WP-Cache, APC, Memcache – What to choose for a complete comparison lists of various other caching techniques.

Lets see in details, what all file each and every directory contain contains.

Root directory files
We have 4 files in our root directory, namely .htaccess, index.php, config.ini.php and 404.php in order of relevance. Lets look through the content of these files:

.htaccess

RewriteEngine on
RewriteBase /

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*) index.php
  1. 1st two lines essentially means that Switch on the apache rewrite module and set RewriteBase as / i.e. the root directory
  2. Last 3 lines mean that, if incoming request is for a file or directory which physically exists under the root directory serve them otherwise route all other requests to index.php in the root directory

Hence now for an incoming request like http://localhost/test1.php, apache will route the request to index.php in the root directory because there is no test1.php under the root directory. Cool, lets see what index.php has to offer.

index.php

<?php

  // include configuration file
  include_once("config.ini.php");

  // include controller files
  include_once("controller/controller.class.php");

?>

index.php doesn’t do much except for including our core configuration file and controller class file. Controller class constructor is initiated as soon as the class file is included.

config.ini.php is our core configuration file. It provides the framework with an array called $config containing various information like: mysql database credentials, requested url details and various other global parameters. Lets see what all parameter does it provide us with.

config.ini.php

<?php

  $config = array(
                  'host'=>array(
                                'name' => "http://".$_SERVER['HTTP_HOST'].'/',
                                'uri' => $_SERVER['REQUEST_URI'],
                                'url' => parse_url($_SERVER['REQUEST_URI']), // note it contains the parsed url contents
                               ),
                  'mysql'=>array(
                                 'host' => 'localhost',
                                 'name'=> 'testdb',
                                 'user' => 'root',
                                 'pass' => 'password',
                                ),
                  'cache'=>array(
                                 'template' => 'On', // template caching switched on by default
                                 'memcached' => 'Off', // switch off memcached caching
                                ),
                 );

?>

$config[‘host’] array saves various parameter about the host itself, e.g. hostname, hosturi (the requested uri, hosturl (it contains the parse_url(hosturi)).

$config[‘mysql’] array contains mysql database parameters. However in this blog post we will not interact with databases.

$config[‘cache’] tells the framework what all caching modules are switched on.

404.php

<html>
  <head>

  </head>
  <body>
    <h1>404 Page</h1>
  </body>
</html>

Controller directory files
For this blog post, controller directory consists of a single class file. i.e. controller.class.php. We saw this being included by index.php in the root folder above. As soon as controller class is included, it’s constructor is invoked. Before we dig in more, lets see the controller class file:

controller.class.php

<?php

  // include logger class
  include_once("log/logger.class.php");

  // include cache class (contains template caching)
  include_once("cache/cache.class.php");

  // include model class
  include_once("model/model.class.php");

  class controller {

    function __construct($config) {
        global $config;

        // generate requested template name and path
        $config['template']['name'] = $config['host']['uri'] == '/' ? 'index.php' : substr($config['host']['uri'], 1, strlen($config['host']['uri']));
        $config['template']['path'] = "view/".$config['template']['name'];

        // check 404
        if(!file_exists($config['template']['path'])) {
            $config['template']['name'] = "404.php";
            $config['template']['path'] = "404.php";
        }
        logger::log("Requested template name ".$config['template']['name'].", path ".$config['template']['path']);

        // invoke template caching engine
        $template_cache = new template_cache();

        // include the template
        include_once($config['template']['path']);

        // cache template
        $template_cache->setTemplate();
    }

  }

  $controller = new controller($config);

?>

At the top, controller class includes the logger.class.php, cache.class.php and model.class.php files. At the bottom, the controller object is instantiated.

The constructor performs the following 5 tasks:

  1. At first it generates a template name and a template path for the incoming request i.e. for http://localhost/, $config['template']['name']='index.php' and for http://localhost/test1.php, $config['template']['name']='test1.php'.
  2. Second it checks for 404. For the above generated template path, e.g. $config['template']['path']='view/test1.php', it checks whether this file exists inside root directory. If it doesn’t template path and names are set to 404.php
  3. Thirdly, It invokes the template caching engine. i.e. $template_cache = new template_cache();
  4. Forth, it includes the generated template path above i.e. include_once($config['template']['path']);
  5. Fifth and finally, it caches the generated HTML, js, css code by the template file includes above. This is achieved by the following code, $template_cache->setTemplate();

Before we move our attention to, lets see in short the content of log and view directories.

Log directory files
Log directory contains our logger class. This class is auto loaded for each incoming request that is being routed to index.php in the root directory (as we saw above). The logger.class.php provides a static logger::log($log_message) method, which can be used throughout the framework for logging messages. We will be using it everywhere.

logger.class.php

<?php

  class logger {

    static $log_file = "log/log.log";

    static function log($log) {
      if($log != '') {
        $fh = fopen(self::$log_file, "a");
        fwrite($fh,date('Y-m-d H:i:s')."n".$log."nn");
        fclose($fh);
      }
    }

  }

?>

The logger class by default logs all data to a file called log.log.

View directory files
For this blog post, we have two simple test pages in view directory namely test1.php and test2.php, which can be access by typing http://localhost/test1.php and http://localhost/test2.php respectively in the browser.

test1.php

<html>
  <head>

  </head>
  <body>
    <p>
      <?php echo model::test1data(); ?>
    </p>
  </body>
</html>

test1.php simply calls the model class method called model::test1data() (static method). This method extracts some dummy text from the database and returns it back.

Model directory files
Model directory contains the model class file. In production systems, model class file will provide various methods to select and insert data in the databases. However for this blog post we will simply return some static dummy test.

model.class.php

<?php

  class model {

    // This method will return data generally from a database table
    // To keep it simple for the post we return some dummy lipsum text
    static function test1data() {
      logger::log("Returning test1data() from database");
      return "Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin ut nulla ac risus viverra ornare. Nulla consectetur, metus eleifend pharetra posuere, lacus nibh elementum leo, in fermentum lectus lorem in ipsum. Nullam pulvinar purus at erat pharetra volutpat. Pellentesque egestas rutrum lectus, ut rutrum tellus tristique sed. Integer diam est, ornare ac ultricies vel, aliquam non mi. Etiam tempor leo eu lacus tempus sagittis sagittis turpis dictum. Sed leo sapien, pharetra sit amet faucibus et, mollis id nulla. Praesent feugiat mi nec dui scelerisque mollis vehicula magna feugiat. Aliquam erat volutpat. Curabitur quis velit ut nibh rhoncus convallis. Proin mauris nunc, rhoncus vel laoreet vel, aliquet quis nunc. Aenean interdum risus non neque blandit sed adipiscing ipsum mollis. Vivamus enim orci, ultrices at scelerisque vel, laoreet a turpis. Nullam posuere ante sed nisl porta porta aliquam metus suscipit. Fusce enim odio, iaculis at suscipit eget, vestibulum volutpat enim. Nam dictum turpis quis velit posuere in malesuada mi convallis. Donec faucibus, felis id dictum imperdiet, orci tortor tristique neque, vitae lobortis libero tellus sed lorem. Duis tellus magna, commodo eget blandit ut, auctor nec nibh. Maecenas ornare ornare risus nec ultrices. Pellentesque lectus eros, imperdiet ut rhoncus vel, tempus ut nisi.";
    }

    static function test2data() {
      logger::log("Returning test2data() from database");
      return "Vestibulum laoreet nibh sed nulla mollis cursus. Maecenas sodales mauris sit amet ligula euismod a lacinia turpis adipiscing. Nulla gravida porta augue, id adipiscing libero tincidunt ac. Morbi non velit id odio porta tempus id eget massa. Cras nibh purus, gravida sed suscipit ut, tincidunt eu neque. In id est eros, ac sodales orci. Ut lectus augue, feugiat sit amet consectetur id, pharetra quis tellus. Maecenas eget lobortis urna. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Fusce tincidunt eleifend neque. Aenean accumsan orci vitae erat blandit porttitor. Aliquam tristique dolor ac nibh elementum id lacinia diam cursus.";
    }

  }

?>

Cache directory files
For this blog post, cache directory contains the main cache.class.php file which in turn includes various other cache classes e.g. template.cache.class.php

cache.class.php

<?php

  // check for switched on cache modules
  foreach($config['cache'] as $key => $value) {
    // include all cache classes, which are swicted on
    if($value == 'On') {
      // naming convention is <modulename.cache.class.php>
      include_once('cache/'.$key.'.cache.class.php');
    }
  }

?>

Output Control Functions
PHP is a very simple language. You can write a Hello World! code or calculate similarity between two strings (see similar_text()), both with a single line of code. And hence there are a lot of fundamental concepts of PHP, which not only beginners but even some advanced coders can ignore. One such concept is Output Control in PHP.

The Output Control functions allow you to control when the output of your PHP script will be thrown to the browsers (console). i.e. You can pre-process the final html output (append, prepend, chip-chop, inserting ad-codes, url linking, keyword highlighting, template caching), which will otherwise be thrown on the browser. Interesting, isn’t it? Can you feel the power of Output Control functions?

  1. ob_start: This turns on output buffering. i.e. no output is sent from the script, instead the output is saved in an internal buffer. However output buffering doesn’t buffers your headers. ob_start() also takes an optional callback function name. The function is called when output buffer is flushed (see ob_flush()) or cleaned (see ob_clean()). We can access the this internal buffer using functions like ob_get_contents()
  2. ob_end_flush: This function send the content of the buffer (if any) and turns off output buffering. We should always call functions like ob_get_contents() before ob_end_flush(), since any changes after this functions will not reflect on the browser

Building a custom Template Caching Engine
We saw above some of the output control functions PHP has to offer. ob_start(), ob_get_contents() and ob_end_flush(); are the 3 functions we will use to create our custom template caching engine.

template.cache.class.php

<?php

  class template_cache {

    var $template_cache_file = FALSE;
    var $template_cache_file_ext = ".tmp";
    var $template_cache_dir = "cache/template/";
    var $template_cache_ttl = 300; // secs

    function __construct() {

      // initiate template caching
      $this->init();

    }

    function init() {
      // get template path
      $this->template_cache_file = $this->generateTemplatePath();

      // get template from cache if exists
      $this->getTemplate();

      // start output buffering
      ob_start();
    }

    function generateTemplatePath() {
      global $config;

      // generate template file name
      return $this->template_cache_dir.$config['template']['name'].$this->template_cache_file_ext;
    }

    function getTemplate() {
      global $config;

      // check if a cached template exists
      if(file_exists($this->template_cache_file)) {
        if(time() - filemtime($this->template_cache_file) < $this->template_cache_ttl) {
          logger::log("Cache hit for template ".$config['template']['name']);
          $content = file_get_contents($this->template_cache_file);
          echo $content;
          exit;
        }
        else {
          logger::log("Cache stale for template ".$config['template']['name']);
          return FALSE;
        }
      }
      else {
        logger::log("Cache miss for template ".$config['template']['name']);
        return FALSE;
      }
    }

    function setTemplate() {
      global $config;

      // get buffer
      $content = ob_get_contents();

      // save template
      logger::log("Caching template ".$config['template']['name']);
      $fh = fopen($this->template_cache_file, 'w');
      fwrite($fh, $content);
      fclose($fh);

      // Flush the output buffer and turn off output buffering
      ob_end_flush();
    }

  }

?>

As we saw above in controller class, the template engine class was instantiated before including the actual template file. Template engine constructor do the following 3 tasks:

  1. Generate a cached file name for the requested uri by calling the $this->generateTemplatePath(); method. e.g. if http://localhost/test1.php is the requested uri, test1.php.tmp is it’s static cached template
  2. Secondly, it tries to fetch the cached template file by calling the method $this->getTemplate(); (read on for details of this method)
  3. Finally it turns on output buffering by calling ob_start();

List of methods provided by template.cache.class.php are:

  1. generateTemplatePath() method generates a cache file name for incoming request. By default extension of all cached files in “.tmp” and are stored under the /cache/template directory.
  2. getTemplate() method do a number of tasks. First, it checks if a cached template exists for the requested uri. If it does not exists or if it is not a fresh cache (see $template_cache_ttl), this method simply returns control back to controller which go ahead and include the actual template file. However if the file exists and is fresh it reads the content of the file and throw back to browser. At this point control is no longer transferred back to the controller, hence saving various un-necessary processing and database calls.
  3. setTemplate() method is called by controller after including the actual template file from under the view directory. Point to note is that, before getTemplate() returns control back to controller (in case of missed or stale cache), the template cache class constructor does switch on output buffering. And when setTemplate() method is called, we can access this buffer using output functions like ob_get_contents() and then save the template for next incoming request. Bingo!. Finally this method throw away the buffer to the browser using ob_end_flush();

Is it working?
To verify the flow of framework, I hit the url http://localhost/test1.php 3 times, with $template_cache_ttl = 10; (seconds).

  1. Once after clearing the template cache folder
  2. Once within next 10 seconds
  3. And finally after 10 seconds

Here is how the log file looks like:

2009-08-16 19:50:49
Requested template name test1.php, path view/test1.php (1st REQUEST)

2009-08-16 19:50:49
Cache miss for template test1.php

2009-08-16 19:50:49
Returning test1data() from database

2009-08-16 19:50:49
Caching template test1.php

2009-08-16 19:50:54
Requested template name test1.php, path view/test1.php (2nd REQUEST)

2009-08-16 19:50:54
Cache hit for template test1.php

2009-08-16 19:51:03
Requested template name test1.php, path view/test1.php (3rd REQUEST)

2009-08-16 19:51:03
Cache stale for template test1.php

2009-08-16 19:51:03
Returning test1data() from database

2009-08-16 19:51:03
Caching template test1.php

Moving forward, What’s Next? Extending template.cache.class.php
Template cache class can be extended to do a lot more, other than caching the template files. For instance we might want to perform (chip-chop, append, prepend etc) a few tasks, before we cache the final template and throw back to the browser. Few tasks which look quite obvious to me are:

  1. Short Codes: We can insert short codes in our HTML templates, which later on can be expanded into full fledged codes. e.g. For embedding a YouTube video, we can simply put something like [[YouTube yjPBkvYh-ss]] into test1.php. And in setTemplate() method we can call helper/plugin methods to process such short codes. More professionally, we can add hooks for various tasks we might want to perform before caching the template. Read How to add wordpress like add_filter hooks in your PHP framework for a more professional approach.
  2. Inserting page header and footer: Instead of including page header and footer inside test1.php, we can simply put our main <body> code inside test1.php. Then before caching the template file, we can append and prepend header and footer modules to the buffer of each page. Thus avoiding including the same header and footer files across various pages.
  3. HTML module caching: There are several instances where we can have a common module across all pages. For instance, I can have a events module across all my pages, which basically displays a calendar with various events for the week or month marked on it. The event details are extracted from the database. Since this module of mine is a static HTML chunk for atleast a week, I would like to have a difference cache for this module. Intelligently hooking up these modules with template caching engine, can allow us to do module level caching

I can probably write down 10-15 more such applications and probably there might be many more such applications of the above coded template caching engine. (Note: The power actually lies in Output Control Functions provided by PHP).

Let me know if you liked the post or any bug in it.

fixed.js – Solution to IE6 “position:fixed” Bug

IE6 has been a bane for all frontend developers for years. An element can be positioned relative to the browser window using the style position:fixed, it does not move when the page is scrolled. You can do nice layout things (e.g. facebook chat bar) with this in most modern browsers but not for IE6 in windows until you use fixed.js

Sample Code for Facebook Type Chat Bar

<html>
  <head>
    <style type="text/css">
      body {
        margin:0px;
        padding:0px;
        height:1600px;
        position:relative;
      }
      #shoutbox {
        position:fixed;
        background-color:#F2F2F2;
        border-top:1px solid #CCCCCC;
        bottom:0px;
        left:0px;
        width:100%;
        height:25px;
      }
    </style>
    <script type="text/javascript" src="fixed.js"></script>
  </head>
  <body>
    <div id="shoutbox">
      <!-- Our Shoutbox -->
    </div>
  </body>
</html>

The above code will show you a bar at the bottom of the page, which remains fixed at the bottom even if you scroll the page.

How does fixed.js help me?

  1. fixed.js is smart enough to invoke only if the browser is IE6 on Windows machine. This is achieved by these two lines of code in fixed.js.
    /*@cc_on
    @if (@_win32 && @_jscript_version>4)
  2. It specifically tells IE6 how to render elements with position:fixed attributes. Which are otherwise ignored by IE6.
  3. For remaining browsers, fixed.js go to sleep silently. Doesn’t do any processing.

Download fixed.js
fixed.js is developed and maintained at doxdesk.com. Click here to download fixed.js

Happy Coding!

How to generate random password like WordPress using PHP?

WordPress Blogging Engine is a champion in a lot of way. One of the unique thing which you might have noticed is the random password generated by the wordpress, in case you try to generate a new password. Here are a few examples:

  • j0LH(WM9b_-q
  • wr^sqct1cmff
  • )P4-e531#-aL

Lets have a look at the code which can generate such random passwords for us. Later on we will dig deep into the code to understand each and every bit of it:

<?php

  class utility {

    static $random = '';

    // generates a random password
    // By default of length 12 having special characters
    static function generate_password($length = 12, $special_chars=true) {
      $chars = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789';
      if($special_chars) $chars .= '!@#$%^&*_-()';

      $password = '';
      for($i=0; $i<$length; $i++)
        $password .= substr($chars, self::generate_random_number(0, strlen($chars)-1), 1);
      return $password;
    }

    // generates a random number between $min and $max
    static function generate_random_number($min=0, $max=0) {
      // generate seed. TO-DO: Look for a better seed value everytime
      $seed = mt_rand();

      // generate $random
      // special thing about random is that it is 32(md5) + 40(sha1) + 40(sha1) = 112 long
      // hence if we cut the 1st 8 characters everytime, we can get upto 14 random numbers
      // each time the length of $random decreases and when it is less than 8, new 112 long $random is generated
      if(strlen(self::$random) < 8 ) {
        self::$random = md5(uniqid(microtime().mt_rand(), true).$seed);
        self::$random .= sha1(self::$random);
        self::$random .= sha1(self::$random.$seed);
      }

      // take first 8 characters
      $value = substr(self::$random, 0, 8);

      // strip first 8 character, leaving remainder for next call
      self::$random = substr(self::$random, 8);

      $value = abs(hexdec($value));
      // Reduce the value to be within the min - max range. 4294967295 = 0xffffffff = max random number
      if($max != 0) $value = $min + (($max - $min + 1) * ($value / (4294967295 + 1)));
      return abs(intval($value));
    }

  }

  // print new random password
  echo utility::generate_password();

?>

Lets dig into the code
static function generate_password($length=12, $special_chars=true) is a static method of our utility class. It accepts two parameters. $length who’s default value is 12 and $special_chars who’s default value is true. By turning on $special_chars, our random generated password will include characters like !@#$%^&*_-()

static function generate_random_number($min=0, $max=0) is yet another static function of the utility class. It generates random number between $min and $max, the two parameters which can be passed. Default value for both is 0. However internally, generate_random_number() do a lot of trick to get us some really random numbers.

Algorithm
generate_random_number() works with following variables:

  1. $seed: which is equal to mt_rand()
  2. self::$random is a static variable. To start with this variable equals to ” (nothing). generate_random_number() checks for the length of self::$random. If its length is < 8, it generates a new 112 character long random value (see the code above) and assign it to self::$random. From here on, every time a random number is requested, it uses a chunks of 8 characters from the starting of self::$random, which is then used to generate a random number (see the code above). After each iteration length of self::$random decreases by 8. Because self::$random is 112 characters long, we can use it 14 times to get a random number (14×8 = 112).
  3. $value is the actual 8 digit character extracted from the starting of self::$random, which is later on processed to generate a random number between $min and $max values.

Further these methods can also be used to generate short url’s like tinyurl.com or bit.ly.

Getting started with Autotools – GNU Build System on Debian

The_GNU_logo
If you eat and drink open source, chances are high that you might have downloaded an open source project code, only to see files like: aclocal.m4, configure.ac, Makefile.am, Makefile.in and what not. You might have also used commands like ./configure, make etc but what are these files? Does they really belong to the project you download? Do I need to understand them? In this blog post I look forward to answer all your question, as well as introduce you to not so popular Autotools – A GNU Build System.

Setting up Autotools on Debian?
Before we go ahead and understand what Autotools is, we will try building a HelloWorld package. Lets get started by setting up Autotools on debian machine.

  • apt-get install build-essential
  • gcc –version (verifying install)
  • g++ –version (verifying install)
  • apt-get install automake autoconf

You have your environment ready. Lets start packaging the HelloWorld package.

Hello World Source Code
Download full source code from here

We will need to create 5 files for our basic HelloWorld package. Start by creating a directory structure like this:
HelloWorld
    — configure.ac
    — Makefile.am
    — README
    — src
        — Makefile.am
        — helloworld.c

src/helloworld.c

#include <config.h>
#include <stdio.h>

int main (void) {
    puts ("Hello World!");
    puts ("This is " PACKAGE_STRING ".");
    return 0;
}

Note we don’t have a config.h file but still we include it here. In actual config.h will be autogenerated by the autotools, when we build the package. Similarly, PACKAGE_STRING will be a pre-defined variable inside config.h.

src/Makefile.am

bin_PROGRAMS = helloworld
hello_SOURCES = helloworld.c

Here we tell the build system to generate a binary named helloworld using the sources defined below i.e. helloworld.c

Makefile.am

SUBDIRS=src
dist_doc_DATA=README

Here we give information about the various sub-directory. For a bigger project you might have a man directory, data directory etc. Also we tell the build to package README file with the build.

README

This is a demonstration HelloWorld package for GNU Automake.
Type `info Automake' to read the Automake manual.

configure.ac

AC_INIT([helloworld], [1.0], [emailid@provider.com])
AM_INIT_AUTOMAKE([-Wall -Werror foreign])
AC_PROG_CC
AC_CONFIG_HEADERS([config.h])
AC_CONFIG_FILES([
 Makefile
 src/Makefile
])
AC_OUTPUT

Don’t leave the post on seeing the above file. We will go through each and every one of them. configure.ac contains a series of M4 macros that will expand to some shell code to finally generate the configure script. Autotools have utilities like automake and autoconf (details below) which read this file to generate intermediate and final build files. The variables starting with AC_ are Autoconf macros and those starting with AM_ are Automake macros.

  1. AC_INIT: Initializes autoconf. It takes 3 input parameters: Name of the package, Version of the package and Contact address for bug reports
  2. AM_INIT_AUTOMAKE: Initializes automake. It can take a number of available input parameters. -Wall -Werror specifically tells automake to turn on all warnings and report them as error. While development we will keep error reporting turned on. foreign tells automake that this package doesn’t follow GNU standard. As per GNU standards we should also distribute files like ChangeLog, AUTHORS and at this stage we don’t want automake to complaint about them.
  3. AC_PROG_CC: This line tells configure script to search available C compilers and define variable CC with its name. Later on many intermediate files will use this variable CC for building binary files.
  4. AC_CONFIG_HEADERS: It tells the configure script to generate a config.h file which is pre-included by helloworld.c. Generated config.h will have content like this:
    /* config.h.  Generated from config.h.in by configure.  */
    /* config.h.in.  Generated from configure.ac by autoheader.  */
    
    /* Name of package */
    #define PACKAGE "helloworld"
    
    /* Define to the address where bug reports for this package should be sent. */
    #define PACKAGE_BUGREPORT "emailid@provider.com"
    
    /* Define to the full name of this package. */
    #define PACKAGE_NAME "helloworld"
  5. AC_CONFIG_FILES: This tells configure script list of files from which it should generate it’s *.in templates. This variable is also used by automake utility to know list of Makefile.am it should process. (Note: Each directory should have a Makefile.am file and as you keep adding new directories keep adding them to AC_CONFIG_FILES, else build will not consider your new directories while building packages.
  6. AC_OUTPUT: It is a closing command that actually produces the part of the script in charge of creating the files registered with AC_CONFIG_HEADERS and AC_CONFIG_FILES.

Building a Hello World package for distribution
Lets create our first package for distribution.

  1. cd path/to/helloworld/directory: Migrate to the project directory
  2. autoreconf –install: This command initiates the build system. You should see something like this as output:
    configure.ac:2: installing `./missing'
    configure.ac:2: installing `./install-sh'
    src/Makefile.am: installing `./depcomp'
    

    Also if you scan through the HelloWorld directory, you will find a lot of new files being generated by the build system. Particularly you will see a Makefile.in being generated for each Makefile.am. Apart from these files of interest are configure and config.h.in.

  3. ./configure: It utilizes *.in files generated by the previous step to build the Makefile, src/Makefile and config.h. You should see something like this on your console:
    checking for a BSD-compatible install... /usr/bin/install -c
    checking whether build environment is sane... yes
    checking for a thread-safe mkdir -p... /bin/mkdir -p
    checking for gawk... no
    checking for mawk... mawk
    checking whether make sets $(MAKE)... yes
    checking for gcc... gcc
    
  4. make
  5. src/helloworld: This will output this on the console.
    Hello World!
    This is helloworld 1.0.
  6. make distcheck: This utility finally creates the helloworld-1.0.tar.gz package for distribution. You should see this on your console on running this utility:
    ================================================
    helloworld-1.0 archives ready for distribution:
    helloworld-1.0.tar.gz
    ================================================

Installing distributed HelloWorld package

  1. Copy the generated package into your temp directory and then issue the following commands
  2. tar -xzvf helloworld-1.0.tar.gz
  3. cd helloworld-1.0
  4. ./configure
  5. make
  6. make install

make install will copy the helloworld binary into the /usr/local/bin directory. Try running helloworld from command line and you should see a similar output, as we saw above while building the package. Further it also copies the README file under /usr/local/share/doc/helloworld directory. If your built package includes the man directory, it gets copied to /usr/local/share/man automatically.

What is Autotools?
Autotools is a build system developed by GNU which helps you distribute your source code across various Unix systems. The files you are wondering about are auto generated by the Autotools.

Autotools is a combination of several utilities made available by GNU, including:

  1. Autoconf
  2. Automake

There are many others which can be listed above, but for this blog post we will restrict ourselves to Automake and Autoconf only.

Autoconf
autoconf process files like configure.in to generate a configure script. When we run the configure script, it reads other template files like Makefile.in to generate a final output file, in this case Makefile

Automake
It reads all Makefile.am and generate corresponding Makefile.in, used by the configure script as described above.

Happy Packaging!