How to perform X-FACEBOOK-PLATFORM and Google Talk X-OAUTH2 XMPP authentication with PHP Jaxl library

Ever since Jaxl library first introduced support for X-FACEBOOK-PLATFORM XMPP authentication mechanism, it has changed significantly. Also, Google Talk now supports OAuth 2.0 Authorization, an XMPP extension to allow users to log in using OAuth 2.0 credentials.

Both these mechanisms are a big win for XMPP developers, since real-time conversation experience can now be provided to their application users without asking them for their passwords. In this blog post, I will demonstrate how to perform X-FACEBOOK-PLATFORM and X-OAUTH2 XMPP authentication mechanism using Jaxl v3.x PHP Library.

X-FACEBOOK-PLATFORM XMPP Authentication
Here is a quick guide on how to perform X-FACEBOOK-PLATFORM XMPP authentication using xfacebook_platform_client.php which comes bundled with Jaxl v3.x examples:

  • Visit Facebook Developer Apps page and register your application
  • Once registered, visit access token tool to get required parameters to perform X-FACEBOOK-PLATFORM authentication Facebook Access Token Tool
  • Click on the debug button next to User Token and make sure xmpp_login is one of the extended permissions (scope)
  • Enter downloaded Jaxl library folder and run from command line as follows:

    $ php examples/xfacebook_platform_client.php fb_user_id_or_username fb_app_key fb_access_token

You can now take the source code of xfacebook_platform_client.php and customize it for your application needs.

Google Talk X-OAUTH2 XMPP Authentication
Here is a quick guide on how to perform Google Talk X-OAUTH2 XMPP authentication using xoauth2_gtalk_client.php which comes bundled with Jaxl v3.x examples:

  • Visit Google OAuth Playground and input https://www.googleapis.com/auth/googletalk as the required scope. Press “Authorize API” and then “Allow Access” button on the redirected page
  • In step 2, simply press “Exchange authorize code for tokens” and copy the access token
  • Enter downloaded Jaxl library folder and run from command line as follows:

    $ php examples/xoauth2_gtalk_client.php [email protected] access_token

You can now take the source code of xoauth2_gtalk_client.php and customize it for your application needs.

Wasn’t that simple 🙂

Facebook chat connect with X-FACEBOOK-PLATFORM using Jaxl 2.0

Facebook chat provides two authentication mechanisms for authenticating chat client users. DIGEST-MD5 require chat client users to enter their username and password, while X-FACEBOOK-PLATFORM can be used to provide better user experience by using simple Facebook Platform authentication. In this blog post, I will demonstrate how to use Jaxl library for X-FACEBOOK-PLATFORM authentication.

Echobot using X-FACEBOOK-PLATFORM
Setup Jaxl library on your system and edit packaged sample echobot application with facebook user account details. Alternately you can also specify connecting user details inside jaxl.ini configuration file.

        $jaxl = new JAXL(array(
                'user'=>'fbUsername',
                'pass'=>'', // Not required, we will use user session key instead
                'host'=>'chat.facebook.com',
                'domain'=>'chat.facebook.com'
        ));

Add callback for hook jaxl_get_facebook_key:

JAXLPlugin::add('jaxl_get_facebook_key', array($echobot, 'getFacebookKey'));

Complete getFacebookKey method inside echobot application, which should return back following key information:

function getFacebookKey() {
                        return array(
                                '', // Your application secret key
                                '', // Your application api key
                                '' // Connecting user session key
                        );
                }

Update doAuth method to use X-FACEBOOK-PLATFORM auth mechanism:

                function doAuth($mechanism) {
                        global $jaxl;
                        $jaxl->auth("X-FACEBOOK-PLATFORM");
                }

Finally, run echobot from command line:

[email protected]:/usr/share/php/jaxl/app/echobot# jaxl echobot.php
[1942] 2010-08-08 05:35:10 - Socket opened to the jabber host chat.facebook.com:5222 ...
[1942] 2010-08-08 05:35:11 - Performing Auth type: X-FACEBOOK-PLATFORM
[1942] 2010-08-08 05:35:26 - Auth completed...

Writing your first facebook chat bot in PHP using Jaxl library

Today facebook officially announced availability of it’s chat through jabber/xmpp clients. This is a big win for XMPP, with almost 400 million new probable users adding into XMPP club. In this post, I will demonstrate how to connect to facebook chat servers using Jaxl client library in PHP. It can further be used to make custom chat bots for facebook.

Creating your first facebook chat bot:
Follow the steps to successfully run a facebook chat bot:

  1. Download Jaxl or checkout latest from trunk
    svn checkout http://jaxl.googlecode.com/svn/trunk/ jaxl-read-only
  2. Edit the configuration file config.ini.php as follows:
      // Set an enviornment
      $env = "prod";
    
      $key = array("prod"=>array("user"=>"facebook_username",
                                 "pass"=>"facebook_password",
                                 "host"=>"chat.facebook.com",
                                 "port"=>5222,
                                 "domain"=>"chat.facebook.com"
                                ),
  3. Run from command line:
    [email protected]:/jaxl$ sudo php index.php
    OSType: Linux, Registering shutdown for SIGINT and SIGTERM
    OpenSSL: Enabled for CLI
    Attempting DIGEST-MD5 Authentication...
    Starting Session...
    Requesting Feature List...
    Requesting Roster List...
    Setting Status...
    Done
    

Try to send a message to your running chat bot and you shall receive a default message back from the bot saying “Hi, Thanks for your message”.

See further sample codes and explaination on how to build a full fledged gaming chat bots under xmpp category.

Facebook type image rotation and more using PHP and Javascript

If you are a facebook geek like me, you must have noticed till now the image rotate functionality in the photo albums. Facebook allows you to rotate images 90 degree clockwise and anti-clockwise after image upload. If you haven’t tried that till now, below is a screenshot for your convenience.

facebook-image-rotate

Question:
But the question is how does facebook team succeed doing this in one click. Today I tried looking around for a solution over internet and I came across the inbuilt imagerotate functionality in PHP.

Problem:
Unfortunately the problem is that even if you have GD Library extension enabled in PHP, imagerotate function just doesn’t work. After some research I found that to enable imagerotate function inside PHP, you need to compile and build PHP manually and enable imagerotate while installing PHP. You can check your PHP installation support for imagerotate using the following command line:

php -r "var_dump(function_exists('imagerotate'));"

Solution:
Since I didn’t want to touch my current installation of PHP, ImageMagick came to my rescue. Below I will show you how did I achieve cloning facebook’s 90 degree rotation functionality and also added a custom degree rotation (functionality usually seen in collage tools).

Demo:
You can try a live demo of the application here:
http://abhinavsingh.com/webdemos/imagerotate/

facebook-image-rotate-using-php-javascript

Source Code:
Before you go ahead and try this demo, you will need to install imagemagick on your system. On debian and ubuntu this can be achieved by the following command:

apt-get install imagemagick

To make sure installation is alright, run the following command from the shell:

convert -version

You should see something like this:

Version: ImageMagick 6.2.4 02/10/07 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2005 ImageMagick Studio LLC

Convert is a command line utility provided by imagemagick. Read more about convert here.

The source code consists of the following files and folders:

  • index.php : Generates the Frontend UI part for the application
  • action.php : Responsible for handling requests and rotating image using imagemagick library
  • style.css : Used for styling the UI
  • script.js : Used for handling the horizontal slider
  • images : This folder contains all the required images for the application. The image which we will be rotating left and right is “me.jpg”

action.php

  // Image Path
  $image = "images/me.jpg";
  $original = "images/me-original.jpg";

  if($_POST['restore']) {
    exec("cp ".$original." ".$image);
  }
  else if($_GET['left'] == 1 || $_GET['right'] == 1 || $_POST['degree']) {
    // Rotation Degree
    if(isset($_GET['left'])) $degree = -90;
    if(isset($_GET['right'])) $degree = 90;
    if(isset($_POST['degree'])) $degree = $_POST['degree'];

    // rotate image
    if(is_numeric($degree)) exec("convert ".$image." -rotate ".$degree." ".$image);
  }

From the UI, a user can pass 4 kind of requests:

  • Rotate Image 90 degree clockwise by clicking the button on bottom right corner
  • Rotate Image 90 degree anti-clockwise by clicking the button on bottom right corner
  • Rotate Image x degree, by choosing the value of x using horizontal slider on bottom left
  • Restore the image back to the original self

action.php handles the incoming 4 cases, calculates $degree to rotate and passes as a parameters to the command line utility provided by imagemagick.

Download Code:
Download the complete source code from here:
http://abhinavsingh.googlecode.com/files/image-rotate-application.rar
Unzip into a folder, and make the images directory writable.
PS: Application not tested on IE browser

Don’t forget to share and leave a comment if you liked the post.
Cheers!

Konami Code on Facebook : How to implement it on your site

Yesterday while chatting with friends, I discovered an unusual thing on facebook. Unusual because I didn’t see any sense in what I saw back then. The process to discover this unusual thing is as follows:

How to view lens flare on facebook
1. Login to facebook.com
2. Click once anywhere on your home page
3. Click the following sequence of characters using your keyboard
4. Up, Up, Down, Down, Left, Right, Left, Right, B, A, Enter Key
5. Click again on the page or try to scroll the page up or down

If everything is done correctly you will see something like this:
lensflare on facebook

On more investigation I found out this unusual thing is nothing but “Konami Code“, which is a cheat code that appears in many Konami video games. The Konami code was introduced to many gamers in the 1988 NES version of Contra. Due to the game’s intense difficulty, many Contra players became reliant on the cheat code, which increased the player’s lives from 3 to 30 (9 to 90 including continues), to finish the game. The game’s popularity, combined with the near-necessity of using the code, made it renowned to an entire generation of video game players. This led to the code being used in countless games, and even mentioned across popular culture.

Once the cheat code is entered, facebook extends it to enable Lens Flare on all the pages you visit thereafter. You can use the below code, to enable-disable komani code on your site, and thereafter extend it to anything you want to:

  <head>
    
  </head>
  <body onload="$.init()" onkeydown="$.konamiCode(event)">
  </body>

Try out once on facebook and then come back to grab it for your site.
Enjoy!

Memcached and “N” things you can do with it – Part 1

In my last post MySQL Query Cache, WP-Cache, APC, Memcache – What to choose, I discussed in brief about 4 caching technologies which you might have used knowingly or unknowingly.

Towards the end we came to a conclusion that memcached is the best caching solution when you are looking for speed and number of hits per second. By my experience, memcached is capable of handling more than a 100 Million PV’s per month without any problem. However, towards the end I did also discussed why memcached is unreliable and unsecure.

In this post I will dig a level deeper into memcached. For ease here is the so called table of content:

  1. Basics: Memcached – Revisited
  2. Code Sample: A memcached class and how to use it
  3. N things: What else can I do with memcached
  4. Case Study: How Facebook uses memcached
  5. DONT’s: A few things to be taken care

Basics: Memcached – Revisited
Memcached was developed by Brad when live journal was hitting more than 20 Million PV’s per day. Handling 20 Million PV’s was no joke and he needed a better solution to handle such a high traffic. Since most of the blogs doesn’t change once published, he thought of having a model where he can skip the database read for the same posts again and again or atleast reduce the number of database reads. And hence came Memcached. Memcached is a deamon which runs in the background. By deamon you may think of a process running in the background doing its job.

If you are using ubuntu or debian like me, here are the steps for installing memcached:

  1. sudo apt-get install php5-memcache
  2. Go to /etc/php5/conf.d/memcache.ini and uncomment the line ;extension=memcache.so to enable this module
  3. sudo pecl install memcache
  4. Go to php.ini file and add this line: extension=memcache.so
  5. sudo apt-get install memcached
  6. sudo /etc/init.d/memcached start
  7. Restart Apache

If you are on windows, here are the steps which will help you running memcached on windows machine:

  1. Download the memcached win32 binary from my code.google vault.
  2. Unzip the downloaded file under C:memcached
  3. As we need to install memcached as a service, run this from a command line: C:memcachedmemcached.exe -d install from the command line
  4. Memcache by default loads with 64Mb of memory which is just not sufficient enough. Navigate to HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesmemcached Server in your registry and find the ImagePath entry. Change that to “C:memcachedmemcached.exe” -d runservice -m 512
  5. Start the server by running this command: C:memcachedmemcached.exe -d start
  6. Open folder C:PHPext and check for php_memcache.dll. If you are unlucky not to have that download from here for PHP-4.x and from here for PHP-5.x
  7. Add extension=php_memcache.dll in your php.ini file
  8. Restart Apache
  9. Download the full package consisting of exe’s, classes and dll’s from here.

A few other options which you can use for starting memcached are:
memcached -d -m 2048 -l 10.0.0.40 -p 11211 , This will start memcached as a daemon, using 2GB of memory, and listening on IP 10.0.0.40, port 11211. Port 11211 is the default port for memcached.

By now I assume you have a 🙂 on your face, because you have memcached deamon running on your machine. Windows users can check that by opening up the task manager and looking for a memcached.exe process. I don’t see any reason for you not smiling, but if in case you are that unlucky windows user, please leave that system and move to a unix machine. Atleast try running Ubuntu on windows by reading this post of mine How to configure Ubuntu and LAMP on Windows.

Code Sample: Memcached class
So we have memcached setup on our system. Now we will try to hook up a simple code with it, which will do all the necessary talking with the deamon. Below I will demonstrate a very basic and simple application which will help you getting started. This application contains a total of 5 files namely: database.class.php, memcache.class.php, log.class.php, index.php, memcacheExtendedStats.php.

log.class.php
This is a very basic logger class which will log every thing for you to inspect later. Here is the code:

  class logAPI {
    var $LogFile = "log.txt";

    function Logger($Log) {
      $fh = fopen($this->LogFile,"a");
      fwrite($fh,$Log);
      fclose($fh);
    }
  }

When ever the code connects with memcached deamon, or database or fail to connect to either of them , this class will log the appropriate message in log.txt

database.class.php
This is another basic database class which have methods like getData() and setData(). getData() is used while trying to retrieve rows from the database while setData() is used while updating or inserting new rows in the database. For this demo application we will only be using the getData() method. Here is the code:

  include_once("memcache.class.php");
  include_once("log.class.php");

  class databaseAPI {

    /************************/
    /** Database information **/
    /************************/
    var $dbhost = "localhost";
    var $dbuser = "root";
    var $dbpass = "";
    var $dbname = "gtalkbots";
    var $db = NULL;

    /******************************/
    // CONSTRUCTOR DEFINITION //
    /******************************/
    function __construct() {
      $this->connect();
    }

    /*************************************************/
    // Function establishes a connection to database //
    /*************************************************/
    function connect() {
      // Connect to the dbhost
      $connection = mysql_connect($this->dbhost,$this->dbuser,$this->dbpass) or die(mysql_error());

	  // If connection fails send a mail to $dbmail about the same
      if(!$connection) {
        echo "Failed to establish a connection to host";
        exit;
      }
      else {
        // Connect to dbname
        $database = @mysql_select_db($this->dbname);

	// If fails to connect to the database send a mail to $dbmail
        if(!$database) {
          echo "Failed to establish a connection to database";
          exit;
        }
        else {
          $this->db = $connection;
        }
      }
    }

    /*******************************************/
    // Function closes the database connection //
    /*******************************************/
    function close() {
      mysql_close($this->db);
    }

    /***********************************************************/
    // Function executes the query against database and returns the result set   //
    // Result returned is in associative array format, and then frees the result //
    /***********************************************************/
    function getData($query,$options=array("type"=>"array","cache"=>"on"),$resultset="") {
        // Lookup on memcache servers if cache is on
	if($options['cache'] == "on") {
	    $obj = new memcacheAPI();
	    if($obj->connect()) {
	        // Try to fetch from memcache server if present
		$resultset = $obj->getCache($query);
	    }
    	    else {
	        // Fetch query from the database directly
	    }
	}
	// If $resultset == "" i.e. either caching is off or memcache server is down
        // OR $resultset == null i.e. when $query is not cached
	if($resultset == "" || $resultset == null) {
	    $result = mysql_query($query,$this->db);
	    if($result) {
		if($options['type'] == "original") {
		    // Return the original result set, if passed options request for the same
		    $resultset = $result;
		}
                else if($options['type'] == "array") {
	            // Return the associative array and number of rows
	    	    $mysql_num_rows = mysql_num_rows($result);
		    $result_arr = array();
		    while($info = mysql_fetch_assoc($result)) {
		        array_push($result_arr,$info);
		    }
		    $resultset = array("mysql_num_rows" => $mysql_num_rows,"result" => $result_arr,"false_query" => "no");
		}
  		// Cache the $query and $resultset
		$obj->setCache($query,$resultset);
		$obj->close();
		return $resultset;

                // Free the memory
		mysql_free_result($result);
            }
            else {
	        $resultset = array("false_query" => "yes");
	        return $resultset;
            }
	}
	else {
	    // If $query was found in the cache, simple return it
	    $obj->close();
	    return $resultset;
	}
    }

    /**********************************************************/
    // Function executes the query against database (INSERT, UPDATE) types   //
    /************************************************************/
    function setData($query) {
      // Run the query
      $result = mysql_query($query,$this->db);
      // Return the result
      return array('result'=>$result,'mysql_affected_rows'=>mysql_affected_rows());
    }

  }

memcache.class.php
Memcache class consists of two methods: getCache() and setCache(). getCache() will look up for the (key,value) pair in memory. If it exists, the method unserialize it and returns back. setCache() is used to set (key,value) pair in memory. It accepts the key and value, serialize the value before storing in cache.

  class memcacheAPI {

	/* Define the class constructor */
	function __construct() {
	  $this->connection = new Memcache;
	  $this->log = new logAPI();
	  $this->date = date('Y-m-d H:i:s');
	  $this->log->Logger("[[".$this->date."]] "."New Instance Created
n"); } /* connect() connects to the Memcache Server */ /* returns TRUE if connection established */ /* returns FALSE if connection failed */ function connect() { $memHost = "localhost"; $memPort = 11211; if($this->connection->connect($memHost,$memPort)) { $this->log->Logger("[[".$this->date."]] "."Connection established with memcache server
n"); return TRUE; } else { $this->log->Logger("[[".$this->date."]] "."Connection failed to establish with memcache server
n"); return FALSE; } } /* close() will disconnet from Memcache Server */ function close() { if($this->connection->close()) { $this->log->Logger("[[".$this->date."]] "."Connection closed with memcache server
n"); $this->log->Logger("=================================================================================================
nn"); return TRUE; } else { $this->log->Logger("[[".$this->date."]] "."Connection didn't close with memcache server
n"); $this->log->Logger("=======================================================================================================
nn"); return FALSE; } } /* getCache() function will fetch the passed $query resultset from cache */ /* returned resultset is null if $query not found in cache */ function getCache($query) { /* Generate the key corresponding to query */ $key = base64_encode($query); /* Get the resultset from cache */ $resultset = $this->connection->get($key); /* Unserialize the result if found in cache */ if($resultset != null) { $this->log->Logger("[[".$this->date."]] "."Query ".$query." was found already cached
n"); $resultset = unserialize($resultset); } else { $this->log->Logger("[[".$this->date."]] "."Query ".$query." was not found cached in memcache server
n"); } return $resultset; } /* setCache() function will set the serialized resultset on Memcache Server */ function setCache($query,$resultset,$useCompression=0,$ttl=600) { /* Generate the key corresponding to query */ $key = base64_encode($query); /* Set the value on Memcache Server */ $resultset = serialize($resultset); if($this->connection->set($key,$resultset,$useCompression,$ttl)) { $this->log->Logger("[[".$this->date."]] "."Query ".$query." was cached
n"); return TRUE; } else { $this->log->Logger("[[".$this->date."]] "."Query ".$query." was not able to cache
n"); return FALSE; } } }

index.php
With everything in place, its time to test memcached. We will check if memcached is working fine by running this code file twice one by one. Open command line and point to this code. Run from command line: php index.php . Then again run from command line php index.php.

  include_once("database.class.php");
  $mdb = new databaseAPI();

  $query = "SELECT * from status LIMIT 0,1000";
  $resultset = $mdb->getData($query);
  $mdb->close();

  echo "
";
  print_r($resultset);
  echo "

";

If everything is working fine, you will see a log.txt file being generated which will look as follows.

log.txt

[[2009-01-18 09:52:57]] New Instance Created
[[2009-01-18 09:52:57]] Connection established with memcache server
[[2009-01-18 09:52:57]] Query SELECT * from status LIMIT 0,1000 was not found cached in memcache server
[[2009-01-18 09:52:57]] Query SELECT * from status LIMIT 0,1000 was cached
[[2009-01-18 09:52:57]] Connection closed with memcache server
=================================================================================================
[[2009-01-18 09:53:08]] New Instance Created
[[2009-01-18 09:53:08]] Connection established with memcache server
[[2009-01-18 09:53:08]] Query SELECT * from status LIMIT 0,1000 was found already cached
[[2009-01-18 09:53:08]] Connection closed with memcache server
=================================================================================================

From the log file we can see that the 1st time results were fetched from the database and for the second time from memcached 🙂

Before we proceed further lets explain the flow of the above scripts. In index.php we create a new instance of database.class.php $mdb. Then we try to $query for 100 rows from the database. $mdb->getData($query) initiates this database fetch. As the program control goes to getData() method of database.class.php, it passed the control to getCache() method of memcache.class.php. There the code create a $key = base64_encode($query) and checks if we have the result set cached in memcached. If it doesn’t exists, it passed control back to getData() of database.class.php which fetches it from the database. After the fetch, it passes the $resultset back to setCache() method of memcache.class.php. There the setCache() method serialize the $resultset and cache it as ($key,$value) = (base64_encode($query), serialize($resultset)) in memcache.

Next time when the same query is fired and control goes to getCache() method of memcache.class.php, it fetches the result from cache, unserialize it and returns back the result to getData() of database.class.php. And thats why you see a log similar to above.

memcache.extendedstats.php
Finally it’s time to see some statistics. Here is a simple file which will show memcache status:

  $memcache_obj = new Memcache;
  $memcache_obj->addServer('localhost', 11211);

  $stats = $memcache_obj->getExtendedStats();

  echo "
";
  print_r($stats);
  echo "

";

Running it from command line using: php memcache.extendedstats.php will give you a statistic array like this.

Array
(
    [localhost:11211] => Array
        (
            [pid] => 5472
            [uptime] => 17
            [time] => 1232303504
            [version] => 1.2.5
            [pointer_size] => 32
            [curr_items] => 1
            [total_items] => 1
            [bytes] => 271631
            [curr_connections] => 2
            [total_connections] => 5
            [connection_structures] => 3
            [cmd_get] => 2
            [cmd_set] => 1
            [get_hits] => 1
            [get_misses] => 1
            [evictions] => 0
            [bytes_read] => 271705
            [bytes_written] => 271614
            [limit_maxbytes] => 536870912
            [threads] => 1
        )

)

This array tells you of a number of things about how your memcached deamon and caching architecture is performing. In short here is what each of the variable would mean:

  1. pid: Process id of this server process
  2. uptime: Number of seconds this server has been running
  3. time: Current UNIX time according to the server
  4. version: Version string of this server
  5. rusage_user: Accumulated user time for this process
  6. rusage_system: Accumulated system time for this process
  7. curr_items: Current number of items stored by the server
  8. total_items: Total number of items stored by this server ever since it started
  9. bytes: Current number of bytes used by this server to store items
  10. curr_connections: Number of open connections
  11. total_connections: Total number of connections opened since the server started running
  12. connection_structures: Number of connection structures allocated by the server
  13. cmd_get: Cumulative number of retrieval requests
  14. cmd_set: Cumulative number of storage requests
  15. get_hits: Number of keys that have been requested and found present
  16. get_misses: Number of items that have been requested and not found
  17. bytes_read: Total number of bytes read by this server from network
  18. bytes_written: Total number of bytes sent by this server to network
  19. limit_maxbytes: Number of bytes this server is allowed to use for storage.

However at this stage the figures which you might be interested in knowing are get_hits and get_misses. get_misses means number of times you requested for a key in memcache and it was not found while get_hits means number of times your requested key was successfully retrieved from memcached. Hence as expected we currently have get_misses = 1 and get_hits = 1. Try running php index.php once more and get_hits will get incremented by one.

N things: What else can I do
Till now you have memcached deamon running on your system and you know how to communicate with the deamon. You also know a very basic usage of memcached now, i.e. cache the queries to reduce load from your database. However there is a lot more you can do with memcached.

Here I would like to present before you a few applications of memcached which I learnt from my experiences. I hope they will be enough to make you think otherwise.

  1. Restricting spammers on your site : Often you will find on social networking sites like Digg, Facebook and Orkut that if you try to add several users as your friend within a short span, they (facebook) will show you a warning or they (digg) will restrict you from doing it. Similarly there are cases when you want to send a shout to more than 200 users on digg and you are restricted from doing so. How will you implement this on your site?

    Ordinary user’s solution: One solution is if a user ‘X’ add another user ‘Y’ as a friend, you will check how many friends has ‘X’ added in the past 10 minutes. If that exceeds 20, you won’t allow him to add more friends or show him a warning message. Simple enough and will work coolly. But what if your site have more than a million users, with many hackers around the world trying to keep your servers busy. As a memcached user you must have following solution in your mind:

    Your solution: As ‘X’ adds ‘Y’ in his friend list, you will set a (key,value) pair in memcached, where $key = “user_x” with $TTL = 10 minutes. For the first friend added, $value = 1. Now as ‘X’ tries to add another friend, your simply increment $value for $key = “user_x”. As this $value equals 20, and ‘X’ tried to add another friend, your code will check the value of $key = “user_x” and see if it’s present in memcached. If it’s present check for it’s value. If it’s value is equal to 20, you show a warning message to the user. Hence restricting him from adding more than 20 friends within a time span of 10 minutes. After 10 minutes, $key = “user_x” will automatically expires and your code will allow ‘X’ to add more friends. Similar solution exists if you want to stop spammers from sending message or commenting over a limit on your site. Now I see confidence building in you as a memcached programmer 😀

  2. Detecting online active/inactive users : Often you want a method which can tell your registered users, about their online friends. Or in a forum you want to tell the current reading user, about who all are reading this post. I won’t tell how an ordinary user will implement this, but as a memcached user your solution should be:

    Ordinary user’s solution: You don’t want to think of this.

    Your solution: As user ‘X’ visit post#12345 in the forum, not only you will fetch post data from the database, you also set two (key,value) pairs.

    $key1 = “post_12345”
    $value1 = [[comma separated list of user names]]
    $TTL1 = [[some arbitrary large value is fine]]

    $key2 = “user_x”
    $value2 = “post_12345”
    $TTL2 = 10 minutes, We assume a user to be inactive if he is on a particular post for more than 10 minutes (tunable), and we will mark him as inactive.

    (key1,value1) is a post specific data while (key2,value2) is a user specific data. Now every time a user visits post#12345, you will do two things. Read all comma separated user names from $value1, and then check for their corresponding $key2 value. If corresponding $key2 exists and equals to $value2 = “post_12345”, i.e. on the same post and not idle, we will keep that user name in value of $key1. However if $key2 is not present (i.e. user gone away) or value of $key2 equals to some other post, we will remove his user name from $value1. Confused 😛 , read twice and the picture will be clear.

    Can you think of a better solution? Please let me and others know about it. (Remember we are trying to detect only active/inactive users, which is not same as online/offline users)

  3. Building scalable web services : Another application of memcached lies in building scalable web services and web widgets. Gtalkbots offer a cool widget which you can put on your blog and sites to show off your aggregated status messages. See this widget live in the right hand sidebar. While building this widget, one thing which I kept in mind was that, what if someone with million hits per day put my widget on his site. Even though, Gtalkbots gets a few thousand of hits per day, it will crash, mainly because of this particular widget being placed on a high traffic site. So as a memcached user I have done the following thing.

    Ordinary user’s solution: Deprecated

    Your solution: I simply cache the widget data in memcache with $TTL = 1 hour. So every time this million hits per day site is accessed, which loads my widget million times a day, the query will be returned from cache. And hence saving my server from crashing. Fetch Gtalkbots widget from here and try putting on your site.

Alright by now you can impress your bosses with your cool memcache implementations. But wait there is a lot more you need to know. I can go on putting hundred’s of memcached applications here, but main point is, setting your mind as a memcached user. I personally have this habit of linking everything to memcached while designing a system, and if it suits my need, Bingo!.

  1. Versioning your cache keys : One disadvantage of using cache at times is that, if unfortunately something goes wrong with your data and that buggy data is cached, your users will keep seeing that buggy data until your cache expires or you manually clear off your cache. Suppose you clear your cache and then one of your fucking engineer comes running saying the data is fine.

    Ordinary user’s solution: Stop it, No more plz

    Your solution: As a memcached user, i would love to keep a Versioning system for my caches. Nothing complex, simply append “v1” (version number) to all your keys. i.e. $key = “user_x” will now become $key = “v1:user_x”. and at a place in your code you have $current_cache_version = “v1”. Now suppose you are told your data is buggy, so by the time your engineers are investigation change $current_cache_version = “v2”. This will keep your old caches, which you may want to recover after your investigation and at the same time show new data to your users.

  2. Not so frequent update of Db for trivial data : This is a site dependent application. Suppose you run a site where you are not so serious about database columns like “last_update_date”, “last_logged_in” and so on. However you still keep a track of such for analysis purpose and don’t mind if it’s not so accurate.

    Your solution: One simple solution to this problem is keep such trivial data in memcached and set up a cron job which will run every 10 minutes and populate such data in the database. 🙂

I will leave you with a presentation on memcache which I gave sometime back at office. I hope it will help you gain more understanding of memcache.

Memcache

View SlideShare presentation or Upload your own. (tags: caching memcache)

I hope after reading this you are well equipped on how to design scalable systems. All the best! , do leave a comment or suggestions if any. If you liked this post, do subscribe for the next post in memcache series.

Link to new facebook design: See it building everyday

Hello Everyone, I don’t exactly remember from where I came across this link. The link shows you the latest development on new look and feel of facebook. Click the link below which will take you to the new facebook’s development page.

New Facebook Design Development Link

What’s new in the new design?

  • Well Facebook was always Ajax dominated social networking site, but now it goes a step further. It adapts philosophy from iGoogle and my.Yahoo and provides its users to add new tabs at the top of your profile page.
  • By DEFAULT it comes with Wall, Info, Photos and Boxes tabs at the top. Further you can add the following new tabs namely Notes, Videos, My Stuff and Music iLike. Basically it changes from user to user.
  • More Ajax means less loading time for your page. Now they allows you to switch between various aspects without having to reload the whole page.

So surely world is moving towards customized interface for its users. With iGoogle, my.Yahoo now Facebook trying the same. Will Orkut too see a change yet again.

Enjoy and see it changing everyday. I have been following it since 3-4 days and I see a complete new face every time I login to the new link. 🙂