How to perform X-FACEBOOK-PLATFORM and Google Talk X-OAUTH2 XMPP authentication with PHP Jaxl library

Ever since Jaxl library first introduced support for X-FACEBOOK-PLATFORM XMPP authentication mechanism, it has changed significantly. Also, Google Talk now supports OAuth 2.0 Authorization, an XMPP extension to allow users to log in using OAuth 2.0 credentials.

Both these mechanisms are a big win for XMPP developers, since real-time conversation experience can now be provided to their application users without asking them for their passwords. In this blog post, I will demonstrate how to perform X-FACEBOOK-PLATFORM and X-OAUTH2 XMPP authentication mechanism using Jaxl v3.x PHP Library.

X-FACEBOOK-PLATFORM XMPP Authentication
Here is a quick guide on how to perform X-FACEBOOK-PLATFORM XMPP authentication using xfacebook_platform_client.php which comes bundled with Jaxl v3.x examples:

  • Visit Facebook Developer Apps page and register your application
  • Once registered, visit access token tool to get required parameters to perform X-FACEBOOK-PLATFORM authentication Facebook Access Token Tool
  • Click on the debug button next to User Token and make sure xmpp_login is one of the extended permissions (scope)
  • Enter downloaded Jaxl library folder and run from command line as follows:

    $ php examples/xfacebook_platform_client.php fb_user_id_or_username fb_app_key fb_access_token

You can now take the source code of xfacebook_platform_client.php and customize it for your application needs.

Google Talk X-OAUTH2 XMPP Authentication
Here is a quick guide on how to perform Google Talk X-OAUTH2 XMPP authentication using xoauth2_gtalk_client.php which comes bundled with Jaxl v3.x examples:

  • Visit Google OAuth Playground and input https://www.googleapis.com/auth/googletalk as the required scope. Press “Authorize API” and then “Allow Access” button on the redirected page
  • In step 2, simply press “Exchange authorize code for tokens” and copy the access token
  • Enter downloaded Jaxl library folder and run from command line as follows:

    $ php examples/xoauth2_gtalk_client.php username@gmail.com access_token

You can now take the source code of xoauth2_gtalk_client.php and customize it for your application needs.

Wasn’t that simple :)

JAXLXml – Strophe style XML Builder : Working with Jaxl – A Networking Library in PHP – Part 2

Prior to Jaxl v3.x, the most ugliest piece of code inside Jaxl library was handling of XML packets. If you are working with XMPP protocol which is all about sending and receiving XML packets, it can become a nightmare if you don’t have a proper XML manipulation library in your toolkit. For Jaxl v3.x, first thing I decided to write was JAXLXml class, which is a custom XML packet implementation with no external dependencies and is an extension over the ideas from Strophe.Builder class written by Jack Moffitt.

JAXLXml is generic enough to find a place inside any PHP application that requires easy and elegant XML packet creation. In this blog post, I will give an exhaustive overview of how to create XML packets using JAXLXml class.

JAXLXml Constructor
Depending upon the need, there are several different ways of initializing a JAXLXml object:

  • $xml_obj = new JAXLXml($name, $ns, $attrs, $text);
  • $xml_obj = new JAXLXml($name, $ns, $attrs);
  • $xml_obj = new JAXLXml($name, $ns, $text);
  • $xml_obj = new JAXLXml($name, $attrs, $text);
  • $xml_obj = new JAXLXml($name, $attrs);
  • $xml_obj = new JAXLXml($name, $ns);
  • $xml_obj = new JAXLXml($name);

where:

  • $name – the XML node name
  • $ns – the XML namespace
  • $attrs – Key-Value (KV) pair of XML attributes
  • $text – XML content

Here are a few examples for each constructor style shown above:

$ ./jaxlctl shell
jaxl 1> $name = 'node-name';
jaxl 2> $ns = 'my:xml:ns';
jaxl 3> $attrs = array('k1'=>'v1', 'k2'=>'v2');
jaxl 4> $text = 'this is an example';
jaxl 5>
jaxl 5> $xml_obj = new JAXLXml($name, $ns, $attrs, $text);
jaxl 6> echo $xml_obj->to_string();
<node-name xmlns="my:xml:ns" k1="v1" k2="v2">this is an example</node-name>
jaxl 7> 
jaxl 7> $xml_obj = new JAXLXml($name, $ns, $attrs);
jaxl 8> echo $xml_obj->to_string();
<node-name xmlns="my:xml:ns" k1="v1" k2="v2"></node-name>
jaxl 9> 
jaxl 9> $xml_obj = new JAXLXml($name, $ns, $text); 
jaxl 10> echo $xml_obj->to_string();
<node-name xmlns="my:xml:ns">this is an example</node-name>
jaxl 11> 
jaxl 11> $xml_obj = new JAXLXml($name, $attrs, $text);
jaxl 12> echo $xml_obj->to_string();
<node-name k1="v1" k2="v2">this is an example</node-name>
jaxl 13>
jaxl 13> $xml_obj = new JAXLXml($name, $attrs);
jaxl 14> echo $xml_obj->to_string();
<node-name k1="v1" k2="v2"></node-name>
jaxl 15>
jaxl 15> $xml_obj = new JAXLXml($name, $ns);
jaxl 16> echo $xml_obj->to_string();
<node-name xmlns="my:xml:ns"></node-name>
jaxl 17> 
jaxl 17> $xml_obj = new JAXLXml($name);
jaxl 18> echo $xml_obj->to_string();
<node-name></node-name>
jaxl 19>

JAXLXml will sanitize attributes and text values as shown below:

$ ./jaxlctl shell
jaxl 1> $xml_obj = new JAXLXml('msg', array(), '<a href=""></a>');
jaxl 2> echo $xml_obj->to_string();
<msg>&lt;a href=&quot;&quot;&gt;&lt;/a&gt;</msg>
jaxl 3>
jaxl 3> $xml_obj = new JAXLXml('msg', array('a'=>'some < tag data >'), '<a href="javascript:void(0);"></a>');
jaxl 4> echo $xml_obj->to_string();
<msg a="some &lt; tag data &gt;">&lt;a href=&quot;javascript:void(0);&quot;&gt;&lt;/a&gt;</msg>
jaxl 5>
jaxl 5> quit
$

Manipulating Attributes, Child Nodes and Content
Below is an exhaustive list of methods available over initialized JAXLXml object $xml_obj for manipulating attributes, child nodes and content:

  • c($name, $ns=null, $attrs=array(), $text=null) : Append a child node at current rover and update the rover to point at newly added child node. Rover is nothing but a pointer indicating the level in the XML tree where this and other methods will perform. When an JAXLXml instance is initialized, rover points to the top level node.
  • cnode($node) : Append a child node given by $node (a JAXLXml object) at current rover and update the rover to point at newly added child node.
  • t($text, $append=FALSE) : Update text of the node pointed by current rover
  • top() : Move rover back to the top in the XML tree
  • up() : Move rover one step up the XML tree
  • attrs($attrs) : Merge new attributes specified as KV pair $attrs with existing attributes at the current rover.
  • match_attrs($attrs) : Accepts a KV pair of attributes $attrs, return bool if all keys exist and have same value as specified in the passed KV pair.
  • exists($name, $ns=null, $attrs=array()) : Checks if a child with $name exist. If found, return matching child as JAXLXml object otherwise false. If multiple children exist with same name, this function will return on first matching child
  • update($name, $ns=null, $attrs=array(), $text=null) : Update $ns, $attrs and $text (all at once) of an existing child node $name
  • to_string($parent_ns=null) : Return string representation of JAXLXml object

Method Chaining
The best thing one will find while working with JAXLXml class is that all the above methods are chain-able i.e. Any complex XML structure can be built with a single line of code.

Here is an example building a fairly nested XML structure in a single line of code:

$ ./jaxlctl shell
jaxl 1> $xml_obj = new JAXLXml('message', array('to'=>'1@a.z', 'from'=>'2@b.c'));
jaxl 2> $xml_obj->c('body')->attrs(array('xml:lang'=>'en'))->t('Hello World!')->up()
....... ->c('thread')->t('id-1234')->up()
....... ->c('nested-stuff')
....... ->c('nest')->t('nest1')->up()
....... ->c('nest')->t('nest2')->up()
....... ->c('nest')->t('nest3')->up()->up()
....... ->c('c')->attrs(array('hash'=>'84jsdmnskd'));
jaxl 3> echo $xml_obj->to_string();
'<message to="1@a.z" from="2@b.c"><body xml:lang="en">hello</body><thread>1234</thread><nested><nest>nest1</nest><nest>nest2</nest><nest>nest3</nest></nested><c hash="84jsdmnskd"></c></message>'
jaxl 4>
jaxl 4> quit
$

Working with Jaxl – A Networking Library in PHP – Part 1 – An Introduction, Philosophy and History

Development of Jaxl library started way back in December’07 while I was working on a self-initiated project called Gtalkbots. The project is now dead, if you are interested in knowing more about it go through Gtalkbots BlogSpot. Jaxl v1.x was first released in Jan’09 and about a year later in Aug’10 Jaxl v2.x was released. First two versions were released as JAbber XMPP Library for writing clients and external server components.

While working on my startup Jaxl – A Platform As A Service (PAAS) for developing real-time applications, I started experiencing v2.x limitations when my external server side components were unable to process XMPP packets at the speed they were sent by ejabberd server. I started restructuring and refactoring the library which gave birth to Jaxl v3.x. Since v3.x was initially being used for developing the entire infrastructure, it shaped up as a networking library in PHP with stable support for XMPP protocol. However, later I had to rewrite several infrastructure components in Erlang Programming Language due to several issues that PHP as a language couldn’t solve (after all PHP wasn’t made for such tasks). Finally in April’12, Jaxl v3.x was open sourced.

Jaxl v3.x is an asynchronous, non-blocking, event based networking library in PHP for writing custom TCP/IP client and server implementations. From previous versions, Jaxl library inherits a full blown stable support for XMPP protocol stack. In v3.0, support for HTTP protocol stack was also introduced. At the heart of every protocol stack sits a Core stack. It contains all the building blocks for everything that we aim to do with Jaxl library. Both XMPP and HTTP protocol stacks are written on top of the Core stack. Infact the source code of these protocol implementations knows nothing about the standard (inbuilt) PHP socket and stream methods.

Philosophy
Jaxl is designed to work asynchronously in a non-blocking fashion and provides an event based callback API. Now what does all that mean?

By non-blocking and asynchronous it means, when a library function like:
$jaxl->send($stanza); is called, it will return immediately i.e. this function call will NOT block any further execution of your application script until $stanza has actually been sent over the connected TCP socket. Infact, when this function is called, passed $stanza object is put into an output buffer queue, which will be flushed as and when underlying TCP socket is available for writes. Similarly, most of the available methods (wherever required and possible) inside Jaxl library are non-blocking and asynchronous in nature.

By event based callback API it means, application code will need to register/add callbacks over necessary events as they occur inside Jaxl instance lifecycle. A list of available event callbacks with some explanation can be found here. For example, most of the XMPP applications will usually register a callback over on_auth_success event. As and when this event occurs inside Jaxl instance lifecycle, registered function will be callback’d with necessary parameters (if any).

Related Links

  • Read library documentation
  • Download the latest and greatest source from GitHub.
  • Have any Question? Want to discuss? Need Help? Use Google Group/Forum.
  • Found something missing or a bug in the source code? Kindly report an issue.
  • Fixed a bug? Want to submit a patch? Want to improve documentation? Checkout source code and contribute to the library

XMPP Application Examples

HTTP Application Examples

Stay Tuned
In coming weeks, under this series of blog posts titled “Working with Jaxl – A Networking Library in PHP”, I will cover following major topics with sample code:

  • Explanation of each Core stack class and how to use them
  • Design of each XMPP and HTTP stack class
  • XMPP over HTTP
  • XMPP File Transfer and Multimedia Sessions
  • Understanding and Using External Jabber Components
  • Asynchronous Job/Task Queues
  • Developing Concurrent and Parallel Systems

If you have any specific topic that you would like me to be cover, kindly let me know via your comments here.

Announcing Jaxl v3.x – asynchronous, non-blocking I/O, event based PHP client/server library

Jaxl v3.x is a successor of v2.x (and is NOT backward compatible), carrying a lot of code from v2.x while throwing away the ugly parts. A lot of components have been re-written keeping in mind the feedback from the developer community over the last 4 years. Also Jaxl shares a few philosophies from my experience with erlang and python languages.

Jaxl is an asynchronous, non-blocking I/O, event based PHP library for writing custom TCP/IP client and server implementations. From it’s previous versions, library inherits a full blown stable support for XMPP protocol stack. In v3.0, support for HTTP protocol stack was also added.

At the heart of every protocol stack sits a Core stack. It contains all the building blocks for everything that we aim to do with Jaxl library. Both XMPP and HTTP protocol stacks are written on top of the Core stack. Infact the source code of protocol implementations knows nothing about the standard (inbuilt) PHP socket and stream methods.

Source code on GitHub

Examples

Documentation

Group and Mailing List

Create a bug/issue

Read why v3.x was written and what traffic it has served in the past.

How to Write a Spelling Corrector in Erlang (ESpell)

Erlang is a beautiful programming language from Ericsson which i first came across while cutomizing authentication flow of ejabberd about 2 years back. Ever since then I have been using erlang for all my application backend needs including custom http server, custom bosh conn. manager, xmpp components and clients, … Recently i have even started churning my application html pages via erlang using erlydtl (an Erlang implementation of the Django Template Language).

Years ago, i gave a successful shot at implementing Peter Norvig’s Spell Corrector in PHP. Last weekend i attempted the same “Spell Corrector” algorithm in about 45 lines of Erlang code.

ESpell:

Complete code file with comments and explaination can be found here:
https://github.com/abhinavsingh/espell

-module(espell).
-define(alphabet, "abcdefghijklmnopqrstuvwxyz").
-export([start/0, correct/1]).

%%
%% API Functions
%%

%% @doc start spell checker
start() -> train(words()).

%% @doc returns most probable correct candidate with score
correct(Word) ->
	lists:foldl(
		fun(Candidate, {Correction, Score}) -> 
			case ets:lookup(?MODULE, list_to_binary(Candidate)) of
				[{_, Counter}] when Counter > Score -> {Candidate, Counter};
				_ -> {Correction, Score}
			end
		end, {Word, 0}, get_candidates(Word)).

%%
%% Local Functions
%%

words() ->
	{ok, Bin} = file:read_file("../priv/big.txt"),
	{ok, Words} = regexp:split(binary_to_list(Bin), "[^a-zA-Z]"),
	lists:map(fun(X) -> string:to_lower(X) end, Words).

train(Features) ->
	io:fwrite("training initial word list...~n"),
	ets:new(?MODULE, [set, named_table]),
	lists:foreach(fun(X) ->
		case ets:insert_new(?MODULE, {list_to_binary(X), 1}) of
			false -> ets:update_counter(?MODULE, list_to_binary(X), 1);
			true -> true
		end
	end, Features),
	io:fwrite("training complete...~n"),
	ok.

edits1(Word) ->
	Splits = lists:foldl(fun(I, Acc) -> Acc ++ [{string:substr(Word, 1, I), string:substr(Word, I+1, string:len(Word)-I)}] end, [{"", Word}], lists:seq(1, string:len(Word))),
	Deletes = [A ++ string:substr(B, 2) || {A,B} <- Splits, B =/= []],
	Transposes = [A ++ string:substr(B, 2, 1) ++ string:substr(B, 1, 1) ++ string:substr(B, 3) || {A,B} <- Splits, string:len(B) > 1],
	Replaces = [A ++ binary_to_list(<>) ++ string:substr(B, 2) || {A,B} <- Splits, B =/= [], C <- ?alphabet],
	Inserts = [A ++ binary_to_list(<>) ++ B || {A,B} <- Splits, C <- ?alphabet],
	lists:usort(Deletes ++ Transposes ++ Replaces ++ Inserts).

%%edits2(Word) -> lists:usort([E2 || E1 <- edits1(Word), E2 <- edits1(E1)]).
known_edits2(Word) -> lists:usort([E2 || E1 <- edits1(Word), E2 <- edits1(E1), ets:member(?MODULE, list_to_binary(E2))]).
known(Words) -> lists:usort([Word || Word <- Words, ets:member(?MODULE, list_to_binary(Word))]).

get_candidates(Word) ->
	C1 = known([Word]),
	if 
		length(C1) > 0 -> C1;
		true -> C2 = known(edits1(Word)),
			if
				length(C2) > 0 -> C2;
				true ->	C3 = known_edits2(Word),
					if length(C3) > 0 -> C3; 
					true -> [Word] end
			end
	end.

Try It Out:
espell provides 2 simple function for all it’s working:

  • start() : start espell which initiates reading initial data and training phase
  • correct(Word) : this accepts 1 parameter, which is the word you want to correct. It returns a 2-tuple, where 1st element is the correct word and 2nd element is a score (which right now simply means number of times correct word was seen in training data set)
$ cd espell
$ erlc -o ebin/ src/espell.erl
$ erl -pa ebin/
Erlang R14B03 (erts-5.8.4) [source] [smp:4:4] [rq:4] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.8.4  (abort with ^G)
1> espell:start().
training initial word list...
training complete...
ok
2> espell:correct("speling").
{"spelling",4}
5> espell:correct("somthing").
{"something",683}

This code makes extensive use of list comprehensions in erlang, which is hugely responsible for cutting down espell code to just 45 lines of erlang.

JAXL library – List of available hooks for various XMPP events

Jaxl 2.x provides an event mechanism using which developers can register callbacks for various xmpp events inside their application code. This blog post will demonstrate how to register callbacks for required xmpp events and go through a list of all available hooks. Finally, we will discuss parameters that are passed to called back methods by Jaxl core.

Registering callback on XMPP events
Applications can register callback for various XMPP events. Jaxl core will then callback application methods (with 2 parameters) every time associated XMPP event occurs. Shown below are some sample examples for registering callbacks.

When application callback’d method is a function:

function postAuth($payload, $jaxl) {

}
$jaxl->addPlugin('jaxl_post_auth', 'postAuth');

When application callback’d method is a public static method of a class:

class MyXMPPApp {
    public static function postAuth($payload, $jaxl) {

    }
}
$jaxl->addPlugin('jaxl_post_auth', array('MyXMPPApp', 'postAuth'));

When application callback’d method is a public method inside a class:

class MyXMPPApp {
    function postAuth($payload, $jaxl) {

    }
}
$MyXMPPApp = new MyXMPPApp();
$jaxl->addPlugin('jaxl_post_auth', array($MyXMPPApp, 'postAuth'));

In all the above examples jaxl_post_auth is one of the available hook for registering callbacks.

List of available hooks
Below is a complete list of available hooks in order of their occurrence within a Jaxl instance life cycle:

Hooks for events related to instance connection and authentication steps in various modes:

  • jaxl_post_connect
  • jaxl_get_auth_mech
  • jaxl_get_facebook_key
  • jaxl_post_auth_failure
  • jaxl_post_auth
  • jaxl_post_handshake
  • jaxl_pre_shutdown
  • jaxl_post_disconnect
  • jaxl_get_empty_body

Hooks for events related to XMPP stream and stanza’s:

  • jaxl_get_stream_error
  • jaxl_get_presence
  • jaxl_get_message
  • jaxl_get_iq_get
  • jaxl_get_iq_set
  • jaxl_get_iq_error
  • jaxl_send_message
  • jaxl_send_presence

Hooks for events related to reading/writing of XMPP packets and internal packet routing:

  • jaxl_get_xml
  • jaxl_send_xml
  • jaxl_send_body
  • jaxl_pre_handler
  • jaxl_post_handler

TO-DO: Update when every hook is called inside your application life cycle and list of parameters passed for each callback. As of now you can var_dump($payload); inside your callback method.

How to write External Jabber Components in PHP using Jaxl library?

Jabber Component Protocol (XEP-0114) documents how XMPP protocol can be used to communicate between servers and “external” components over the Jabber network. XMPP components “bind” to a domain, usually a sub-domain of the main XMPP service, such as service.example.org.

All incoming stanzas addressed to that domain (to='service.example.org') or to entities on that domain (to='user@service.example.org') will be routed to your Jaxl (Jabber XMPP Library) based code. In this blog post, I will demonstrate a sample external jabber component bot written in PHP using Jaxl library.

Refer Jaxl Installation, Usage guide and Example apps if you are new to Jaxl. Demonstrated component bot code can be obtained from Jaxl@github.

Using Jabber Component Protocol
Include Jaxl implementation of XEP-0114 in your application code to setup necessary environment for using Jabber component protocol. Here is how this can be done at the top of your application code:

        // Initialize Jaxl Library
        $jaxl = new JAXL(array(
                'component' => JAXL_COMPONENT_HOST,
                'port' => JAXL_COMPONENT_PORT
        ));

        // Include required XEP's
        jaxl_require('JAXL0114', $jaxl); // Jabber Component Protocol

Register callback for XMPP events
Above we have setup the necessary environment for writing external Jabber component bots. Next we register callback for necessary XMPP events inside our componentbot class.

        // Sample Component class
        class componentbot {

        }

        // Add callbacks on various event handlers
        $componentbot = new componentbot();
        JAXLPlugin::add('jaxl_pre_handshake', array($componentbot, 'doAuth'));
        JAXLPlugin::add('jaxl_post_handshake', array($componentbot, 'postAuth'));
        JAXLPlugin::add('jaxl_get_message', array($componentbot, 'getMessage'));

Component bot class
Finally, lets complete the missing pieces inside componentbot class.

        // Sample Component class
        class componentbot {

                function doAuth() {
                        $jaxl->log("Going for component handshake ...", 1);
                        return JAXL_COMPONENT_PASS;
                }

                function postAuth() {
                        $jaxl->log("Component handshake completed ...", 1);
                }

                function getMessage($payloads) {
                        global $jaxl;

                        // echo back
                        foreach($payloads as $payload) {
                                $jaxl->sendMessage($payload['from'], $payload['body'], $payload['to']);
                        }
                }

        }

Configure, Setup and Run
If you have a local “ejabberd” installed, add following lines inside ejabberd.cfg to make example component bot to work:

  {5559, ejabberd_service, [
                          {host, "component.localhost", [{password, "pass"}]}
                           ]},

Update jaxl.ini if you choose to have different password, port or host name above:

        // Connecting jabber server details
        define('JAXL_HOST_NAME', 'localhost');
        define('JAXL_HOST_DOMAIN', 'localhost');

        // Component bot setting
        define('JAXL_COMPONENT_HOST', 'component.'.JAXL_HOST_DOMAIN);
        define('JAXL_COMPONENT_PASS', 'pass');
        define('JAXL_COMPONENT_PORT', 5559);

Finally, run from command line:

root@ubuntu:~/usr/share/php/jaxl/app/componentbot# jaxl componentbot.php
[15008] 2010-08-24 01:40:03 - Socket opened to the jabber host localhost:5559 ...

Tail jaxl.log for details:

[15008] 2010-08-24 01:40:04 - Going for component handshake ...

[15008] 2010-08-24 01:40:04 - [[XMPPSend]] 63
<handshake>4d6c2e762d5ba5dca2cbd3a90a4deeb6a6fa0838</handshake>

[15008] 2010-08-24 01:40:05 - [[XMPPGet]]
<handshake/>

[15008] 2010-08-24 01:40:05 - Component handshake completed ...

Log into your Ejabberd with a client and send a message to anything@component.localhost – You should receive an instant response back – congratulations!