Open Source

Customizing Redis pubsub for message persistence – Part 2

Posted on

In the last post we saw how Redis can easily be modified to persist the last published message on PubSub channels. Without subscribing to the PubSub channel we were able to get the last published message from Redis db. In this post, I will take that idea one step ahead and add native capabilities within Redis to persist all the unprocessed messages published on PubSub channel in channel specific lists. We’ll also preserve our capability to send the last published message to clients upon subscription. But why are we doing this? Popular open source application that provide support for Redis are […]

Open Source

Back to blogging: What to expect

Posted on

Hello Readers, I started this blog as a way to share my experiments and experiences while learning web development and computer science in general. In the first 2 years (between Apr’08 and Aug’10) I wrote as many as 100 blog posts. Quite a frenzy. Ever since, I only managed to write 5-6 posts in the following 4 years, about nearly 45 drafts which may now never get published. Good thing is that, I am back to blogging, which means a lot to share. Briefly, here is what (or what not) to expect in the future posts: PHP – In past, PHP has dominated the […]

PHP

MEMQ : Fast queue implementation using Memcached and PHP only

Posted on

Memcached is a scalable caching solution developed by Danga interactive. One can do a lot of cool things using memcached including spam control, online-offline detection of users, building scalable web services. In this post, I will demonstrate and explain how to implement fast scalable queues in PHP. MEMQ: Overview Every queue is uniquely identified by it’s name. Let’s consider a queue named “foo” and see how MEMQ will implement it inside memcached: Two keys namely, foo_head and foo_tail contains meta information about the queue While queuing, item is saved in key foo_1234, where 1234 is the current value of key […]

Scalability

WordPress style “Duplicate comment detected” using Memcached and PHP

Posted on

If you have a knack of leaving comments on blogs, chances are you might have experienced a wordpress error page saying “Duplicate comment detected; it looks as though you’ve already said that!“, probably because you were not sure that your comment was saved last time and you tried to re-post your comment. In this blog post, I will put up some sample PHP code for Duplicate comment detection using Memcached without touching the databases. Towards the end, I will also discuss how the script can be modified for usage in any environment including forums and social networking websites. Duplicate comment […]

Scalability

How to build a custom static file serving HTTP server using Libevent in C

Posted on

Libevent is an event notification library which lays the foundation for immensely successful open source projects like Memcached. As the web advances into a real time mode, more and more websites are using a mix of technologies like HTTP Pub-Sub, HTTP Long-polling and Comet with a custom light weight HTTP servers in the backend to create a real time user experience. In this blog post, I will start with necessary prerequisites for setting up the development environment. Further, I will demonstrate how to build a HTTP server capable of serving static pages. Finally, I will put up a few use […]

Scalability

How to use locks for assuring atomic operation in Memcached?

Posted on

Memcached provide atomic increment and decrement commands to manipulate integer (key,value) pairs. However special care should be taken to ensure application performance and possible race conditions while using memcached. In this blog post, I will first build a facebook style “like” application using atomic increment command of memcached. Also, I will discuss various technical difficulty one would face while ensuring atomicity in this application. Finally, I will demo how to ensure atomicity over a requested process using custom locks in memcached. Where should I care about it? Lets consider a sample application as depicted by the flow diagram below: The […]

Scalability

Memcached and “N” things you can do with it – Part 1

Posted on

In my last post MySQL Query Cache, WP-Cache, APC, Memcache – What to choose, I discussed in brief about 4 caching technologies which you might have used knowingly or unknowingly. Towards the end we came to a conclusion that memcached is the best caching solution when you are looking for speed and number of hits per second. By my experience, memcached is capable of handling more than a 100 Million PV’s per month without any problem. However, towards the end I did also discussed why memcached is unreliable and unsecure. In this post I will dig a level deeper into […]

PHP

MySQL Query Cache, WP-Cache, APC, Memcache – What to choose

Posted on

Hello Cache Freaks, Ever since I changed my job (from Business Intelligence to Web development) and started working with my present employer, I have had a chance to work on a lot of scalable projects. From making my project to scale from 20 Million PV’s to 100 Million PV’s to development of an internal tool, the answer to all scalable applications have been caching. There are a lot of caching techniques which are being employed by sites worldwide. WP-Cache used in wordpress – a file system based caching mechanism APC Cache – an opcode based caching system Memcache – an […]