How to create an upload meter in PHP? Explained

It was about 2 months back when I started hunting for an upload meter “How To?” guide on net. Came across a lot of links actually but I must say none of them actually gave me what I wanted to have. Some talked of some patched PHP version to achieve the same and some talked everything except an upload meter.

Finally I was able to come up with the trick involved (thanks to sources on net and some work on my side). Here I will explain in short how can we achieve this on windows and unix both:

  1. APC (Alternate PHP Cache) : Before we go on to develop an upload meter we will need to have this component configured. Without APC its almost impossible to have an upload tracking script working on your server in PHP. Basically, when we upload a file to a server (windows or linux) the file first goes to the /tmp folder (DEFAULT) and then using the php’s move_uploaded_file($FileTemp,$FileUploadLocation) command we move that file from /tmp folder to the location we finally want it to be.

    Now the only problem is that, the file name in the /tmp is quite nondeterministic. There is no way to predict the file name in /tmp. Hence by simple methods we cannot track the status of upload for that particular file. We need a handler to that file, through which we can track its upload progress.

    Although APC is not meant for this, but it does provide us a method to bind a unique name to this uploading file. Lets first see how to install APC on windows and unix.

  2. Installing APC on windows : On windows the installation process is quite simple. Simply download this zipped folder from here. Extract it and you will find, folders corresponding to each PHP version released. Identify your PHP version, copy the corresponding DLL file from that folder and place it under C:/PHP/ext and C:/Windows (if you have your php.ini file placed there).

    Now open your php.ini file, and add the following lines:

    extension=php_apc.dll
    apc.shm_segments=1
    apc.optimization=0
    apc.shm_size=128
    apc.ttl=7200
    apc.user_ttl=7200
    apc.num_files_hint=1024
    apc.mmap_file_mask=/tmp/apc.XXXXXX
    apc.enable_cli=1
    apc.rfc1867=on
    apc.max_file_size=10M

    These lines must be placed after all the extension definition in the php.ini file. I won’t go into explaining the various parameters here, you can search for them and you will find plenty of documentation for that. As of now you can care only about apc.rfc1867=on (without this file upload meter won’t work, this tells APC to track file uploading) and apc.max_file_size=10M (this specifies max file size to cache which is otherwise 1Mb by default).

    Now simply restart your apache server, and see your php information i.e. phpinfo(). You must see a section APC in the phpinfo(). If you can see that section, Congratulations APC installation is done for you 🙂

  3. Installing APC on unix (Debian or Ubuntu) : I have done this on debian and ubuntu, but similar must be the process for other OS too. Simply install the following modules:
    • apt-get install apache2-dev,apache2-threaded-dev,php-pear,php5-dev,make
    • ln -s /usr/bin/apxs2 /usr/bin/apxs (APC won’t install without this symlink)
    • pecl install apc (input Yes if prompted)
    • Add extension=apc.so in php.ini
    • /etc/init.d/apache2 restart
    • pecl upgrade apc
    • /etc/init.d/apache2 restart
    • Add apc.rfc1867=on in php.ini for tracking file upload
  4. Coding your Upload meter (index.php): So we are almost there. Just a few lines of code and we have our upload tracker ready. Let us first create an html form.
    <html>
      <head>
        <script type=”text/javascript” src=”upload.js”></script>
      </head>
      <body>
        <form id=”upload” name=”upload” target=”frame” action=”utility.php” method=”POST” enctype=”multipart/form-data”>
         
          <!– Setting this enables apc upload tracking –>
          <input type=”hidden” name=”APC_UPLOAD_PROGRESS” id=”fileKey” value=”<?php echo md5(uniqid(rand(),true)); ?>”/>
         
          <!– Hidden field for the upload form –>
          <input type=”hidden” id=”uploadfile” name=”uploadfile” value=”uploadfile”/>
         
          <!– DIV containing the file chooser box, it will be replaced by the filename after upload –>
          <div id=”attachment”>
            <input id=”fileloc” name=”fileloc” size=”30″ type=”file” onchange=”getProgress();document.upload.submit();return false;”/>
          </div>
         
          <!– DIV showing the upload progress bar –>
          <div id=”showProgress” style=”margin-top:5px;width:205px;height:15px;border:1px solid #93B9D9;display:none”>
            <div id=”currentStatus” style=”width:0px;height:15px;background-color:#B2D281″></div>
          </div>
         
          <!– iframe handling all the upload –>
          <iframe name=”frame” style=”display:none”></iframe>
         
          <!– DIV which catches all the error and other messages –>
          <div id=”upload-note”></div>
        </form>
      </body>
    </html>

    The code is self explainatory 😛 , if not I have put in some comments in there. Here is a short explaination again:

    • upload.js file will handle the ajax calls later on and also will take care of incrementing the upload status bar.
    • <input type=”hidden” name=”APC_UPLOAD_PROGRESS” id=”fileKey” value=”<?php echo md5(uniqid(rand(),true)); ?>”/> , this is the factor which allows us to keep a track of our file. This must always be declared before the file uploading input box. Basically it tells APC to associate value=”[SPECIFIED_ABOVE]” with the uploading file. This values serves as a key and using this key we will later on track the file.
    • The form submits into an iframe. To understand more on this one read through one of my previous post on Gmail Type Attachment – How to make one? .
    • <div id=”attachment”></div> contains the file chooser input box, which submits the form as soon as it receives a file to upload.
    • id=”showProgress” and name=”frame” are self explanatory. I have used <div id=”upload-note”></div> to post upload error messages in case of an error.
  5. Coding your Upload meter (utility.php): utility.php is the file which handles the file uploading process. The form submits into this file through the iframe.
    <?php

      if((isset($_POST[‘uploadfile’])) && ($_POST[‘uploadfile’] == “uploadfile”)) {
       
        // Gather File data (name,size,mimetype,tmp_name)
        $FileName = $_FILES[‘fileloc’][‘name’];
        $FileSize = round($_FILES[‘fileloc’][‘size’]/1024);
        $FileType = $_FILES[‘fileloc’][‘type’];
        $FileTemp = $_FILES[‘fileloc’][‘tmp_name’];
       
       
        // Get file extension
        $extension = strtolower(substr($FileName,strrpos($FileName,’.’)+1));
       
        // If filename is blank, exit with a message
        if($FileName == “”) {
          echo “<script type=’text/javascript’>”;
          echo “parent.document.getElementById(‘upload-note’).innerHTML = ‘<font size=2 color=#990000>Kindly choose a file to upload</font>'”;
          echo “</script>”;
          exit;
        }
       
        // If filesize is > 10 Mb, exit with a message
        if($FileSize >= “10240”) {
          echo “<script type=’text/javascript’>”;
          echo “parent.document.getElementById(‘upload-note’).innerHTML = ‘<font size=2 color=#990000>OOPS! The file exceeds the maximum limit of 10 Mb</font>'”;
          echo “</script>”;
          exit;
        }
       
        // Exit if file’s extension is not jpg
        if($extension != “jpg”) {
          echo “<script type=’text/javascript’>”;
          echo “parent.document.getElementById(‘upload-note’).innerHTML = ‘<font size=2 color=#990000>OOPS! Only jpg file formats are supported</font>'”;
          echo “</script>”;
          exit;
        }
       
        // Choose a final upload location
        $FileUploadLocation = “upload/”.$FileName.”.”.strtolower($extension);
       
        // Move the uploaded file from temp to final upload location
        if(move_uploaded_file($FileTemp,$FileUploadLocation)) {
          echo “<script type=’text/javascript’>”;
          echo “parent.document.getElementById(‘upload-note’).style.display=’none’;”;
          echo “parent.document.getElementById(‘attachment’).innerHTML = ‘<input CHECKED type=”checkbox”><font color=#003399 size=2><b>”.$FileName.”</b> (“.$FileType.”) “.$FileSize.” Kb</font>’;”;
          echo “</script>”;
        }
        else {
          echo “<script type=’text/javascript’>”;
          echo “parent.document.getElementById(‘upload-note’).innerHTML = ‘<font size=2 color=#990000>OOPS! An error occurred, please try again</font>'”;
          echo “</script>”;
        }
       
      }
     
    ?>

    I have tried to include enough comments to make this code self explanatory. If you find any thing which is not self explainable kindly drop a comment and I shall reply to it.

    In short this file takes the file size, file name, file tmp url. Check for the file, if its valid and within constraints. i.e. Max file size 10 Mb, File format jpg etc etc…..

  6. Coding your Progress Status script (progress.php): progress.php is the script to which we will make all the ajax calls, passing it the “filekey” defined in index.php above. Using that filekey it will return the upload status of the file associated with that key.

    Essentially, it returns 3 things. Done, Current, Total i.e. Done (meaning if the upload is complete), Current (meaning how much has been uploaded already), Total (meaning whats the total size expected).

    Hence using these 3 params we will make over upload meter progress bar. Lets see the javascript function doing all these ajax requests and maintaining the progress bar.

  7. Coding the JavaScript (upload.js): As soon as the form gets submitted, it calls the getProgress() function inside this javascript function. This function starts making calls to progress.php. And then depending upon what it returns, it manages the progress bar. Here is the javascript doing the same:
    function getProgress() {
      var xmlhttp = false;
      var id = document.getElementById(‘fileKey’).value;
      var url = “http://localhost/FileUploadMeter/progress.php?id=” + id + “&nc=” + (Math.random()*100000);
      try {
        xmlhttp = new ActiveXObject(“Msxml2.XMLHTTP”);
      }
      catch(e) {
        try {
          xmlhttp = new ActiveXObject(“Microsoft.XMLHTTP”);
        }
        catch(oc) {
          xmlhttp = null;
        }
      }

      if (!xmlhttp && typeof XMLHttpRequest != “undefined”) {
        xmlhttp = new XMLHttpRequest();
      }
      if(!xmlhttp) {
        document.getElementById(‘xmlhttp-note’).style.display = ‘inline’;
        return false;
      }
      xmlhttp.onreadystatechange = function() {
        if(xmlhttp.readyState!=4) {
        }
        if ((xmlhttp.readyState==4)&&(xmlhttp.status == 200)) {
          var response = xmlhttp.responseText;
          if(response != “”) {
            if(response != “Success”) {
              var d = eval(“(” + response + “)”);
              // Update Progress Bar
              if(d.total != 0 && d.current != 0) {
                document.getElementById(‘showProgress’).style.display = ‘block’;
                var percentDone = (d.current/d.total)*100;
                // Calculate the length of the uploaded file
                var percentLength = parseInt(document.getElementById(‘showProgress’).style.width)*percentDone/100;
                // Update the length of uploaded file
                document.getElementById(‘currentStatus’).style.width = Math.round(percentLength) + ‘px’;
                if(d.current < d.total) {
                  setTimeout(“getProgress();”,1000);
                }
                else {
                  document.getElementById(‘showProgress’).style.display = ‘none’;
                }
              }
            }
            else {
              document.getElementById(‘showProgress’).style.display = ‘none’;
            }
          }
          else {
            if(document.getElementById(‘upload-note’).innerHTML == ”) {
              setTimeout(“getProgress();”,1000);
            }
          }
        }
      }
      xmlhttp.open(‘GET’,url,true);
      xmlhttp.send(null);
    }

    In the middle we can see that it checks for the returned response. If response == “”, it means we made an ajax call even before the upload started. If response != “” AND response != “Success”, it means that the upload is in progress and we need to show the appropriate progress bar.

    Progress bar is calculated based on the statistics returned. I hope its quite a simple one and again self explainatory.

    The links which I found on internet were using hell lot of unnecessary things, just to make this progress bar. Some were using Yahoo’s YUI library, some JQuery and what not and I thought may be I should make it as simple as possible.

So thats it. Here you have your upload meter ready and running.

Download the source code from google code repository

svn checkout http://abhinavsingh.googlecode.com/svn/trunk/ abhinavsingh-read-only

Feel free to post a comment or suggestion for improvement. Or if in case the scripts doesn’t work for you.

Unfortunately I couldn’t provide a web demo as always for this one, as this site of mine is still on shared hosting and I was refused to install the apc package on the shared hosting. Anyways I have tested a enough on my VPS servers and it just works perfectly.

Enjoy and Thanks for reading till here 😉

The Scariest Path in the world

For a change, let me post something interesting other than web development.

Watching this video I was jumping around my chair in order to prevent that guy from falling. Hilarious, amazing and just incredible.

If I was there would have stopped in between, called back home, asked them to arrange for a helicopter by hook or crook. I couldn’t have proceeded forward by any mean.

original source of video:
http://www.livefortheoutdoors.com/Videos/Search-Results/Fun–misc/The-scariest-path-in-the-world/

Enjoy something other than technical 😉

How to make faster websites and enhance your site user experience – Part 1

In all my posts till now I concentrated over how to get started with web development and related introductory stuffs. In next few posts (series of 3 posts), I will write some of my learning in the field of “How to make your website faster and enhance your site user experience”.

So lets assume I made a site, which rocks with all those web2.0 features in it. But still my user complaints like:

  1. My site hangs their browser while loading.
  2. The page takes too long to load.
  3. Blah Blah Blah…… 🙁

So how to go about and make sure that you do your best to make things rocking. Here are a few of my tips on things that you should concentrate upon and which will definitely help in fast loading of your pages:

  1. GZip your site static content: This is one of the most important stuff which helps a great deal in enhancing your site download speed. Let me explain the process:
    • When I type www.yahoo.com in my browser and press enter, an interaction starts between my browser and the www.yahoo.com servers. My browser asks for the yahoo.com homepage from the yahoo servers. However all Grade-A browsers sends a few useful stuff along with this homepage request. They tells the server that I support gzip encoding, and if you serve me the contents in g-zipped format I will happily except it, unzip at my end and render it for the user.
    • The Yahoo server sees these headers being sent by the browser and depending upon its configuration it gzips the content before serving back to the browser. However, if Yahoo server’s are not configured for the gzip encoding, it will server the content in its raw form. Further, if Yahoo server’s are configured for gzip encoding but my browser isn’t a Grade-A browser, servers are smart enough to return back the content in the form which suits my browser.
    • For instance here are the request header sent by my browser to the yahoo servers when I type in http://yahoo.com in my browser:

      In the above image we can very well see that my browser sends in a header saying Accept-Encoding : gzip,deflate. On seeing this in the headers Yahoo server responds as expected i.e. it serves back the gzipped contents to the browser.

      In the above image we see the response headers being sent by the Yahoo server to my browser. We can see that Yahoo server tells back my browser that it has served the contents in Content-Encoding : gzip , so please unzip them at your end before rendering them for the user.

      So how does the end user’s experience gets enhanced?

      If we see the above image we can very well see that the Yahoo homepage which is actually of 125.1Kb comes in as 31.9Kb only. Further the javascript files gets compressed and hence faster data transfer through the channels and thus faster experience for your site users.

      Though gzipping the site content increases your server side CPU load, but that gets compensated by the fast that users are able to see your pages load faster.

    • What all contents should I gzip? Its always better to gzip the javascript files, CSS files, HTML files, Plain Text files (In some cases XML’s too but I have had a few problems with gzipped XML’s). We generally don’t compress images, pdf’s etc as they are already in best compressed format and further compressing won’t give us any smaller file sizes.
  2. Setting an ETag for your sites content: This is another factor which helps the page load faster for the users. By setting up an ETag , server sends an information about the contents being served to the browser, telling the browser about its last modified date. So when I type in http://yahoo.com in my browser and presses enter, my browser requests for content from the Yahoo servers. In reply if it sees that the ETags haven’t changed for the requested component, it simply uses the component from browser’s cache. Hence this time we saves the data transfer part, as browser is capable enough to cache enough static data from one single domain.

    In the above image Yahoo servers doesn’t reply back the ETags possibly because of two reasons. Advertisements and Expires Headers which we will see in the next point. As Yahoo have advertisements on all their pages, they simply don’t want those contents to be cached by our browsers. So that everytime we open yahoo.com , we see a new advertisement. (Experts, Kindly correct me if I am wrong in my explanation here)

  3. Setting up the Expires headers: This factor even saves the browser pinging the server time.

    In the above image we can see Yahoo servers setting up an expire header telling the browser that, this component is not going to change before the setup expired header. Hence next time you type in Yahoo.com in your browser and press enter, browser sees the content which still haven’t expired and picks them up from the browser cache. Thus they don’t even ping the servers, saving your servers BW and enhancing your users page load time.

    What if I have setup expired headers 2 months from today and I want to change the content before that?
    Simple enough, next time in place of http://l.yimg.com/us.js.yimg.com/lib/bc/bc_2.0.4.js will be change to something like http://l.yimg.com/us.js.yimg.com/lib/bc/bc_2.0.5.js ,  i.e. changing the version of the javascript file being served from server side. Hence browser will be forced to download the new javascript file from the servers rather than using them from the cache.

  4. Use of sub-domain or different domain for serving static contents: We can very well see in the above image that Yahoo serves all its javascript files from a server called http://l.yimg.com which is supposedly one of its static server. But why the hell do I need to do this? This is because currently all Grade-A browsers allows 2 parallel connections to a host. i.e while loading a page yahoo.com, my browser can download only two files at a time from the yahoo yahoo.com server. Hence this will make other contents to go in queue, wait for the already downloading contents to finish before they get downloaded.

    Hence for the same reasons, we generally server our static content from one or more different sub0domains or full-domains all together. In future Internet Explorer 8 and Firefox are planning to increase this 2 requests per domain to 4-8 requests per domain.

  5. Minify your Javascripts and CSS files: What is minify? It was something developed and introduced by Douglas Crockford. Minifying your sites javascripts and CSS files two purpose at the same time. They decreases the size of your javascripts and CSS files and also hides (make it tougher to parse by others) your javascript calls. I minified javascript file will look like this:

    We can see it makes the size of javascript file by eliminating unwanted white spaces and blank lines. Plus it also makes it difficult for a hacker to tweak in the javascript file.

  6. Put CSS on top and JS at the bottom , avoid inline javascripts: By putting CSS on top ensures that user’s browser will render the page content for your users as and when it gets the HTML content. We want to put Javascripts at the bottom and avoind inline scripts, because while the page loads we do not want browser to start executing the contents in javascripts, which may create a lock time for other contents before they gets downloaded.

Thats it! There are a lot of other things which you may want to do, for making your site page load faster. But the above described 6 points dominates the page load time. I am no expert and still learning the tid-bits of scalable websites, hence if you find any mis-leading content , kindly leave a comment. I will be more than happy to edit the post.

PS: These are a few of the frontend things that we can do on the frontend to make our sites faster. In my next post of this series I will try to bring in the key backend things which can help in faster loading of the webpages.

Enjoy making cooler, faster webpages for your users 🙂