Upload Progress Bar / Meter / Monitor in PHP

How to upload huge files in PHP?

Long time i was looking for a method to provide our customers with file exchange services. Sometimes, our customers need to upload very large files beyound 500MB size to us. I have tried to create customer area on our shared webhosting server, but it simply didn't work for large files, because there were too many restrictions. So we descided to host customer area directly on our office network by creating an extranet site hosted on IIS and latest PHP. Since i have root access to our servers, i could try every solution available. That is what i did. Solututions requiring client side changes or plugins were immediately out of question. Most available server-only solutions i have tried were too sophisticated, tricky, or simply didn't work. I have tried all progress bars and variants i could find in google, including uploadprogress (php_uploadprogress), CGI based uber uploader and some more. uploadprogress was simply too buggy, had almost no support for windows and didn't work for that OS. uber uploader was too heterogenous, a mix of CGI and PHP. Due to lack of my Pearl knowledge it was very hard to customize its CGI part and get it to cross communicate with PHP. Beside this i had massive problems with temp directories not being closed and therefore locked, when the client was cancelling upload.

I needed something simple, robust and working!.

And finally i found a solution in form of Alternative PHP Cache (APC). Less known for its upload monitoring capabilities, this package is mainly used for PHP caching. With introduction of PHP 5.2 and its new upload file hooks, APC implemented interface for cachings and monitoring file uploads, making upload brogress bars possible almost out-of-the-box. After investigating its functionality and implementing it in our customer file exchange platform, i have descided to bring my new experience to the "paper", in hope that it will help you save lots of time and frustration.

This soultion should work for Windows Server, IIS and PHP 5.2. Other configurations should work either. Howerver you need root access to enable APC and set up php.ini file.

This solution will most likely NOT WORK on shared webservers, where APC is disabled by default by ISP, and you have no permission to change PHP configuration. Sometimes, depending on webserver configuration, you can override php.ini settings of shared webhost by placing your own php.ini configuration into the directory containing your PHP scripts. It may or may not work.

The key features of this upload method are:

Installation

This upload method requires PHP 5.2 or higher and PHP APC extension. PHP versions lower than 5.2 will not work, since they do not implement the required hooks to monitor file uploads. PHP APC extension version 3.0.13 is shipped with default PHP 5.2 installation. The latest compiled php_apc.dll can be downloaded at http://pecl4win.php.net/ext.php/php_apc.dll. Remember only PHP 5.2 version of APC can handle RFC1867 HTTP file uploads.

After you have placed php_apc.dll in your PHP\ext folder, you have to configure some settings in your php.ini. First enable the APC extension and turn on form-based file upload caching defined by RFC1867 specification.

extension=php_apc.dll apc.rfc1867 = on

Next you have to check that file uploads are turned on and the directory for uploaded temporary files is set correctly. You also need to set read/write permissions for that folder. At least user accound used for webserver process must have read/write privileges. Alternatively you can enable full access permissions to all users (less safe).

file_uploads = On upload_tmp_dir = D:\UPLOAD_TEMP

Finally, if you plan to provide upload for really huge files, you should rise some PHP limits. 60 seconds for max_execution_time should be enough to process completely uploaded form data and move uploaded files to desired destination. Be aware that if youre moving files cross partitions, copying raw file data can take significantly longer. Rise this number to something higher then, for example like 3600 seconds. The best practice is however not to move uploaded files cross partitions. max_input_time is the time limit for uploading form data including files. For example if use is uploading 1GB file at the speed of 128KB/s it will take him 7800 seconds to complete form upload. Keep this value very high for large files. The default memory_limit of 128M should be enough for processing upload php scripts. You should even lower this value on production servers with many simultaneous uploads. post_max_size and max_input_time are the actual limits for maximum upload size. Change these values to your needs, but keep them equal to each other. Avoid too high numbers in all settings, this may result in stucked threads, unnecessarily consuming server ressources and may lead to server instability or crash.

max_execution_time = 60 max_input_time = 7200 memory_limit = 128M post_max_size = 1000M upload_max_filesize = 1000M

After changes have been made to php.ini restart your webserver. For example, if you are usind IIS, you can restart it from command prompt with iisreset. You should now chech with phpinfo() that APC extension is loaded and configured properly.

Sample Code

Copy following source code and save it to upload.php or any other named .php file on your server. Run it to test how upload works. In this code the uploaded files will be dismissed at the upload end. This final part of upload processing you have to implement yourself. I have tried to keep source code as simple as possible but still keep it's nice functionality. All functionality is kept in one single file for simplicity, you can of cource break it apart.

The source is self-explanatory. In short: for each form call, we generate unique ID to identify our upload. PHP casts this ID to all places in our HTML source. A special required hidden field APC_UPLOAD_PROGRESS will pass the ID to APC, so that we can identify the uploading form from other PHP processes. Then the form, filled with file data, get submitted for processing to itself. At the begin of form submission UpdateProgress() is called through onsubmit event. This function will call itself every 500 milliseconds and create new JavaScript object in our HTML document on every call. The source of this JavaScript object is set again to the php file itself, but with DownloadProgressID indicating that we want to track download progress. The JavaScript object receives one single JavaScript function generated by apc_fetch. apc_fetch is the only function of APC we need for our purpose. Provided with upload ID it returns array containing all the information we need to monitor files being uploaded. That info is then sent back through UpdateClient(...) JavaScript function which will dynamicly update our Progress bar. CachePrevention parameter is used to bypass all known caching mechanisms of browsers, webservers or any other proxys in between. Here we use technique which allows us to avoid frames or iframes. This is made possible by the fact, that modern browsers allow dynamic updates to HTML document, while form data upload is being processed. Finally, when upload is complete, the page will be automaticly redirected to itself, stopping our javascript monitoring by the page reload. On the new page load we identify that form has been sumbitted by checking $_SERVER['REQUEST_METHOD'] == 'POST' variable. At this point you can process files and form date in usual way.

Sample PHP source code:

UploadForm.php

<html> <head> <title>UploadProgress</title> <script type="text/javascript"> function UpdateProgress(){ document.getElementById("UploadProgress").style.display = ""; document.getElementById("UploadForm").style.display = "none"; var e = document.createElement('SCRIPT'); e.type = 'text/javascript'; e.src = "JavaScriptInjection.php?DownloadProgressID=" + UploadForm.APC_UPLOAD_PROGRESS.value + "&CachePrevention=" + (new Date()).getTime(); document.getElementsByTagName('head').item(0).appendChild(e); setTimeout("UpdateProgress()", 500); } function UpdateClient(data) { try { document.getElementById("ProgressText").innerHTML = "Uploaded " + (data["current"]/1024/1024).toFixed(2) + "MB of " + (data["total"]/1024/1024).toFixed(2) + "MB"; document.getElementById("ProgressBar").style.width = (data["current"]/data["total"]*100) + "%"; } catch (e) { null; } } </script> </head> <body> <form id="UploadForm" enctype="multipart/form-data" action="ProcessForm.php" method="POST" onsubmit="UpdateProgress();"> <input type="hidden" name="APC_UPLOAD_PROGRESS" value="<?php echo uniqid() ?>"/> <input type="file" name="up_file_1"/><br> <input type="file" name="up_file_2"/> <input type="submit" value="Upload!"/> </form> <div id="UploadProgress" style="display:none;"> <span id="ProgressText"></span><br/> <div style='width:600px; background:#CCCCCC;'><div id="ProgressBar" style="background-color:#00CC66; width:0%;"></div></div> </div> </body> </html>

JavaScriptInjection.php

<?php if ($e = apc_fetch("upload_{$_GET['DownloadProgressID']}")) echo "UpdateClient(" . json_encode($e) . ")"; ?>

ProcessForm.php

<?php // At this point file upload is complete. // move_uploaded_file($_FILES["up_file_1"]["tmp_name"], "c:\\upload\\" . $_FILES["up_file_1"]["name"]); ?> Upload finished.

Contact

I hope my soulution was of any help to you!

If you want to give me some feedback or have questions send email to yurij@arcor.de