Search This Blog

Loading...

Tuesday, January 27, 2009

How to Compress PHP and Other Text-Based Files with 1&1 (and Other Shared Hosts)


If you have a shared Linux hosting package through 1&1, you have no doubt discovered that the usual method of changing your .htaccess file does not allow you to compress your files using Apache because 1&1 has disabled the mod_deflate and mod_gzip modules; you also cannot specify php values in .htaccess files in 1&1 shared packages, so a solution like this one also will not work. However, there is a solution that does work, and it works even on the most basic 1&1 Beginner’s Package.

Note: If you plan to implement the Minify! option I blog about in How to Easily Combine, Minify, and Cache JavaScript and CSS..., you can safely skip steps 1 and 2 and include only the first line shown in step 3.

Step 1:

Determine what your root path is; if you are using a 1&1 package, it will look something like this:
/kunden/homepages/12/d123456789/htdocs/
You can do that by creating a text file in Notepad or other editor,* entering the following lines, then saving it with as phpinfo (or whatever you want) followed by the .php extension.
<?php
phpinfo();
?>
(* If you’re using Notepad, you’ll probably want to put quotation marks around the name "phpinfo.php" when saving, to keep it from saving the file as phpinfo.php.txt.

Personally, after passing from Geany on Windows (used Notepad++ some), and Komodo Edit on Mac OSX, my editor of choice is now a customized version of vim. Incidentally, the three previously mentioned IDEs mutually rely on the Scintilla editor as their base.)

Now browse to the newly uploaded file with your web browser and search for DOCUMENT_ROOT. This variable will tell you what the exact path is to your server.

Step 2:

Now create another php file in the same way, this time adding the following information so that when you compress your CSS and JS files, they are served with the right headers, or else they will not load properly in Firefox and other browsers:
<?php
if (isset($_SERVER['SCRIPT_FILENAME'])) {
    $timestamp = filemtime(__FILE__);
    header('Last-Modified: ' . $timestamp);
    $expires = 60*60*24*14;
    header("Pragma: public");
    header("Cache-Control: maxage=".$expires);
    header('Expires: ' . gmdate('D, d M Y H:i:s', time()+$expires) . ' GMT');
    header('Vary: Accept-Encoding');
    $pathinfo = pathinfo($_SERVER['SCRIPT_FILENAME']);
    $extension = $pathinfo['extension'];
    if ($extension == 'css') {
        header('Content-type: text/css');
    }    
    if ($extension == 'js') {
        header('Content-type: text/javascript');
    }
}?>
Make sure that you don’t space down between the lines or add an extra space at the end, and save this file as headers.php, uploading it to wherever you keep your scripts. For the sake of this example, I will assume that you store your scripts in a directory named webscripts.

[Update: As Brett reminds me in the comment below, you have now given PHP control of rendering your pages, so you will also need to include in this file any other header information you know you’ll need by declaring each header on its own line. For example, notice this extra line:

header('Vary: Accept Encoding');

Based on the readership of these articles, this will be particularly useful when optimizing for speed using a free tool like Google's Page Speed and the Yahoo! based YSlow. Add as many extra headers as you need, one per line.]

Step 3:

Create a third file, this time a PHP configuration file, entering the following information (though of course changed with the information you looked up in step 1 and whatever directory you use for step 2—unfortunately, you can’t use environmental variables in .ini files, as they are straight ASCII):
zlib.output_compression=1
auto_prepend_file=/kunden/homepages/12/d123456789/htdocs/webscripts/headers.php
Name your configuration file the conventional php.ini and save it. (If for some reason zlib.output_compression=1 doesn't work, try output_handler=ob_gzhandler instead.)

The first line tells the server to use PHP’s built-in g-zip handler for compression; the second line points to the exact location of the file you created in step 2: /kunden/homepages/12/d123456789/htdocs/ is the document root you located in step 1, and webscripts/ is the directory where you uploaded the headers.php script.

auto_prepend_file tells the server to automatically “prepend,” or “attach to the beginning of” all pages it serves (versus auto_append_file, which would “append,” or “attach to the end” of all files). The if/then conditional statements in step 2 ensures that the headers.php file will only attach headers if the file in question has a .css or .js extension.

Note: Check with your web host. With 1&1, at least, you will need to upload a copy of your php.ini file to EVERY directory containing files you want compressed.

Step 4:

Create an .htaccess file or modify your existing one by adding the following line (if you’re not familiar with an .htaccess file, see here):

CAUTION: If you plan to implement the Minify! solution I blog about in How to Easily Combine, Minify, and Cache JavaScript and CSS..., I do not recommend including the .css and .js file extensions on this line. Doing so processes these files twice, as Minify! already has its own mechanism for compressing files. And, as a result of double compression, certain versions of Internet Explorer choke and die silently.

AddType x-mapp-php5 .php .shtml .html .htm .txt .js .css
This line tells the server to process all the file extensions specified through PHP5; change it to x-mapp-php4 if you want to use PHP4 instead. You can add other extensions or take away from the ones here: whatever extensions are listed on the line after the x-mapp-php5 will be processed via PHP.

Note: On other Linux hosts, you may need to use php5-script instead of x-mapp-php5.

[June 4, 2011 Update: I was setting up my sister's site recently, also on a 1&1 Beginner’s Package, and the style sheets were not working correctly. The fix was to add this line to the .htaccess file immediately above the line shown in the example above:
RemoveHandler .css
I first tried removing all file extensions I was using, including .htm, but this caused problems. Just know that you have the option of removing file handlers and play around with it as needed.

Step 5:

Now test to see whether or not your pages are being compressed by using the free tool from GIDNetwork.

35 comments:

  1. i found that using this technique prevented my .htaccess from adding expiration headers to css and js files. i'm a bit of a noob, so my solution may not be the most elegant. i changed the headers.php file to:

    <?
    $pathinfo=pathinfo($_ENV['SCRIPT_FILENAME']);
    $extension=$pathinfo['extension'];
    $offset = 60 * 60 * 24 * 30;
    if($extension=='css'){
    header('Content-type: text/css; charset=utf-8');
    header("Expires: ".gmdate("D, d M Y H:i:s", time() + $offset)." GMT");
    }
    if($extension=='js'){
    header('Content-type: text/javascript; charset=utf-8');
    header("Expires: ".gmdate("D, d M Y H:i:s", time() + $offset)." GMT");
    }
    ?>

    ReplyDelete
  2. i forgot to say thanks for the ideas and that it did help a lot.

    ReplyDelete
  3. No problem, Brett. And you bring up a good point: PHP is now processing your files, so any file header information that you want sent will also need to be specified in your file, much as you have done. Thanks for passing along the new information. :)

    ReplyDelete
  4. Thanks a lot for this work - everything is working fine - except of the wp-frontend. I do not have the php.ini in the wp-admin-folders - but the layout of the dashboard and the editor is broken.

    Do you have an idea?

    Thanks a lor and many greetings from germany
    Oliver

    ReplyDelete
  5. Hello to you from America, Plerzelwupp.

    I suspect that if everything is working everywhere else but your admin panel, that it is because PHP is serving both your javascript and your css with "text/html" as the transfer type. Although, most modern browsers should handle the issue.

    If adding a php.ini in the admin folder doesn't pan out for you, my next suggestion is to go to the WordPress forum (http://wordpress.org/support/) and see if anybody's posted anything first, and, if not, ask for advice there.

    Hope that helps.

    ReplyDelete
  6. For 1and1 shared hosting an even easier solution is to just add the following line to the top of your index.php file:

    ob_start("ob_gzhandler");

    http://www.magentocommerce.com/boards/viewthread/46891/

    Although this probably only compresses the source file (Xhtml) and not the included assets (Css/js etc)

    ReplyDelete
  7. Well, that solution works on a page-by-page basis, unless you have a templating system that shares the main index.php file (as I assume is the case with Magento). If all other files use this file as the page template, simply including their own content, then you're good to go.

    However, you are correct: It will neither compress your script and css files nor any files that are not built into the index.php file.

    So the difference is that between site-wide compression that should work anywhere versus page-by-page compression that will work in some instances.

    ReplyDelete
  8. Hi,

    Using this and the minify doesnt work for my site in firefox. i am on 1and1 so added the line
    output_handler=ob_gzhandler
    to a new php.ini and now it isnt using css at all.
    Google site performace tells me "The following publicly cacheable, compressible resources should have a "Vary: Accept-Encoding" header: "
    how do i do this?

    My site is

    http:www.ourgoodlife.co.uk

    if that helps at all.

    Thanks in advance!

    ReplyDelete
  9. The only thing we are doing in the first and second steps in the directions above is specifying the path to the script that tests for the .css or .js extensions and serves up different headers if a match is found. The reason the directions say to skip these steps if using Minify! is that Minify! already has its own mechanisms for handling JS and CSS files.

    It sounds to me like you might have encountered an issue with Minify! itself. I would go to http://code.google.com/p/minify/ and check out the Wiki tab first, then post your question under Issues if that still doesn't resolve the issue. I wish I could be of more help. :(

    By the way, your site looks very nice and clean. ;)

    ReplyDelete
  10. thanks i will look into it!

    ReplyDelete
  11. Hmm. I've been working on the blog a bit and happened to re-read your question and realized that Brett's post--very first one--might contain a solution.

    Look at his top post as reference, then add the following line to it, once for each "if" block for the CSS and JS extensions:

    header("Vary: Accept-Encoding");

    That should work, though know I have not tested.

    ReplyDelete
  12. I found the site, in respect to this post, by looking for info on HTTP headers

    http://www.caucho.com/resin/admin/http-proxy-cache.xtp

    and found the info valuable, though it clearly wasn't written in PHP and I wanted to know definitely what server environment it was before passing along the link.

    It looked kinda interesting: an open-source Java and PHP server environment software.

    Particularly for the creative persons who have full administrative privileges on their servers and who aren't afraid to try new technologies, it might be just the thing. The two softs are Quercus and Resin, Quercus included with Resin.

    Resin is available in both a free, open-source model (GPL License) and a paid model starting at $699 per server/per year, providing full technical support.

    Main download page is here: http://www.caucho.com/download/

    ReplyDelete
  13. Hi, this works on 1and1 with php css and html files :)

    Using webpagetest.org I still have some issues:-

    Specify a cache validator - The css file is missing a cache validator

    Leverage browser caching - short freshness lifetime, expiration not specified
    (I think this is the same prob as above)

    Minifying the index.php (can this be easily done?)

    Minifying the CSS file (can this be easily done?)

    Combine images into CSS sprites (can I just reduce their size in a similar way to jpgs?)

    Optimizing gif images (how is this done?)

    Thanks, Phil.

    ReplyDelete
  14. I did not get a chance to reply to you, Philip, when you first posted, then forgot to get back with you.

    In any case, check out http://developer.yahoo.com/performance/rules.html for ideas on cache validation. For minifying your index.php file, see http://mrrena.blogspot.com/2011/02/how-to-minify-html-using-php-and-minify.html Note that the effects may be negligible or even worsen your page speed.

    All my pages recommended using Minify, linked from this article or http://mrrena.blogspot.com/2009/02/how-to-easily-compress-javascript-and.html That will solve minifying your CSS and JavaScript files.

    Compressing images will not achieve the same result as image sprites. Image sprites make only one call to the server--versus however many individual image calls--and are quite easy to implement. You can generate them online easily with http://spriteme.org/ or for more information, see http://websitetips.com/articles/css/sprites/ or http://www.elated.com/articles/css-rollover-buttons/

    Do note, however, that depending on where and how the images are used, they may require either extra blank space or else not be spritable at all (like tiled background images).

    Finally, to easily optimize GIF images, just install Google's PageSpeed as a plugin to Firefox http://code.google.com/speed/page-speed/download.html and it will compress all such images and allows you to download the newly compressed file--pretty cool.

    ReplyDelete
  15. @Eric, thanks for solution. I thought it is impossible to do with 1and1 hosting.

    ReplyDelete
  16. Using this method I was able to finally gzip my site! Combining this php script to automatically copy the php.ini to every folder made things much easier: http://tips-scripts.com/php_ini_copy

    ReplyDelete
  17. hi,
    very usefull, I get the gzip thru addhandler and headers.php as you suggest, but then it does not do anything with the commands of htaccess for css or js like
    - addExpires
    - Header set

    neither thru filesmatch or ExpiresByType

    any other solution instead of setting it headers.php? why does not affect those commands

    thnx 4 the tutorial

    ReplyDelete
  18. I don't have a great deal of time at the moment to research your question. I will simply say that there is a debate about which is better: using .htaccess to control caching or using headers. The point is, both can be used to accomplish the same end.

    That means that if your .htaccess file no longer affects your JS and CSS files, you should be able to add equivalent headers to your headers.php file as mentioned under the "Update" heading above. In most cases you can use both; for example, for all my images (which aren't using the technique described here), I have the following lines in my root .htaccess file:

    ExpiresActive On
    ExpiresByType image/gif "access plus 1 year"
    ExpiresByType image/jpeg "access plus 1 year"
    ExpiresByType image/png "access plus 1 year"

    ...which tells Apache to cache my images for a year.

    While I don't have time to research which headers you'll need for your JS and CSS files, I do have time to share with you the links to some of my personal bookmarks of pages I have found very helpful in the process of getting everything up to speed.

    For cache control in general:
    "Cache Control Directives Demystified" -- http://palisade.plynt.com/issues/2008Jul/cache-control-attributes/
    "Caching Tutorial for Web Authors and Webmasters" -- http://www.mnot.net/cache_docs/

    For Apache specifically:
    "Use Server Cache Control to Improve Performance" -- http://www.websiteoptimization.com/speed/tweak/cache/
    "Stupid htaccess Tricks" -- http://perishablepress.com/press/2006/01/10/stupid-htaccess-tricks/
    "more .htaccess tips and tricks.." -- http://corz.org/serv/tricks/htaccess2.php
    "mod_rewrite Quick Reference and Cheat Sheet" -- http://semlabs.co.uk/journal/mod_rewrite-quick-reference-and-cheat-sheet

    Hope that helps. The first two resources in the first list should help you out most in using an equivalent solution for headers.php that you are currently using in your .htaccess file.

    ReplyDelete
  19. P.S. To target individual files, you'll need to include some conditional logic in your headers.php file. For example, something like this might do the trick:

    if ($_SERVER['SCRIPT_NAME'] == '/scripts_dir/my_special_file.js') {
    //add these additional headers
    }

    To see a list of available PHP variables, save a PHP file containing just the following three lines and upload to your server, then just access the page through your browser:



    ...saved as "phpinfo.php"

    ReplyDelete
  20. Sorry... the three lines got chopped off because I didn't use HTML Entities for the "<" and ">" symbols. Here is again:

    <?
    phpinfo();
    ?>

    ReplyDelete
  21. A simple way to get 1and1 to compress php files/pages is to simply create a php.ini file (which as you describe must be placed in every folder) containing the single line:

    zlib.output_compression = On

    Once this is in place php files will be compressed. It does not work for other file types, e.g. .html, .css etc., although if you choose to name your css and other files .php they will then be compressed, although served with the incorrect MIME type of text/html which may cause problems with some user-agents.

    If you elect to use minify to compress and serve css and js files (as described in your article on this), then you also ‘cure’ your css and js file issue. This then only leaves .html/.htm or other static files which of course you could simply name .php to gain compression; you don’t actually need to alter the content of the files in any way.

    One benefit of using the php.ini solution above is that it has the advantage of using the zlib compression which is the php recommended approach in preference to use of ob_gzhandler (ref.: http://php.net/manual/en/function.ob-gzhandler.php).

    Having got this to work I have carried on looking at getting .html/.htm/.shtml and other file types parsed via php as you describe, and on 1and1 in Germany at least, had no luck at all. In looking at your article you add the following to .htaccess:

    AddType x-mapp-php5 .php .shtml .html .htm .txt .js .css

    However, AddType adds a MIME type and so results in any files marked-up like this being served, not as text/html or whatever, but as ‘Content-type: x-mapp-php5’. This then causes the browser to ask what you want to do with this unknown MIME type document and asking whether you want to save the file rather than render it. Not the intended outcome :-)

    I have tried AddHandler as a means to associate a handler with particular file types but this has failed and I have even managed to generate the odd internal server error or two :-)

    ReplyDelete
  22. And while going through all this a few more tricks I’ve been playing…

    Adding/correcting file MIME types with:

    AddType image/x-icon ico
    AddType application/javascript js

    (1and1 otherwise serve .ico files with no/an unknown MIME type and .js files as application/x-javascript…although there’s argument about the relative merits of application/javascript vs. text/javascript)

    Also setting client-side caching with:

    <IfModule mod_expires.c>
    ExpiresActive On
    ExpiresDefault A0
    ExpiresByType image/gif A1200
    ExpiresByType image/jpeg A1200

    </IfModule>

    This add a max-age of a number of seconds, so A1200 = 20 minutes to whatever types you define. And why 20 mins and not a month? Well, if you have lots of users returning to your site numerous times over the course of a month, or year, fine, although you can then stale cache issues unless you use resource version numbers (e.g. image.jpg?version=20110912), however, if people typically come for 10 or 15 mins, having them cache an object for a year doesn’t really buy anyone very much.

    Custom error pages with:

    ErrorDocument 400 /errorPages/bad_request.html

    Although oddly I’ve found on 1and1 that this works if the incorrect URL isn’t a php URL. So a request for ‘missing.jpg’ gives you the custom 404 page, but a request for ‘missing.php’ still gives you a 1and1 page saying the domain has just been registered!! Not what you want, so I’ve not cured that one yet.

    I’ve also found if you get ‘near misses’ 1and1 substitute a file of similar file extension. So create a simple test page and call it ‘simple.shtml’ (note the ‘s’-html). Now request ‘simple.html’ (note the no-‘s’) and you don’t get a 404, you instead get the .shtml file! If you request ‘simple.htm’ you even get a page suggesting you might like ‘simple.shtml’! You can also get similar .php offered to you instead of either a 404 or your custom 404 page. Interesting stuff going on in the 1and1 httpd.conf!

    And finally I’ve been playing with ‘friendly’ (or short) URLs, with:

    Redirect 301 /short/ http://www.yourdomain.com/somepath/anotherlevel/controller.php?page=101&theme=xyz

    ReplyDelete
  23. Hi Iain,

    Thanks for your comments!

    So far as I'm aware, when you use AddType x-mapp-php5, you're running those file extensions through PHP 5's file handler. I would think that just as step 2 in the article above delves into changing the header information sent for JS and CSS files, you'd probably also need to add HTML and possibly TXT files to the list as well:

    if ($extension == 'htm' || $extension == 'html' || $extension == 'shtml') {
    header('Content-type: text/html; charset=utf-8');
    }

    You might try that, and if it works, definitely post back here for everybody else's benefit.

    And I'll admit, this solution's not glamorous, but it is a way around the default settings on 1&1's shared servers. ;)

    ReplyDelete
  24. I'm afraid AddType isn't a directive for the server to parse a file, it simply defines a MIME type that files with a particular file extension should be served as. I quote:

    [quote]The AddType directive maps the given filename extensions onto the specified content type. MIME-type is the MIME type to use for filenames containing extension. This mapping is added to any already in force, overriding any mappings that already exist for the same extension. This directive can be used to add mappings not listed in the MIME types file (see the TypesConfig directive).

    Example: AddType image/gif .gif

    It is recommended that new MIME types be added using the AddType directive rather than changing the TypesConfig file.

    The extension argument is case-insensitive, and can be specified with or without a leading dot.[/quote]

    See: http://httpd.apache.org/docs/2.0/mod/mod_mime.html#addtype

    So you could for instance define [code]AddType text/html .gif[/code] and have images rendered as a meaningless text stream by your browser. Not something you’d want to do, but illustrates the point. Therefore:

    AddType x-mapp-php5 .php .shtml .html .htm .txt .js .css

    Means that files with the listed extensions are served with a MIME type of x-mapp-php5 which means nothing to the browser, hence it asks you what you want done with the file. If you look at the HTTP response with this directive in place you see:

    HTTP/1.1 200 OK
    Date: Mon, 26 Sep 2011 18:12:54 GMT
    Server: Apache
    Content-Encoding: gzip
    Vary: Accept-Encoding
    X-Powered-By: PHP/5.2.17
    Keep-Alive: timeout=2, max=200
    Connection: Keep-Alive
    Transfer-Encoding: chunked
    Content-Type: x-mapp-php5

    Note that last line…Content-Type. It should read, and indeed does without the AddType x-mapp-php5… directive, as:

    Content-Type: text/html


    In contrast if you look at AddHandler:

    [quote]Files having the name extension will be served by the specified handler-name. This mapping is added to any already in force, overriding any mappings that already exist for the same extension. For example, to activate CGI scripts with the file extension .cgi, you might use:

    AddHandler cgi-script .cgi

    Once that has been put into your httpd.conf file, any file containing the .cgi extension will be treated as a CGI program.

    The extension argument is case-insensitive, and can be specified with or without a leading dot.[/quote]

    See: http://httpd.apache.org/docs/2.0/mod/mod_mime.html#addhandler

    The documentation then shows the following example:

    [quote]Server Side Includes example

    Another common use of .htaccess files is to enable Server Side Includes for a particular directory. This may be done with the following configuration directives, placed in a .htaccess file in the desired directory:

    Options +Includes
    AddType text/html shtml
    AddHandler server-parsed shtml

    Note that AllowOverride Options and AllowOverride FileInfo must both be in effect for these directives to have any effect.[/quote]

    So maybe ‘AddHandler server-parsed shtml’ (or css, js etc.) is a route to try? I’ll experiment…

    ReplyDelete
  25. Another observation with 1and1...

    If you set a error document directive as in:

    ErrorDocument 404 /errorPages/not_found.html

    this is only effective for non php URLs. So /missing.gif or /missing.html will give you your custom error page, but /missing.php will still lead you to the horrible 1and1 "This domain name has just been registered." error page!

    ReplyDelete
  26. Cool. Thanks for the heads up. :)

    ReplyDelete
  27. Is there a way to use your method to compress my website while also allowing my htaccess file to continue hiding the .html file extensions of each webpage?

    This big of code seems thrown out of sorts when I paste in your code. (Probably because of the rerouting it through php part?)

    #
    Options +FollowSymLinks
    RewriteEngine On
    #
    # REDIRECT /folder/index.html to /folder/
    RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*index\.html\ HTTP/
    RewriteRule ^(([^/]+/)*)index\.html$ http://weaverinnovations.com/$1 [R=301,L]
    #

    ReplyDelete
  28. Apache and PHP should play okay together.

    Try this (from a comment @ http://php.net/manual/en/security.hiding.php -- I just changed the file extension from "php" to "html" per your example above):

    Options +FollowSymlinks
    RewriteEngine On
    RewriteBase /
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_FILENAME}.html -f
    RewriteRule ^(.+)$ /$1.html [L,QSA]

    It might not get rid of your "index.html" page in "folder," but if you add the following at the top of your .htaccess file, it will at least ensure that "http://weaverinnovations.com/folder/" will be able to access "index.html" without showing it. (It will not, however, keep it from showing if the full path is accessed.)

    DirectoryIndex index.html

    If this works out for you, you might comment back to leave a record for the next weary web traveler. :)

    Peace out,
    Eric (aka mrrena)

    ReplyDelete
  29. I should add that if you're on 1&1 (and maybe even if you're not),

    DirectoryIndex index.html

    would go inside the .htaccess file inside the "folder" directory, not the root directory (unless it too has a "index.html" file, and then you'll need it both places).

    ReplyDelete
  30. hi i am trying this on 1and1 and the second i put the .js and .css in the htaccess file the site poops out. doesnt use js or css. i have done everything else perfectly as outlined above. even tried the [ header("Vary: Accept-Encoding");
    if ($extension == 'css') { ]

    also tried the remove handler note as well. not sure if i did something wrong, but i followed this step by step

    ReplyDelete
  31. Awesome dude, Ive been searching the net for weeks looking for an answer to this conundrum, 1and1 certainly cant help! Brilliantly explained for the novice too. Thanks and keep up the good work.

    ReplyDelete
  32. Hello
    Many thanks for this post
    For me it didn't totally work. I've followed your guide step by step, and gain some speed, but GIDZip confirm I do not have Gzip active on my site www.fengshuienfrance.fr
    Very frustrating as I'm using a great plugin to speed up my site, but it's limited if I can't use Gzip. Of course I am with 1&1...Any helps appreciated!

    ReplyDelete
  33. Have you got any tips on how to make this file ignore a folder e.g. /blog as its working perfectly on my site but my blog which is wordpress has now stopped working properly e.g. its not reading the .css in the admin section.

    ReplyDelete
  34. An invaluable guide! -Much Thanks!

    ReplyDelete