Front-end performance and frameworks

March 24, 2008 – 5:17 am Tags: ,

Yahoo has been researching various different techniques for improving website performance. There are a lot of obvious ones, such as making sure images are optimized, JavaScript/CSS is minified and so on.

There are also a couple of less known tricks, such as putting JavaScript as the last in the document, and “flushing early”.

Flushing early basically means that when you are generating your page, you should send data to the browser as soon as possible. A lot of frameworks (including Zend) buffer the whole reply before sending it, and it started bothering me…

The theory

So what’s the idea behind this?

Imagine the typical setup: You have a page, with CSS and all the usual stuff. Usually the “header” of the page is relatively static: There is just the title, meta tags, stylesheets and such.

Then, after the header, you usually have the contents and such. They are fetched from the database or such, and usually it may take a bit to query the DB, process the results and so on.

Then you have the footer.

Now, as we can see, the header could be sent to the browser even before the database is being queried for content. This could improve the speed the page is loaded and the perceived performance. The user’s browser could already start loading things like CSS, and your site would be querying and processing data the same time, and it could then send more data when it’s done doing its thing.

This is what frameworks usually aren’t doing. They do all the heavy processing before sending out a single byte.

In practice

In the case of Zend Framework, this means we will have to hook in the whole process of what’s going on behind the scenes. ZF is sticking all the HTML and such in the response object. The last thing which happens in the process is the response being sent to the browser.

Now, we could just take the response object and extend it so, that it immediately sends out any data put in the response’s body. In theory this could work, but there are some problems caused by the way ZF works.

The problems

  • The response is only getting any data in the body after the heavy processing is done. You would have to manually put something in there, even before any action controllers are called.
  • Zend_Layout uses the response’s body to insert data in the layout
  • View helpers like HeadTitle and HeadMeta would lose their purprose, as when you send the header part of your page, you no longer would have any control over what’s in there.

Implementation ideas

To make something like this feasible, the framework would need some kind of a method to tell when the content is ready to be delivered.

- I’m done modifying the header. Please send it to the client.

Doing this would be possible by perhaps modifying Zend_Layout and the response class so, that they could support flushing the output buffer. You could modify the page’s title and such in your action, before actually performing any possibly slow operations, like DB queries.

The response class would need to be extended or modified to support something like response segments, which would allow you to send the response in segments: First maybe just the HTTP headers, then perhaps the initial html, head, title and link elements, then the actual content fetched from the DB and the footer.

To support this, you would need to be able to define segments in the layouts. I’m not sure how this would be best implemented, but you would need to be able to tell which parts of the layout can be sent beforehand. For example, you could just do your HeadTitle modifications in your action, then call something like $this->_response->sendSegment(‘header’);, which would send all the unsent segments before header and then the header segment.

However, implementing something like that in an efficient and easy to use manner would not be very simple. If it was, I’d be providing you some code right here to do it, and not just talking about it. The first thing to spring to mind is that if it worked like I outlined above, you would need to put sendSegment(‘header’) in all of your actions so that it would get sent before any heavy processing…

Conclusion, for now

Seeing that Yahoo does not provide any actual data on how much the performance is improved by sending data early, the problems caused by that and the difficulty of implementing things like layouts and the head-helpers, greatly seem to outweight the benefits of such optimization methods.

Share this:

RSS feed Subscribe to my RSS feed

About the author

Jani is a 15 year veteran of the software industry. He's currently available for consulting

  1. 8 Responses to “Front-end performance and frameworks”

  2. I have read that yahoo article too and was thinking the same things you did. First of all, it’s not that hard implementing those modifications in the ZF. Although there are multiple levels of output buffering, they all merge back to the dispatcher (assuming you are using the front controller classes). You can either turn off output buffering by setting the parameter ‘disableOutputBuffering’ or implementing the flushing routine into a view helper and calling it where you want content to be flushed to the browser (on the latest ZF 1.5, the flushing routine appears at line 285 in Controller/Dispatcher/Standard.php).

    Furthermore, it will not have to appear in all your actions, rather it should not appear in any action at all. If you are using a layout, then optimally after you finish building your document’s ‘head’ tag, you would flush directly from the layout.

    It remains to be seen if that is indeed a recommended practice – pages that are flushed incorrectly can show very messy results if the user decides to hit refresh or navigation away between flushes (I have seen ASP code spilled into a page while hitting refresh more than once… never seen it with PHP though).

    Might be worth a try though

    By Eran Galperin on Mar 24, 2008

  3. Actually, if you look at the process, the layout does not get rendered until postDispatch, thus flushing from the layout would have very little effect.

    You would need to modify the way the whole layout works to get the effect: the layout would have to be ran *before* any actions get called, and before any processing happens in the actions, so that you could still modify it with the head helpers and so that it could still send whatever is available “beforehand”.

    By Jani Hartikainen on Mar 24, 2008

  4. Actually I am not using the viewRenderer since it is quite annoying. I have extended the Action Controller and overloaded the render() method to do all the piecing together of different layout pieces, and I call it manually when I want an action to output something (not all actions send output). If you are using the viewRenderer then I guess you are correct that it would happen in postDispatch.

    In any case, you would still want the headers to be called after the dispatch has been sent to the controller/action (and even module) since there could be (probably are) changes in the head elements from controller to controller and action to action (thats the whole point of using the head helpers – to modify the head for different renders. Otherwise you would just set them once and forget about them)

    By Eran Galperin on Mar 24, 2008

  5. Actually, the ViewRenderer is not the issue here at all. The ViewRenderer writes to the response object, plain and simple — which is really no different than if you echo from a controller (as output buffering is on by default).

    However, if you use a Two Step View pattern in an MVC application — Zend_Layout or some other component — you’re going to run into this issue. The whole idea of a Two Step View is that you render an application view, and then inject it into another view. With Zend_Layout, this gives us some pretty cool flexibility, as you can then set hints in your application view logic that will affect the layout — such as indicating what scripts or alternate CSS files to load, or changing the page title on the fly. The problem, of course, is that this means that you *must* spit out the entire content at once, because your layout cannot render until it has all information it can possibly get.

    An alternate approach is to use a single action for every request, and have your view object then call out to other actions using the new action() helper. This would give you the ability to easily build dynamic pages, but at the cost of not being able to alter what ends up in your document head easily.

    I personally am not convinced that flushing early is going to help that much. Most server-side processing is taking place in fractions of a second — and those fractions decrease dramatically when you introduce good caching strategies. As a result, you’re *already* flushing pretty much as early as potentially possible — and giving a full page immediately, which will help reduce visual artifacts caused by JS and CSS that have not completely loaded or triggered.

    By Matthew Weier O'Phinney on Mar 25, 2008

  6. I have been wondering this myself also …

    @Matthew: this would be really useful when you’re doing some heavy, time-consuming server-side processing, or when you’re waiting on a third party system for responses. I know you can just render a ‘loading’ page then use Ajax to poll for updates, but still … you never had to with old-skool php.

    Eran suggested above that this could be achieved in a View Helper.

    I just had a try. It worked for me (try calling “$this->flush(); sleep(5);” in a view script), but now I’ve tried it I’m pretty certain it’s not the *right* way to do things ..

    <?php
    class My_View_Helper_Flush
    {
    function flush()
    {
    $level= ob_get_level();
    $content=”;
    for($i=0;$i<$level;$i++)
    {
    $content.= ob_get_contents();
    ob_end_clean();
    }
    echo $content;
    flush();
    for($i=0;$i<$level;$i++)
    {
    ob_start();
    }
    }
    }

    @Eran: Is that the kind of thing you meant? It works, but seems all wrong to me. For one thing, you’d lose the headers that were set in the response object (View Helpers and Views have no access to the Response object).

    I would sooner make two ‘render’ calls in my action controller, implementing a similar flush() function, to call in between these calls …

    Like so:

    getResponse()->sendResponse();
    $this->getResponse()->clearBody();
    $level= ob_get_level();
    $content=”;
    for($i=0;$i<$level;$i++)
    {
    $content.= ob_get_contents();
    ob_end_clean();
    }
    echo $content;
    flush();
    for($i=0;$irenderScript(‘headerandsomecontent.phtml’);
    $this->flush();

    sleep(3); //heavy processing or polling or whatever goes here
    $this->renderScript(‘latercontentandfooter.phtml’);
    }

    }

    The flush method would live in an abstract controller, but i plonked it in the concrete controller for the sake of brevity.

    Ofcourse, this is still incompatible with Zend_Layout and placeholders, and still a bit hacky, but I kindo like it.

    This is just a first attempt so your input is most welcome (particularly Mr O’Phinney!)

    By Amir Laher on Mar 30, 2008

  7. Balls! WordPress cut off the beginning of my Controller class. Here it is again:

    class IndexController extends Zend_Controller_Action
    {
    protected function flush()
    {
    $this->getResponse()->sendResponse();
    $this->getResponse()->clearBody();
    $level= ob_get_level();
    $content=”;
    for($i=0;$i<$level;$i++)
    {
    $content.= ob_get_contents();
    ob_end_clean();
    }
    echo $content;
    flush();
    for($i=0;$irenderScript(‘headerandsomecontent.phtml’);
    $this->flush();
    sleep(3); //heavy processing or polling or whatever goes here
    $this->renderScript(‘latercontentandfooter.phtml’);
    }

    }

    By Amir Laher on Mar 30, 2008

  8. Sorry about this – it’s not escaping *some of* my pointy brackets (and ignoring my indentation) – what can you do eh?

    Here it is one last time. After this I’m giving up!:

    class IndexController extends Zend_Controller_Action
    {
    protected function flush()
    {
    $this->getResponse()->sendResponse();
    $this->getResponse()->clearBody();
    $level= ob_get_level();
    $content=”;
    for($i=0;$i<$level;$i++)
    {
    $content.= ob_get_contents();
    ob_end_clean();
    }
    echo $content;
    flush();
    for($i=0;$i<$level;$i++)
    {
    ob_start();
    }
    }

    public function indexAction()
    {
    $this->renderScript(‘headerandsomecontent.phtml’);
    $this->flush();
    sleep(3); //heavy processing or polling or whatever goes here
    $this>renderScript(‘morecontentandfooter.phtml’);
    }
    }

    By Amir Laher on Mar 30, 2008

  9. Yep, something like that. It feels like a ‘hack’ I know, and personally I wouldn’t go to such lengths to send the ‘head’ section first since:
    a. I am gzipping everything and this would complicate even more the headers handling
    b. It’s not at all obvious how much of a performance increase this would result in

    In my opinion, if you have a one or two pages that get very heavy load, you could implement custom rendering when needed to help performance. Those pages would probably need other performance tunings (= hacking) anyway if they incur such a heavy load.
    The framework overall is very good for the majority of pages, but it wasn’t built for performance (the cost of abstraction).

    By Eran Galperin on Mar 30, 2008

Post a Comment

You can use some HTML (a, em, strong, etc.). If you want to post code, use <pre lang="PHP">code here</pre> (you can replace PHP with the language you are posting)