Lazy Function Definition Pattern
This article examines a functional-programming design pattern I've been calling Lazy Function Definition. I've repeatedly found this pattern useful in JavaScript especially when writing cross-browser libraries that are efficient at runtime.
Warm-Up Problem
Write a function foo that returns a Date object that holds the time that foo was first called.
Solution #1: Ancient Technology
This simplistic solution uses a global variable t to hold the Date object. The first time foo is run it stores the time in t. In subsequent executions, foo just returns the value stored in t.
var t;
function foo() {
if (t) {
return t;
}
t = new Date();
return t;
}The above code has two problems. First, the variable t is an extra global variable and could be altered between calls to foo. Second, at runtime the code is not optimally efficient because the conditional must be evaluated each time foo is called. In this example, evaluating the conditional is inexpensive but in real-world examples there are often several expensive conditionals in a if-else-else-... structure.
Solution #2: The Module Pattern
We can remedy one problem in the first solution using the Module Pattern attributed to Cornford and Crockford. By using a closure we can hide the global t variable so that only the code in foo has access.
var foo = (function() {
var t;
return function() {
if (t) {
return t;
}
t = new Date();
return t;
}
})();This is still not optimal at runtime because the conditional must be evaluated for each call to foo.
The module pattern is very powerful tool but the wrong one in this case, I believe.
Solution #3: Functions are Objects
By recognizing JavaScript functions are objects that can have properties, we can achieve a similar quality solution as the module pattern solution.
function foo() {
if (foo.t) {
return foo.t;
}
foo.t = new Date();
return foo.t;
}The fact that function objects can have properties results in very clean solutions in some situations. In my mind, this solution is conceptually simpler than the module pattern solution for this situation.
This solution avoids the t global variable of the first solution; however, the conditional is still run with each execution of foo.
Solution #4: Lazy Function Definition
And now, the reason you all came...
var foo = function() {
var t = new Date();
foo = function() {
return t;
};
return foo();
};When foo is called the first time, we instantiate a new Date object and reassign foo to a new function which has that Date object in it's closure. Then before the end of the first call to foo the new function value of foo is called and supplies the return value.
Subsequent calls to foo simply return the value of t that is stored in it's closure. This is a fast lookup and efficient especially if the conditionals used in the previous solutions are many and complex.
Another way of thinking about this this pattern is that the outer function that is first assigned to foo is a promise. It promises that the first time it is run it will redefine foo to a more useful function. The term "promise" loosely comes from Scheme's lazy evaluation mechanism. Any JavaScript programmer really ought to study Scheme as there is more written about functional programming for Scheme then exists for JavaScript.
Determining Page Scroll
When writing cross-browser JavaScript, frequently several different browser-specific algorithms are wrapped by a single JavaScript function name. This normalizes browser APIs by hiding browser discrepancies and makes building and maintaining complex page-specific JavaScript much simpler. When the wrapper function is called, the appropriate browser-specific algorithm must be run.
In a drag and drop library it is usually necessary to use the cursor location information supplied by mouse events. Mouse events give cursor coordinates relative to the browser window not the page. Adding the amount by which the page has been scrolled to the mouse's window coordinates gives the mouse's page coordinates. So we need a page scroll reporting function. For demonstration, this example defines a function called getScrollY. Since drag and drop libraries are working continuously during a drag, our getScrollY must be as efficient as possible.
Unfortunately there are three different browser-specific page scroll reporting algorithms. Richard Cornford wrote about these four algorithms in his feature detection article. The big catch is that one of the four page scroll reporting algorithms uses document.body. JavaScript libraries are usually loaded in the <head> of an HTML document and the document.body property does not exist at that time. So we can't use feature detection to determine which of the four algorithms to use when the library is loaded.
Given these problems most JavaScript libraries do one of two things. The first option is to browser sniff navigator.userAgent and create an efficient, minimal getScrollY specific to the browser. Browser sniffing is repugnant because it is brittle and error prone. The second and far better option is to use feature detection to determine the correct algorithm each time getScrollY is run. This second option is not efficient, however.
The good news is that getScrollY in a drag and drop library will not be used until the user interacts with elements in the page. If elements in the page exist then document.body property will also exist. The first time getScrollY is run, we can use the Lazy Function Definition pattern in combination with feature detection to create an efficient getScrollY.
var getScrollY = function() {
if (typeof window.pageYOffset == 'number') {
getScrollY = function() {
return window.pageYOffset;
};
} else if ((typeof document.compatMode == 'string') &&
(document.compatMode.indexOf('CSS') >= 0) &&
(document.documentElement) &&
(typeof document.documentElement.scrollTop == 'number')) {
getScrollY = function() {
return document.documentElement.scrollTop;
};
} else if ((document.body) &&
(typeof document.body.scrollTop == 'number')) {
getScrollY = function() {
return document.body.scrollTop;
}
} else {
getScrollY = function() {
return NaN;
};
}
return getScrollY();
}Just as a side note, the above may seem to be a surprisingly large effort to determine page scroll. Many people and libraries are satisfied with the following attempt to determine page scroll. This technique is even mentioned in the comments of this article.
var getScrollY = function() {
return window.pageYOffset ||
(document.documentElement && document.documentElement.scrollTop) ||
(document.body && document.body.scrollTop);
}With the above code, if the page has not been scrolled and either of the first two options are appropriate and document.body.scrollTop is undefined then the function returns undefined rather than 0. This make the getScrollY function insufficient for checking if the browser is capable of scroll reporting.
Summary
The Lazy Function Definition pattern has allowed me to write some dense, robust, efficient code. Each time I encounter this pattern I take a moment to admire JavaScript's functional programming abilities.
JavaScript supports both functional and object-oriented programming. There are book shelves worth of object-oriented design pattern books available that can be apply to JavaScript programming. Unfortunately there are few books with examples of functional programming design patterns. It will take time for the JavaScript community to aggregate a collection of good functional patterns.
Translations Chinese, Russian, Ruby
Update August 13, 2007 With Firefox/Safari/Opera getter methods it is possible to simulate lazy definition for properties that aren't functions.
this.__defineGetter__("foo", function() {
var t = new Date();
this.__defineGetter__("foo", function() {
return t;
});
return t;
});
// To the user, foo appears as a plain old
// non-function valued property of the global object.
console.log(this.foo);
setTimeout(function(){console.log(this.foo);}, 3000);Update August 15, 2007 If you are interested in this type of pattern Oliver Steel's JavaScript Memoisation Article is well worth reading. He explores other uses of this pattern avoiding repeated expensive computations in situations outside feature detection.
Update August 17, 2007 In his comment below, FredCK (of FCKeditor) makes a well reasoned argument that closures and memory leaks must be considered when implementing the lazy function definition pattern. The examples I have used in the article do not suffer from these problems but naive application of this pattern (or any JavaScript using closures) could suffer from memory leaks.
Update August 17, 2007 There are also reader comments on Ajaxian and reddit
Update August 22, 2007 Added an aside to the article about why a common and shorter version of getScrollY is not enough.
Update September 12, 2007 Richard Cornford posted this pattern to Usenet's comp.lang.javascript under the name The Russian Doll Pattern at least as early as August 3, 2004.
Update November 9, 2007 A variation of lazy function definition with reset.
Comments
Have something to write? Comment on this article.
#2 and #4 are the same. To see it write #2 as:
var foo = (function() {
var t = new Date();
return function() {
return t;
}
})();Bill,
In the version you are proposing, calls to foo will return a Date object that corresponds to the time when foo was first defined, not when foo was first called.
Great idea, though extending the function looks easy, too.
Utterly wonderful. The idea of functions redefining themselves hadn't quite clicked in my head up until this point.
Can I ask you for permission to translate this article in non-commercial use (into russian)?
shabunc,
If you would like to translate it please do. If you send me a link I will post it here also. It will be interesting to see this translated into Russian!
thank you. as soon as i'll translate it (probably, tomorrow), i'll inform you.
I was about to rip this apart, suggesting that there was no purpose to re-defining the function on the fly... until I re-read the bit about the document.body being available, and I thought a bit more about "constants" and properties within a function.
Quite clever. I suppose this would be being very picky, but I wonder what the overhead of the "first call" is, to redefine the function. I would think it would be minor, and a worthwhile expense, as long as the function will be called 2 or more times.
Seems like a lot of work for a simple page scroll. Most libraries don't sniff that much.
var getScrollY = function() { return window.pageYOffset || document.documentElement.scrollTop || document.body.scrollTop; }
Don't forget about references to the function. If you have a code similar to this, it won't work:
var foo = function() {
var t = new Date();
foo = function() {
return t;
};
return foo();
};
bar = foo;
alert( foo() );
alert( foo() );
alert( bar() );
alert( bar() );Each time bar() is called, it will redefine foo and return a new Date object.
Another option:
var foo = function() {
return arguments.callee.t || (arguments.callee.t = new Date);
}Second bit will only be executed the first time, so it's actually quite efficient, and you're not dependent on knowing the function's name to do it, which kinda helps in refactoring :)
Nice! You might also be interested in this, which is basically the same thing for methods definitions.
Bezier.prototype.getLength = function() {
var length = ... // expensive computation
this.getLength = function(){return length};
return length;
}Nick, if document.documentElement does not exist in your example then trying to access the scrollTop property will cause an error.
Romulo, JavaScript always gives the user enough rope to hang himself. In the warm-up problem you have shown a weakness. In the real world problems I've used this, like the getScrollY example, in calling the equivalent of your foo() would have the efficiency benefits discussed and calling your bar would be slower but wouldn't cause any damage.
Jonas, Indeed. Your example is nice and short and you don't refer to foo twice. I debated about using a a couple different compact forms for the warm-up exercise but solutions in real examples never seem take the same compact forms.
Oliver, Your example is a great variant and really interesting because Bezier.prototype.getLength() is never redefined but rather it is shadowed by a getLength() property on a Bezier instance. Very nice.
Congrats Peter. As you could see in the comments (here and elsewhere), not everybody got the idea yet, but this is an interesting approach, and certainly quite useful.
For shortnesses, the function can be returned on assignment, like:
var foo = function() {
var t = new Date();
return (foo = function() {
return t;
})();
};Peter, the error you are talking about will never happen because one of the other vars will have picked up a value before you even get to scrollTop. If the first two values haven't picked up a value, it's safe to use scrollTop.
Another one-liner which avoids any possible null object errors.
var getScrollY = function() { return window.pageYOffset || (document.documentElement && document.documentElement.scrollTop) || (document.body && document.body.scrollTop); }you may not need it very often, but this is one really strong pattern
You can combine some of these ideas to get this:
var test = function () {
if (arguments.callee.f) {
return arguments.callee.f();
} else {
alert('function creation');
var t = new Date();
arguments.callee.f = function() {
return t;
};
return t;
}
}
var testLazyFunDef = test;This will cost you an extra "if" statement with guaranteed constant (quick) time, and in return give you the ability to assign one function reference to another, as many times as you like, and still get the same behavior, no matter which function reference you choose to call later on.
I have no idea why anyone would need something this strict. I think Peter's solution is good enough. But if you absolutely insist on being able to assign one function reference to another, you can still do it.
Leo, Interesting combination. Your combination was based on the "Functions are Objects" example. I thought I'd try your idea based on the "Module Pattern" example...
var test = (function() {
var f, t;
return function() {
if (f) {
return f();
}
t = new Date();
f = function() {
return t;
}
return t;
}
})();Peter,
Nice. How about this to make it nicer to use?
function makeLazyDef(decideWhichFun) {
var f;
return function() {
if (f) {
return f();
}
f = decideWhichFun();
return f();
};
}
var test2 = makeLazyDef(function() {
var t = new Date();
// expensive if statements are here
alert('fun creation');
return function() {
return t;
}
});
var test3 = test2;
var test4 = makeLazyDef(function() {
alert('deciding what to use...');
if (typeof window.pageYOffset == 'number') {
return function() {
return window.pageYOffset;
};
} else if ((typeof document.compatMode == 'string') &&
(document.compatMode.indexOf('CSS') >= 0) &&
(document.documentElement) &&
(typeof document.documentElement.scrollTop == 'number')) {
return function() {
return document.documentElement.scrollTop;
};
} else if ((document.body) &&
(typeof document.body.scrollTop == 'number')) {
return function() {
return document.body.scrollTop;
}
} else {
return function() {
return NaN;
};
}
});
var test5 = test4;Brilliant article. Whilst I've often defined the same function multiple times depending on certain conditions, I'm sure the lazy pattern will really help when creating fast event-driven code.
It just goes to show how misunderstood JavaScript is - developers seem to be finding new techniques every day.
Great namespace technique!
Here is a python version:
def bar():
from datetime import datetime
t = datetime.now()
global bar
def bar():
return t
return bar()It seems that by using this technique, there is no need to import packages/facilities into global namespace for one-usage-tasks.
hey, great example of the use of closures to solve real problems cleanly. Thanks for writing it.
I'm sorry, but your pattern is just a anti-pattern. It's not efficient because compare to the solution #3, it will not get any significant improvement of performance. It's not robust, on the contrary, using this pattern without carefulness will cause memory leak, oh no, I'm not talking about the bug of IE. It will leak memory because the objects (for example, the intermediate objects in a complex calculation) of the original function will be referred by the new closure and never be GC. In fact, they have no chance to be recycled except assigning them from null in the original function-Oops, setting null manually is not a lovely pattern, isn't it? At last, it it dense? I have no idea... Here is the Chinese translation of your post: http://realazy.org/blog/2007/08/16/lazy-function-definition-pattern/ And I have written a more complete analysis, you can read it if you know Chinese ;)
Rethinking, the only problem with your solution is that we would be filling the memory with variables that would never be released, just because of the closures.
1# Case
Suppose "foo" should return a static value, result of an intensive and complex code.
Using the proposed pattern, we would have something like this:
var foo = function() {
var result ;
var a,b,c,d,e ;
// using a,b,c,d,e to fill return
foo = function() {
return result;
};
return foo();
};In the above example, "a,b,c,d,e" will never get released just because of our closure, even if not anymore needed. We just need "result".
I was tempted to mix your solution with Jona's proposal, which seams to bring better results regarding memory usage:
var foo = function()
{
return arguments.callee.t || (arguments.callee.t = (function(){
var result ;
var a,b,c,d,e ;
// using a,b,c,d,e to fill return
return result ;
})());
};
But, in the above example, the calculation will run for all calls if the final "result" is null, false, 0, NaN, etc. So, this is not the definitive way to go.
After further thoughts, I've also considered that, the "return a || b" approach, internally, has the same value (and possibly the same performance) as "return a ? a : b", because in both cases the compiler will have to check the value of "a" before returning it (it means that it is not true that "a" is "immediately" returned in the first case). So, this is not a real shortcut.
So, theoretically, the following could be a definitive solution for this case:
var foo = function()
{
if ( typeof arguments.callee.t != 'undefined' )
return arguments.callee.t ;
var result ;
var a,b,c,d,e ;
// using a,b,c,d,e to fill return
return ( arguments.callee.t = result ) ;
};Which means that we are back to the "Ancient Technology", not opting to use the confusing "Module Pattern", considering that "Functions are Objects", but considering also that arguments.callee.t could be any kind of "defined" valued.
2# Case
Supposing instead the "foo" should return a value that uses different algorithms, chosen after a complex calculation.
Here we have the getScrollY example again, which will have the same memory allocation issue explained in the previous case.
Just to exemplify, suppose we code in the following way:
var getScrollY = function() {
if (typeof window.pageYOffset == 'number') {
return (getScrollY = function() {
return window.pageYOffset;
})();
}
var compatMode = document.compatMode;
var documentElement = document.documentElement;
if ((typeof compatMode == 'string') &&
(compatMode.indexOf('CSS') >= 0) &&
(documentElement) &&
(typeof documentElement.scrollTop == 'number')) {
return (getScrollY = function() {
return documentElement.scrollTop;
})();
}
var body = document.body ;
if ((body) &&
(typeof body.scrollTop == 'number')) {
return (getScrollY = function() {
return body.scrollTop;
})();
}
return (getScrollY = function() {
return NaN;
})();
};In the above case, we have a complete chaos, because "compatMode", "documentElement" and "body" will be part of the closure scope forever, even if not needed in all cases (and potentially leaking).
In this case, a sane proposal, achieving the performance enhancements aimed by your pattern, would be using an old way of avoiding closures, by defining all functions outside foo>
var getScrollY = function() {
if (typeof window.pageYOffset == 'number')
return (getScrollY = getScrollY.case1)();
var compatMode = document.compatMode;
var documentElement = document.documentElement;
if ((typeof compatMode == 'string') &&
(compatMode.indexOf('CSS') >= 0) &&
(documentElement) &&
(typeof documentElement.scrollTop == 'number'))
return (getScrollY = getScrollY.case2)();
var body = document.body ;
if ((body) &&
(typeof body.scrollTop == 'number'))
return (getScrollY = getScrollY.case3)();
return (getScrollY = getScrollY.case4)();
};
getScrollY.case1 = function() {
return window.pageYOffset;
};
getScrollY.case2 = function() {
return documentElement.scrollTop;
};
getScrollY.case3 = function() {
return body.scrollTop;
};
getScrollY.case4 = function() {
return NaN;
};Here again, we have complete freedom in the getScrollY main calculations, avoiding closures. By using the "Functions are Objects" approach, just one of the "case1", "case2", "case3" and "case4" references will remain after the first call to getScrollY, freeing unneeded memory allocation too.
Conclusion
Your pattern is interesting, but must be very well controlled while coding it. The above two cases are examples where it would work, but would not perform well regarding memory management (essentially because of the closures), and the proposed patterns could give better results, achieving similar performance enhancements.
Ps.: I've written all the code in this textarea... so, please don't give much attention to possible typos.
Hi FredCK,
Thanks for your in-depth thoughts and comment. I'm glad my post is provoking thought past my simple illustrative warm-up problem to real world cases and associated problems.
In the first code block you post, it is possible to avoid memory leaks by manually freeing up the temporary variables. This is not fun or really safe because of the chance of missing something needing freeing. This feels like coding in C.
var foo = function() {
var result;
var a,b,c,d,e;
// using a,b,c,d,e to fill return
a = b = c = d = e = null;
foo = function() {
return result;
};
return foo();
};Excellent point about using the return a || b style if there is a possibility that a could be set to a "false" value meaning the expensive computation would need repeated execution.
In your fourth code block you define compatMode and documentElement which refer to objects that will never be freed anyway since they are host objects. So for this particular case there isn't any fear of a memory leak. I understand your point and in general these could be manually freed but if that is necessary then it really is a mess.
I really like your final code block example to avoid memory leaks without having to worry about memory leaks. Where the style of your last example is possible, and memory leaks are a concern, it seems like a winner.
The core idea of this article is that during the execution of a variable's function value, it is possible to change that variable's function value. I believe this idea is solid. Clearly the implementation of this is tricky in particular cases but can lead to very efficient programs.
Thanks again for your thoughts. I've posted an update to this article encouraging people to scroll down to read your implementation cautions.
PS. I'm sure you wouldn't have made any typos if my comment form had something more sophisticated then a simple textarea. ;)
I also want to point out that reset the function is not very functional pattern, because pure fp don't allow rebinding.
There may be some risk if the user doesn't know the function will change itself.
For example, John have some old codes with a function like getScrollY but named getPageScrollY. He believe yours getScrollY is better, so he decides to use your code instead, but he doesn't want to change more, so he plan to just add one line in his script:
getPageScrollY = getScrollY;
Oh, you know that, every call of getPageScrollY will always be the first call of getScrollY, which decrease the performance a lot!
It can be even worse if we consider FredCK's version...
To be carefully, John write a testcase and call getPageScrollY once and check the result, everything seems ok!
But in the real application... Suprise! You can guess what will happen. Remember, in real application, getPageScrollY will be called many times.
That's why I don't like your idea, because it's the burden of the users to understand the fact that the function will be changed silently!
@Peter
PS. I'm sure you wouldn't have made any typos if my comment form had something more sophisticated then a simple textarea. ;)
:D That was funny... Thanks for your comments!
@hax
You've pointed something to consider, but of course all proposed patterns here must be well thought for each case. The getScrollY function is "just an example" for the article. I think Peter is not really proposing a definitive solution for getScrollY, so the focus must not go there, but to the way that function has been coded (the pattern).
For example, I found the idea very useful when coding "private style" class methods. In any case, it is up to the developer to decide where to use a pattern, considering the risks of it, privileging, for example, the performance enhancements it could bring.
To conclude, I'm sure that good developers will precisely know when to use such pattern, so it is not the case to simply say that it is not a valid thing.
Hax, regarding your second comment:
I believe you and John have made risky choices migrating to the new library. In JavaScript, any identifier that has a function value now may have a different function value later or even an integer or string value later. So if you write
getPageScrollY = getScrollY;you can only expect getPageScrollY and getScrollY to have the same function value immediately after the assignment. You can't be sure what the library will do with the getScrollY identifier's value during it's lifetime. If the getScrollY library documentation makes a guarantee that getScrollY has a constant function value then your simple aliasing will be sure to work.
A more conservative and robust way you and John can achieve your goal would be to wrap the new function
getPageScrollY = function() {
return getScrollY();
};Of cousre, every developer should choose the patterns what they need and take the risk of it. What I argued here is this pattern doesn't provide any siginificent advantage (there is no performance imporovment here in my opinion). In fact, it's an anti-pattern which you should avoid in most cases.
As a js library programmmer, I am sure this pattern can't be used on any public functions of a library, because you should consider for your users first. Peter, you said "You can't be sure what the library will do with the getScrollY identifier's value during it's lifetime." From the perspective of experienced js programmers, that's true, but from the perspective of normal users and library programmers, that's totally wrong.
We are talking about practice, not the theory. I agree that the safe method to reuse a function is to wrap it instead of simple assignment. But this safe method need a extra function call, what decrease the performance. So where is your benifit? The most important thing is that though every object in JS can be changed, however, most users believe the functions should be a constant, just like many other languages. The users won't notice and understand the side effect of your silenet change of the functions. On the other hand, no library programer will document whether the function is constant as far as I know, and, in fact, they shouldn't. Because every public function SHOULD be constant in my opinion. Or you will make your users fall in hell sometimes. Document is not execuse.
At last, i'm sorry for my poor english...Please don't mind if you feel uncomfortable to my words, believe me, it's not my purpose.
The getter version would be better like this:
this.__defineGetter__('foo', function() {
delete this.foo;
return this.foo = new Date ();
});Promise is a good solution, solution 4 is not. The way in solution 4 will cause some problem when coding like this:
var a = foo;
var b = foo();
......
var c = a(); //bugs
......
var d = foo();Reading a comment above made me think of a way to avoid the binding of worthless variables in the closure, yet retain the binding of the needed value variable. Based on the example he wrote, observe the following:
var foo = function() {
var result = (function() {
var a,b,c,d,e, r;
// do stuff with vars
return r;
})();
foo = function() {
return result;
};
return foo();
};what about the functions prototype ?
function foo() { return this.t || (this.t = new Date()) }this works also:
function foo() { return t || (this.t = new Date()) }seb,
I think you mean to do something like Jonas Galvez's comment. In your version, you are appending the Date object to the this object which is not what you intended, I believe.
You can now use get to define a property getter:
var obj = {
get foo() {
var t = new Date();
this.__defineGetter__('foo', function() { return t; });
return t;
},
};The previous definition of makeLazyDef (posted 3 years ago - so this is probably moot) can get rid of the if statement.
function makeLazy(fn, self) {
return (function() {
var f;
f = function () {
var result = fn.apply(self || this, arguments);
f = function() { return result; };
return result;
};
return function() {
return f();
};
}());
}Have something to write? Comment on this article.
Great pattern!