At PyCon 2013 I saw a presentation, with a common function signature:
def call_later(when, function, *args):
...
This got me thinking about some guidelines I wrote recently on our internal tech blog about how to write such proxy functions. The current recommendation I have is for a different signature, for the reason I shall now explain:
Let’s say that you have a function that calls another function for some reason. You start with something like this:
def mywrapper(func, *args, **kwargs): do_something() return func(*args, **args)
At some point though, you add another higher level wrapper:
def mybigwrapper(func, *args, **kwargs): do_something() return mywraper(func, *args, **args)
This is ok, until someone notices that this is rather slow. The reason is, that arguments are constantly being packed and unpacked. Unnecessarily so, because no one is really looking at them. So a clever software engineer comes up with a solution:
def mywrapper(func, *args, **kwargs): return mywrapper_without_the_stars(args, args) def mywrapper_without_the_stars(func, args, kwargs): do_something() return func(*args, **args) def mybigwrapper(func, *args, **kwargs): do_something() return mywraper_without_the_stars(func, args, args)
What has happened? Yes, we have created a set of functions that do not take variable arguments, but rather just take the argument tuple and keyword dict. When you nest a number of those, there is no argument packing and unpacking going on and they are all passed through verbatim. We then have a thin layer outside that does the argument packing, for api backwards compatibility.
But there is a lesson here: Perhaps it is not such a good idea to do this style of interface in the first place. Why didn’t we just write:
def mywrapper(func, args=(), kwargs={}):
do_something()
return func(*args, **args)
to begin with? In my opinion, this is actually a much better interface. To illustrate, lets say that we want to wrap a call to myfunc(1,2,3). Compare these two styles:
return mywrapper(myfunc, 1, 2, 3)
return mywrapper(myfunc, (1, 2, 3))
In the former case, we are mixing the callable (myfunc) and its arguments (1, 2, 3) into one big list. This doesn’t really make the distinction that “myfunc” is the callable and “1″ is its first argument, but rather they look semantically to be equivalent, as if they were all just a chunk of arguments. In my opinion it is much clearer, when using this sort of proxy functions, to make a distinction between the callable and its arguments.
Therefore, this is currently the recommended way within CCP to write such wrappers. They take the argument tuple (and keyword dict) as a non-variable argument to the function. Variable argument lists are only used in two cases:
- When writing a function where that is appropriate, such as logging functions
- When writing wrapper functions that emulat other function’s signature.
But recently, I have been thinking even more about this because passing around “args” and “kwargs” everywhere seems unnecessarily clunky. And we arrive at the thesis of this blog post:
Wrapper functions should be written and used like this:
# wrapper takes an argument-less callable
def mywrapper(func):
do_something()
return func()
# call myfunc with default args
a = mywrapper(myfunc)
# call myfunc with some arguments
a = mywrapper(lambda : myfunc(1, 2, 3))
# call myfunc with something from this context
def call():
return myfunc(foo, bar)
a = mywrapper(call)
In other words: How about using Python’s powerful lambda and closure semantics to add those arguments if and when they are needed, rather than to write layer upon layer of functions that manually carry around argument tuples and keyword dicts?
Recent Comments