Chapter 11, The Quick Python Book: Modules and Scoping rules

11. 7 Python scoping rules and namespaces

The core concept here is that of a namespace. A namespace in Python is a mapping from identifiers to objects and is usually represented as a dictionary. When a block of code is executed in Python it will have three namespaces: local, global, and built-in (figure 11.2).

When an identifier is encountered during execution, Python first looks in the local namespace for it. If it is not found, the global namespace is looked in next. If it still has not been found the built-in namespace is checked. If it does not exist there, this is considered an error and a NameError exception occurs.

For a module, a command executed in an interactive session or a script running from a file, the global and local namespaces are the same. The creation of any variable or function or importing anything from another module will result in a new entry or binding being made in this namespace.

However, when a function call is made, a local namespace is created and a binding is entered in it for each parameter of the call. A new binding is then entered into this local namespace whenever a variable is created within the function. The global namespace of a function is the global namespace of the containing block of the function (that of the module, script file, or interactive session). It is independent of the dynamic context from which it is called and there is no nested scoping.

In all of the above situations, the built-in namespace will be that of the __builtin__ module. This is the module that contains, among other things, all the built-in functions we've encountered (such as len, min, max, int, float, long, list, tuple, cmp, range, str, and repr) and the other built-in classes in Python such as the exceptions (like NameError).

One thing that sometimes catches new Python programmers is the fact that you can override items in the built-in module. If, for example, you created a list in your program and put it in a variable called list, you would not subsequently be able to use the built-in list function. The entry for your list would be found first. There is no differentiation between names for functions and modules and other objects. The first occurrence of a binding for a given identifier will be used.

Enough talk, time to explore this with some examples. We use two built-in functions, locals and globals. They return dictionaries containing the bindings in the local and global namespaces respectively.

Starting a new interactive session:

 
>>> locals() 
{'__ doc__': None, '__ name__': '__ main__', '__ builtins__': 
     <module '__ builtin__'>} 
>>> globals() 
{'__ doc__': None, '__ name__': '__ main__', '__ builtins__': 
     <module '__ builtin__'>} 
>>> 

The local and global namespaces for this new interactive session are the same. They have three initial key/ value pairs that are for internal use: (1) an empty documentation string __doc__, (2) the main module name __name__ (which for interactive sessions and scripts run from files is always __main__), and (3) the module used for the built-in namespace __builtins__ (the module __builtin__). Now, if we continue by creating a variable and importing from modules, we will see a number of bindings created:

>>> z = 2 
>>> import math 
>>> from cmath import cos 
>>> globals() 
{'math': <module 'math'>, '__doc__': None, 'z': 2, 'cos': 
<built-in function cos 
>, '__name__': '__main__', '__builtins__': <module '__builtin__'>} 
>>> locals() 
{'math': <module 'math'>, '__doc__': None, 'z': 2, 'cos': 
<built-in function cos>, '__name__': '__main__', '__builtins__': 
<module '__builtin__'>} 
>>> math.ceil(3.4) 
4.0 

As expected, the local and global namespaces continue to be equivalent. Entries have been added for z as a number, math as a module, and cos from the cmath module as a function.

We can use the del statement to remove these new bindings from the namespace (including the module bindings created with the import statements).

 
>>> del z, math, cos
>>> locals() 
{'__doc__ ': None, '__name__ ': '__main__ ', '__builtins__ ': 
<module '__builtin__ '>} 
>>> math. ceil(3.4) 
Traceback (innermost last): 
File "<stdin>", line 1, in ? 
NameError: math 
>>> import math 
>>> math. ceil(3.4) 
==> 4 

The result was not drastic, as we were able to import the math module and use it again. Using del in this manner can be handy when in the interactive mode. ( Using del and then import again will not pick up changes made to a module on disk. It is not actually removed from memory and then loaded from disk again. The binding is simply taken out of and then put back in your namespace. You still need to use reload if you want to pick up changes made to a file.)

For the trigger happy, yes it is also possible to use del to remove the __doc__, __main__, and __builtins__entries. But resist doing this, as it would not be good for the health of your session!

Now let's take a look at a function created in an interactive session:

 
>>> def f( x): 
...   print "global: ", globals() 
...   print "Entry local: ", locals() 
...   y = x 
...   print "Exit local: ", locals() 
... 
>>> z = 2 
>>> globals() 
{'f': <function f at 793d0>, '__doc__': None, 'z': 2, ' 
__name__': '__main__ ', '__builtins__': <module '__builtin__ '>} 
>>> f(z) 
global: {'f': <function f at 793cd0>, '__doc__': None, ' 
z': 2, '__name__': '__main__', '__builtins__': <module 
'__builtin__'>} 
Entry local: {'x': 2} 
Exit local: {'x': 2, 'y': 2} 
>>> 

If we dissect this apparent mess, we see that, as expected, upon entry the parameter x is the original entry in f's local namespace but y is added later. The global namespace is the same as that of our interactive session, as this is where f was defined. Note that it contains z, which was defined after f.

In a production environment we will normally be calling functions that are defined in modules. Their global namespace will be that of the module they are defined in. Assume we've created the following file:

File scopetest. py 
""" scopetest: our scope test module""" 
     v = 6 
 def f( x): 
     """ f: scope test function""" 
     print "global: ", globals(). keys() 
     print "entry local:", locals() 
     y = x 
     w = v 
     print "exit local:", locals() 

Note that we will be only printing the keys (identifiers) of the dictionary returned by globals. This will reduce the clutter in the results. It was very necessary in this case due to the fact that in modules as an optimization, the whole __builtin__ dictionary is stored in the value field for the __builtins__ key.

 
>>> import scopetest 
>>> z = 2 
>>> scopetest. f( z) 
global: ['v', '__doc__', 'f', '__file__', '__name__ ', 
'__builtins__'] 
entry local: {'x': 2} 
exit local: {'w': 6, 'x': 2, 'y': 2} 

The global namespace is now that of the scopetest module and includes the function f and integer v (but not z from our interactive session). Thus, when creating a module you have complete control of the namespaces of its functions.

We've now covered local and global namespaces. Next, let's move on to the built-in namespace. We'll introduce another built-in function, dir, which, given a module, returns a list of the names defined in it.

 
>>> dir(__builtins__) 
==> ['ArithmeticError', 'AssertionError', 'AttributeError', 
'EOFError', 'Ellipsis', 'EnvironmentError', 'Exception', 
'FloatingPointError', 'IOError', 'ImportError', 'IndexError', 
'KeyError', 'KeyboardInterrupt', 'LookupError', 'MemoryError', 
'NameError', 'None', 'NotImplementedError', 'OSError', 
'OverflowError', 'RuntimeError', 'StandardError', 
'SyntaxError', 'SystemError', 'SystemExit', 'TypeError', 
'ValueError', 'ZeroDivisionError', '_', '__debug__', '__doc__', 
'__import__', '__name__', 'abs', 'apply', 'callable', 'chr', 
'cmp', 'coerce', 'compile', 'complex', 'delattr', 'dir', 
'divmod', 'eval', 'execfile', 'exit', 'filter', 'float', 
'getattr', 'globals', 'hasattr', 'hash', 'hex', 'id', 'input', 
'int', 'intern', 'isinstance', 'issubclass', 'len', 'list', 
'locals', 'long', 'map', 'max', 'min', 'oct', 'open', 'ord', 
'pow', 'quit', 'range', 'raw_ input', 'reduce', 'reload', 'repr', 
'round', 'setattr', 'slice', 'str', 'tuple', 'type', 'vars', 
'xrange'] 

There are a lot of entries here. Those ending in Error and System Exit are the names of the exceptions built-in to Python. These will be discussed in "Exceptions" (chapter 14).

The last group (from abs to xrange), are built-in functions of Python. We have already seen many of these in this book and will see more. However, they won't all be covered here. When interested, you can find details on the rest in the Python Library Reference. You can also at any time easily obtain the documentation string for any of them:

>>> print max.__doc__ 
max(sequence) -> value 
max(a, b, c, ...) -> value 
With a single sequence argument, return its largest item. 
With two or more arguments, return the largest argument. 
>>> 

As mentioned earlier, it is not unheard of for a new Python programmer to inadvertently override a built-in function.

>>> list("Peyto Lake") 
['P', 'e', 'y', 't', 'o', ' ', 'L', 'a', 'k', 'e'] 
>>> list = [1,3,5,7]
>>> list("Peyto Lake") 
Traceback (innermost last): 
  File "<stdin>", line 1, in ? 
TypeError: call of non-function (type list) 

The Python interpreter will not look beyond our new binding for list as a list, even though we are using function syntax.

The same thing will of course happen if we try to use the same identifier twice in a single namespace. The previous value will be overwritten, regardless of its type:

 
>>> import string 
>>> string = "Mount Rundle" 
>>> string.split("Bow Lake") 
Traceback (innermost last): 
  File "<stdin>", line 1, in ? 
AttributeError: 'string' object has no attribute 'split' 

Once aware of this, it isn't a significant issue. Reusing identifiers, even for different types of objects, wouldn't make for the most readable code anyway. If we do inadvertently make one of these mistakes when in interactive mode, it's easy to recover. We can use del to remove our binding, to regain access to an overridden built-in, or import our module again, to regain access.

>>> del list
>>> list("Peyto Lake") 
==> ['P', 'e', 'y', 't', 'o', ' ', 'L', 'a', 'k', 'e'] 
>>> import string 
>>> string.split("Bow Lake") 
==> ['Bow', 'Lake'] 

The locals and globals functions can be quite useful as simple debugging tools. The dir function doesn't give the current settings but if called without parameters, it returns a sorted list of the identifiers in the local namespace. This will help catch the mistyped variable error that compilers may usually catch for you in languages that require declarations:

>>> x1 = 6 
>>> xl = x1 -2 
>>> x1 
==> 6
>>> dir() 
==> ['__builtins__', '__doc__', '__name__', 'x1', 'xl'] 

The debugger that is bundled with IDLE has settings where you can view the local and global variable settings as you step through your code, and what it displays is the output of the locals and globals functions.

Daryl Harms holds a Ph.D. in computer science. He has been working on the design and the development (or the management of the development) of small and large software systems since the mid-1980's. He is currently a software development consultant working out of Calgary, Alberta and can be reached at ddharms@yahoo.com

Kenneth McDonald is a longtime programmer/analyst and advocate for free software. He holds the B.Sc. and M.Sc. in computer science and has been, at various times, a Unix system administrator, independent software developer, and investor. His most recent position was with the Washington University School of Medicine, where he worked as part of the Human Genome Project as a programmer/analyst.