Language differences and workarounds¶

This appendix contains a listing of the differences between Python 2 and Python 3 and example code that will run both in Python 2 and Python 3 without 2to3 conversion.

This listing is incomplete. What is listed here is only the intentional changes that are not bug fixes and even so there may be accidental omissions.

`apply()`¶

2to3 fixer ☑ six support ☐

The Python 2 builtin apply() has been removed in Python 3. It’s used to call a function, but since you can call the function directly it serves no purpose and has been deprecated since Python 2.3. There is no replacement.

`buffer()`¶

2to3 fixer ☑ six support ☐

The Python 2 buffer() builtin is replaced by the memoryview class in Python 3. They are not fully compatible, so 2to3 does not change this unless you explicitly specify the buffer fixer.

This code will run in both Python 2 and Python 3 without 2to3 conversion:

>>> import sys
>>> if sys.version_info > (3,):
...     buffer = memoryview
>>> b = buffer('yay!'.encode())
>>> len(b)
4

`callable()`¶

2to3 fixer ☑ six support ☑

The Python 2 builtin callable() was removed in Python 3.0, but reintroduced in Python 3.2. If you need to support Python 3.1 you can try to call the object under scrutiny and catch the TypeError if it is not callable.

If you need to know if something is callable without calling it, there are several solutions for Python 3:

>>> def afunction():
...     pass

>>> any("__call__" in klass.__dict__ for
...     klass in type(afunction).__mro__)
True

>>> import collections
>>> isinstance(afunction, collections.Callable)
True

If you need code that runs in both Python 2 and Python 3 without 2to3 conversion, you can use this:

>>> hasattr(bool, '__call__')
True

The six module also defines a callable function for use under Python 3.

Classes¶

2to3 fixer ☐ six support ☐

In Python 2 there is two types of classes, “old-style” and “new”. The “old-style” classes have been removed in Python 3.

Comparisons¶

2to3 fixer ☐ six support ☐

The Python 2 builtin cmp() has been removed in Python 3.0.1, although it remained in Python 3.0 by mistake. It is mostly used when defining the __cmp__ comparison method or functions to pass as cmp parameters to .sort() and the support for this has been removed in Python 3 as well.

Should you need cmp() you can define it like this:

def cmp(a, b):
    return (a > b) - (a < b)

See Unorderable types, __cmp__ and cmp for more information.

`coerce()` and `coerce`¶

2to3 fixer ☐ six support ☐

The coerce() builtin function and the support for the __coerce__ method has been removed in Python 3. coerce() would convert the numeric arguments to have the same type according to the coercion rules for Pythons arithmetic operators and was only useful in early versions of Python when implementing new numeric types. There is no replacement in Python 3; coercion should instead be done by the numeric operator methods.

Dictionary methods¶

2to3 fixer ☑ six support ☐

In Python 2 dictionaries have the methods iterkeys(), itervalues() and iteritems() that return iterators instead of lists. In Python 3 the standard keys(), values() and items() return dictionary views, which are iterators, so the iterator variants become pointless and are removed.

If you need to support both Python 2 and Python 3 without 2to3 conversion and you must use the iterator methods, you can access it via a try/except:

>>> d = {'key1': 'value1',
...      'key2': 'value2',
...      'key3': 'value3',
... }

>>> try:
...     values = d.itervalues()
... except AttributeError:
...     values = d.values()

>>> isinstance(values, list)
False

>>> for value in values:
...     print(value)
value3
value2
value1

Also, the has_key() method on dictionaries is gone. Use the in operator instead.

`except`¶

2to3 fixer ☑ six support ☐

In Python 2 the syntax to catch exceptions have changed from:

except (Exception1, Exception2), target:

to the clearer Python 3 syntax:

except (Exception1, Exception2) as target:

Other differences is that the target no longer can be a tuple and that string exceptions are gone. 2to3 will convert all this, except string exceptions.

Both syntaxes work in Python 2.6 and Python 2.7, but if you need code that is to run in earlier versions as well as Python 3 without 2to3 conversion you can get the exception object through sys.exc_info():

>>> import sys
>>> try:
...     raise Exception("Something happened")
... except Exception:
...     e = sys.exc_info()[1]
...     print(e.args[0])
Something happened

Exception objects¶

2to3 fixer ☑ six support ☐

In Python 2 the exception object is iterable and indexable:

>>> e = Exception('arg1', 'arg2')
>>> e[1]
'arg2'
>>> for a in e:
...   print a
...
arg1
arg2

In Python 3 you must use the args attribute, which will work under Python 2 as well.

>>> e = Exception('arg1', 'arg2')
>>> e.args[1]
'arg2'
>>> for a in e.args:
...   print a
...
arg1
arg2

There was also a message attribute on exceptions introduced in Python 2.5, but it was deprecated already in Python 2.6, so it’s unlikely that you will use it.

`exec`¶

2to3 fixer ☑ six support ☑

In Python 2 exec is a statement:

>>> g_dict={}
>>> l_dict={}
>>> exec "v = 3" in g_dict, l_dict
>>> l_dict['v']
3

In Python 3 exec is a function:

>>> g_dict={}
>>> l_dict={}
>>> exec("v = 3", g_dict, l_dict)
>>> l_dict['v']
3

The Python 3 syntax without the global and local dictionaries will work in Python 2 as well:

>>> exec("v = 3")
>>> v
3

If you need to pass in the global or local dictionaries you will need to define a custom function with two different implementations, one for Python 2 and one for Python 3. As usual six includes an excellent implementation of this called exec_().

`execfile`¶

2to3 fixer ☑ six support ☐

The Python 2 execfile statement is gone on Python 3. As a replacement you can open the file and read the contents:

exec(open(thefile).read())

This works in all versions of Python.

`file`¶

2to3 fixer ☐ six support ☐

In Python 2 there is a file type builtin. This is replaced with various file types in Python 3. You commonly see code in Python 2 that uses file(pathname) which will fail in Python 3. Replace this usage with open(pathname).

If you need to test for types you can in Python 3 check for io.IOBase instead of file.

`filter()`¶

2to3 fixer ☑ six support ☐

In Python 2 filter() returns a list while in Python 3 it returns an iterator. 2to3 will in some cases place a list() call around the call to filter() to ensure that the result is still a list. If you need code that runs in both Python 2 and Python 3 without 2to3 conversion and you need the result to be a list, you can do the same.

Imports¶

2to3 fixer ☑ six support ☐

In Python 2, if you have a package called mypackage and that contains a module called csv.py, it would hide the csv module from the standard library. The code import csv would within mypackage import the local file, and importing from the standard library would become tricky.

In Python 3, this has changed so that import csv would import from the standard library, and to import the local csv.py file you need to write from . import csv and from csv import my_csv needs to be changed to from .csv import my_csv. These are called “relative imports”, and there is also a syntax to import from one level up module above; from .. import csv.

If you need to support both Python 2 and Python 3 without 2to3 the from . and from .. syntax has been available since Python 2.5, together with a from __future__ import absolute_import statement that changes the behavior to the Python 3 behavior.

If you need to support Python 2.4 or earlier you have to spell out the whole package name so import csv becomes from mypkg import csv and from csv import my_csv becomes from mypkg.csv import my_csv. For clarity and readability I would avoid relative imports if you can and always spell out the whole path.

2to3 will check if your imports are local and change them.

Indentation¶

2to3 fixer ☐ six support ☐

In Python 2 a tab will be equal to eight spaces as indentation, so you can indent one line with a tab, and the next line with eight spaces. This is confusing if you are using an editor that expands tabs to another number than eight spaces.

In Python 3 a tab is only equal to another tab. This means that each indentation level has to be consistent in its use of tabs and spaces. If you have a file where an indented block sometimes uses spaces and sometimes tabs, you will get the error TabError: inconsistent use of tabs and spaces in indentation.

The solution is of course to remove the inconsistency.

`input()` and `raw_input()`¶

2to3 fixer ☑ six support ☐

In Python 2 there is raw_input() that takes a string from stdin and input() that takes a string from stdin and evaluates it. That last function is not very useful and has been removed in Python 3, while raw_input() has been renamed to input().

If you need to evaluate the input string you can use eval():

>>> eval(input('Type in an expression: '))
'Type in an expression: ' 1+2
3

If you need code that runs in both Python 2 and Python 3 without 2to3 conversion you can conditionally set input() to be raw_input():

>>> try:
...     input = raw_input
... except NameError:
...     pass

>>> input('Type in a string: ')
Type in a string: It works!
'It works!'

Integer division¶

2to3 fixer ☐ six support ☐

In Python 2, the result of dividing two integers will itself be an integer; in other words 3/2 returns 1. In Python 3 integer division will always return a float. So 3/2 will return 1.5 and 4/2 will return 2.0.

If you want the old behavior you should instead use the floor division operator //, available since Python 2.2. If you need to support both Python 2 and Python 3 without 2to3 conversion the following __future__ import works since Python 2.2 and enables the new behavior:

>>> from __future__ import division
>>> 1/2
0.5

`long`¶

2to3 fixer ☐ six support ☑ (partial)

Python 2 has two integer types int and long. These have been unified in Python 3, so there is now only one type, int. This means that the following code fails in Python 3:

>>> 1L
1L
>>> long(1)
1L

It’s quite unusual that you would need to specify that an integer should be a long in Python 2, as Python’s integer functions all will return long when needed. If you do require it the following code works on both Python 2 and Python 3 without 2to3 conversion:

>>> import sys
>>> if sys.version_info > (3,):
...     long = int
>>> long(1)
1L

However, the representation is still different, so doctests will fail.

If you need to check if something is a number you need to check against both int and long under Python 2, but only int in Python 3. The best way to do that is to set up a integer_types tuple depending on Python version and test against that. six includes this:

>>> import sys
>>> if sys.version_info < (3,):
...     integer_types = (int, long,)
... else:
...     integer_types = (int,)
>>> isinstance(1, integer_types)
True

`map()`¶

2to3 fixer ☐ six support ☐

In Python 2 map() returns a list while in Python 3 it returns an iterator. 2to3 will in some cases place a list() call around the call to map() to ensure that the result is still a list. If you need code that runs in both Python 2 and Python 3 without 2to3 conversion and you need the result to be a list, you can do the same.

In Python 2 map() will continue until the longest of the argument iterables are exhausted, extending the other arguments with None.

>>> def fun(a, b):
...    if b is not None:
...        return a - b
...    return -a
>>> map(fun, range(5), [3,2,1])
[-3, -1, 1, -3, -4]

In Python 3 map() will instead stop at the shortest of the arguments. If you want the Python 2 behavior in Python 3 you can use a combination of starmap() and zip_longest().

>>> from itertools import starmap, zip_longest
>>> def fun(a, b):
...    if b is not None:
...        return a - b
...    return -a
>>> list(starmap(fun, zip_longest(range(5), [3,2,1])))
[-3, -1, 1, -3, -4]

The Python 2 map() will accept None as it’s function argument, where it will just return the object(s) passed in. As this transforms map() into zip() it’s not particularly useful, and in Python 3 this no longer works. However, some code depends on this behavior, and you can use the following function as a full replacement for the Python 2 map.

from itertools import starmap, zip_longest

def map(func, *iterables):
    zipped = zip_longest(*iterables)
    if func is None:
        # No need for a NOOP lambda here
        return zipped
    return starmap(func, zipped)

Metaclasses¶

2to3 fixer ☑ six support ☑

In Python 2 you specified the metaclass with the __metaclass__ attribute. In Python 3 you instead pass in a metaclass parameter in the class definition. Supporting metaclasses in Python 2 and Python 3 without using 2to3 requires you to create classes on the fly. If you want this, I highly recommend to use the six module, which has a very clever with_metaclass() function.

`.next()`¶

2to3 fixer ☑ six support ☑

In Python 2 you get the next result from an iterator by calling the iterators .next() method. In Python 3 there is instead a next() builtin.

If you need code that runs in both Python 2 and Python 3 without 2to3 conversion you can make a function that under Python 2 calls iterator.next() and under Python 3 calls next(iterator). The six module contains such a function, called advance_iterator().

Parameter unpacking¶

2to3 fixer ☐ six support ☐

In Python 2 you have parameter unpacking:

>>> def unpacks(a, (b, c)):
...     return a,b,c

>>> unpacks(1, (2,3))
(1, 2, 3)

Python 3 does not support this, so you need to do your own unpacking:

>>> def unpacks(a, b):
...     return a,b[0],b[1]

>>> unpacks(1, (2,3))
(1, 2, 3)

`print`¶

2to3 fixer ☑ six support ☑

The Python 2 print statement is in Python 3 a function. If you need to run the same code in both Python 2 and Python 3 without 2to3 conversion there are various techniques for this. This is discussed in detail in Supporting the print() function.

`raise`¶

2to3 fixer ☑ six support ☑

In Python 2 the syntax for the raise statement is:

raise E, V, T

Where E is a string, an exception class or an exception instance, V the an optional exception value in the case that E is a class or a string and T is a traceback object if you want to supply a traceback from a different place than the current code. In Python 3 this has changed to:

raise E(V).with_traceback(T)

As with the Python 2 syntax, value and traceback are optional. The syntax without the traceback variable is:

raise E(V)

This works in all versions of Python. It’s very unusual that you need the traceback parameter, but if you do and you also need to write code that runs under Python 2 and Python 3 without using 2to3 you need to create different a function that takes E, V and T as parameters and have different implementations under Python 2 and Python 3 for that function. The six module has a nice implementation of that, called reraise().

`range()` and `xrange()`¶

2to3 fixer ☑ six support ☑

In Python 2 range() returns a list, and xrange() returns an object that will only generate the items in the range when needed, saving memory.

In Python 3, the range() function is gone, and xrange() has been renamed range(). In addition the range() object supports slicing in Python 3.2 and later .

2to3 will in some cases place a list() call around the call to range(), to ensure that the result is still a list. If you need code that runs in both Python 2 and Python 3 without 2to3 conversion and you need the result to be a list, you can do the same.

You can import xrange() from the six module to be sure you get the iterator variation under both Python 2 and Python 3.

`repr()` as backticks.¶

2to3 fixer ☑ six support ☐

In Python 2 you can generate a string representation of an expression by enclosing it with backticks:

>>> `sorted`
'<built-in function sorted>'

>>> `2+3`
'5'

The only purpose with this syntax is to confuse newbies and make obfuscated Python. It has been removed in Python 3, since the repr() builtin does exactly the same.

>>> repr(sorted)
'<built-in function sorted>'

>>> repr(2+3)
'5'

Rounding behavior¶

2to3 fixer ☐ six support ☐

The behavior of round has changed in Python 3. In Python 2, rounding of halfway cases was away from zero, and round() would always return a float.

>>> round(1.5)
2.0
>>> round(2.5)
3.0
>>> round(10.0/3, 0)
3.0

In Python 3 rounding of halfway cases are now always towards the nearest even. This is standard practice, as it will make a set of evenly distributed roundings average out.

When called without the second parameter, which determines the number of decimals, round() will in Python 3 return an integer. If you pass in a parameter to set the number of decimals to round to, the returned value will be of the same type as the unrounded value. This is true even if you pass in zero.

>>> round(1.5)
2
>>> round(2.5)
2
>>> round(10.0/3, 0)
3.0

If you need the Python 2 behavior, you can use the following method:

>>> import math
>>> def my_round(x, d=0):
...     p = 10 ** d
...     if x > 0:
...         return float(math.floor((x * p) + 0.5))/p
...     else:
...         return float(math.ceil((x * p) - 0.5))/p

>>> my_round(1.5)
2.0
>>> my_round(2.5)
3.0
>>> my_round(10.0/3, 0)
3.0

Slice operator methods¶

2to3 fixer ☐ six support ☐

In Python 1 you used __getslice__ and __setslice__ to support slice methods like foo[3:7] on your object. These were deprecated in Python 2.0 but still supported. Python 3 removes the support for the slice methods, so you need to instead extend __getitem__, __setitem__ and __delitem__ with slice object support.

>>> class StrawberryTart(object):
...
...    def __getitem__(self, n):
...        """An example of how to use slice objects"""
...        if isinstance(n, slice):
...            # Expand the slice object using range()
...            # to a maximum of eight items.
...            return [self[x] for x in
...                    range(*n.indices(8))]
...
...        # Return one item of the tart
...        return 'A slice of StrawberryTart with ' \
...               'not so much rat in it.'
...
>>> tart = StrawberryTart()
>>> tart[5:6]
['A slice of StrawberryTart with not so much rat in it.']

Sorting¶

2to3 fixer ☐ six support ☐

In Python 2 the .sort() method on lists as well as the sorted() builtin takes two parameters, cmp and key. In Python 3 only the key parameter is supported. There are no fixers for this, so you need to change that in the Python 2 code.

See When sorting, use key instead of cmp for more information.

`StandardError`¶

2to3 fixer ☑ six support ☐

Python 2 has an exception class called StandardError that has been removed in Python 3. Use Exception instead.

String types¶

2to3 fixer ☑ six support ☑

Python 2 had two string types; str and unicode. Python 3 has only one; str, but instead it also has a bytes type made to handle binary data. For more information on this, see Bytes, strings and Unicode and More bytes, strings and Unicode.

Footnotes

Supporting Python 3: An in-depth guide

Language differences and workarounds¶

`apply()`¶

`buffer()`¶

`callable()`¶

Classes¶

Comparisons¶

`coerce()` and `coerce`¶

Dictionary methods¶

`except`¶

Exception objects¶

`exec`¶

`execfile`¶

`file`¶

`filter()`¶

Imports¶

Indentation¶

`input()` and `raw_input()`¶

Integer division¶

`long`¶

`map()`¶

Metaclasses¶

`.next()`¶

Parameter unpacking¶

`print`¶

`raise`¶

`range()` and `xrange()`¶

`repr()` as backticks.¶

Rounding behavior¶

Slice operator methods¶

Sorting¶

`StandardError`¶

String types¶

Search the book

Supporting Python 3: An in-depth guide

Language differences and workarounds¶

apply()¶

buffer()¶

callable()¶

Classes¶

Comparisons¶

coerce() and __coerce__¶

Dictionary methods¶

except¶

Exception objects¶

exec¶

execfile¶

file¶

filter()¶

Imports¶

Indentation¶

input() and raw_input()¶

Integer division¶

long¶

map()¶

Metaclasses¶

.next()¶

Parameter unpacking¶

print¶

raise¶

range() and xrange()¶

repr() as backticks.¶

Rounding behavior¶

Slice operator methods¶

Sorting¶

StandardError¶

String types¶

Search the book

`apply()`¶

`buffer()`¶

`callable()`¶

`coerce()` and `coerce`¶

`except`¶

`exec`¶

`execfile`¶

`file`¶

`filter()`¶

`input()` and `raw_input()`¶

`long`¶

`map()`¶

`.next()`¶

`print`¶

`raise`¶

`range()` and `xrange()`¶

`repr()` as backticks.¶

`StandardError`¶