Python

Notes

• Sample Script: sample.py.
• List attributes of an object: dir(object).
• Reference object attributes and values via: object.__dict__.
• Check if string is numeric: s.isdigit()
• String and Integer formatting: '%s%07d' % ('RPT', 7) gives RPT0000007.
• Identify subclasses: issubclass(A,B) returns True if A is a subclass of B. (e.g. issubclass(self.model,Config))
• Check for attribute: hasattr(object,string) Returns True if the string matches an attribute name.
• Use tuple packing and unpacking for atomicity: x, y = y, x+y.
• Use generator expressions instead of list comprehensions: sum(i**2 for i in range(10)).
• Use decorators to factor-out administrative logic.
• Use with to factor out temporary contexts: with open('data.txt') as f:
• To use UTF-8 characters in code the module must begin with: # coding=UTF-8
• Classes:
  • Define __repr__ to return representation of instance.
• Function calls: Use keyword arguments and return multiple values as namedtuples.
• Sets: set_literal = {1,2,3}
• Strings: Concatenate with join.
• Loops: Use range, reversed, enumerate (not for indices), izip (to combine iterables), sorted(iterable[, key=function]), for... else: Else clause executed on completion of for (i.e. skipped on break).
  • break ends the entire loop; continue ends the current iteration.
• Operators: Increment [var] += 1; Modulo %. Ternary operator: x if a > b else y Return x if a > b otherwise return y.
  • map(function, iterable): Apply function to each item in iterable and return iterable of results.
  • reduce(function, iterable): Function takes two arguments. Apply function to first two items and then to result and next item. Repeat over iterable. Returns single result.

Python Debugger

Add the following line to start a debug session at that point in the code at runtime:

import pdb; pdb.set_trace() # Start debugging

Commands (hit enter to repeat previous command):

• n Next
• p [name] Print (pp Pretty Print)
• c Continue
• r Continue to next Return
• s Step Into
• l List
• b Breakpoint
• q Quit

Dictionaries

• Loop through keys and values in a dictionary: for key, value in lookup_address.items().
• Retrieve matching value or default: var = dict.get(key, default_value). Ditto and remove matching entry: var = dict.pop(key, default_value).
• Use: del d[key] to delete.
• Key filter: d = {k : d[k] for k in d if not k.startswith('r')}.
• Construct from iterables: d = dict(izip(keys, values)).
• Calculate counters by looping through: d[key] = d.get(key, 0) + 1.
• Combine dictionaries: d = ChainMap(command_line_args, os.environ, defaults).

Exceptions

Access ongoing Exception information: sys.exc_info():

• Capture last line of traceback and line describing problem to include in a message:

  from traceback import format_exc
  . . .
  except Exception as e:
      trace_info = format_exc(sys.exc_info()).split("\n")[-4:-2]
  

• Print traceback: print('traceback %s' % (traceback.extract_tb(sys.exc_info()[2]))).

Operators

• Increment [var] += 1; Modulo %.
• Ternary operator: x if a > b else y Return x is a > b otherwise return y (equivalent to a > b ? x : y; in Java).

Interaction with Operating System

• Directory containing current module: cwd = os.path.realpath(os.path.dirname(__file__))
• Relative Paths: DATABASE_NAME = os.path.expanduser('~/django-reversion/default.sqlite') or DATABASE_NAME = os.path.join(os.path.dirname(__file__), 'default.sqlite')
• Build path: PROJECT_PATH = os.path.join(os.path.expanduser('~/code'), 'dmcm/project')
• Run command: p = Popen("grep -r --include='*.py' \"%s\" %s" % (string.strip(), PROJECT_PATH), shell=True, stdout=PIPE)
• Run a script from the Python shell: execfile("/home/ahernp/test.py")
• To find where a module was imported from check its .__file__ attribute.

Slicing

• a[start:end] Items start through end-1.
• a[start:] Items start through the rest of the array.
• a[:end] Items from the beginning through end-1.
• a[:] A copy of the whole array.
• a[::-1] Reverse the array.

A step value can be used with any of the above: a[start:end:step].

The :end value represents the first value that is not in the selected slice. The difference between end and start is the number of elements selected (if step is 1, the default).

Also start or end may be a negative number, which means it counts from the end of the array instead of the beginning.

• a[-1] Last item in the array.
• a[-2:] Last two items in the array.
• a[:-2] Everything except the last two items.

Python is forgiving if there are fewer items than you ask for. For example, if you ask for a[:-2] and a only contains one element, you get an empty list instead of an error.

Recipes

Write File

import codecs
def write_file(filename, data):
    output_filepath = os.path.join(settings.PROJECT_DIR, filename)
    output_file = codecs.open(output_filepath, 'w', 'utf-8')
    output_file.write(data)
    output_file.close()
    print('Data written to %s' % (output_filepath))

Add to existing file: open(output_filepath, 'a', 'utf-8')

Remove Substring

1. Find start of substring: start_pos = big_string.index('/home/')
2. Find end of substring: end_pos = big_string.index("'",start_pos)
3. Remove substring: big_string = big_string[:start_pos] + big_string[end_pos:]

To check if substring is present use: start_pos = big_string.find('/home/')

Display Progress Percentage

Calculate and display percentage progress:

rows = source_class.objects.using('legacy').all()
total = len(rows)
. . .
for i, row in enumerate(rows):
   . . .
   percent = float(i) / total * 100
   sys.stdout.write("\r%2.3f%%" % percent)
   sys.stdout.flush()

Example shows percentage to three decimal places, to show percentage as integer use "\r%2d%%".

Convert to HTML Entities

From Unicode: u'Østlandet'.encode('ascii', 'xmlcharrefreplace') gives 'Østlandet'.

From String: 'Østlandet'.decode('utf-8').encode('ascii', 'xmlcharrefreplace') also gives 'Østlandet'.