memory management – Short Integers in Python

memory management – Short Integers in Python

Nope. But you can use short integers in arrays:

from array import array
a = array(h) # h = signed short, H = unsigned short

As long as the value stays in that array it will be a short integer.

• documentation for the array module

Thanks to Armin for pointing out the array module. I also found the struct module that packs c-style structs in a string:

From the documentation (https://docs.python.org/library/struct.html):

>>> from struct import *
>>> pack(hhl, 1, 2, 3)
x00x01x00x02x00x00x00x03
>>> unpack(hhl, x00x01x00x02x00x00x00x03)
(1, 2, 3)
>>> calcsize(hhl)
8

memory management – Short Integers in Python

Armins suggestion of the array module is probably best. Two possible alternatives:

• You can create an extension module yourself that provides the data structure that youre after. If its really just something like a collection of shorts, then
  thats pretty simple to do.
• You can
  cheat and manipulate bits, so that
  youre storing one number in the
  lower half of the Python int, and
  another one in the upper half.
  Youd write some utility functions
  to convert to/from these within your
  data structure. Ugly, but it can be made to work.

Its also worth realising that a Python integer object is not 4 bytes – there is additional overhead. So if you have a really large number of shorts, then you can save more than two bytes per number by using a C short in some way (e.g. the array module).

I had to keep a large set of integers in memory a while ago, and a dictionary with integer keys and values was too large (I had 1GB available for the data structure IIRC). I switched to using a IIBTree (from ZODB) and managed to fit it. (The ints in a IIBTree are real C ints, not Python integers, and I hacked up an automatic switch to a IOBTree when the number was larger than 32 bits).