I'm trying to pickle an object of a (new-style) class I defined. But I'm getting the following error:
>>> with open('temp/connection.pickle','w') as f: ... pickle.dump(c,f) ... Traceback (most recent call last): File "<stdin>", line 2, in <module> File "/usr/lib/python2.5/pickle.py", line 1362, in dump Pickler(file, protocol).dump(obj) File "/usr/lib/python2.5/pickle.py", line 224, in dump self.save(obj) File "/usr/lib/python2.5/pickle.py", line 331, in save self.save_reduce(obj=obj, *rv) File "/usr/lib/python2.5/pickle.py", line 419, in save_reduce save(state) File "/usr/lib/python2.5/pickle.py", line 286, in save f(self, obj) # Call unbound method with explicit self File "/usr/lib/python2.5/pickle.py", line 649, in save_dict self._batch_setitems(obj.iteritems()) File "/usr/lib/python2.5/pickle.py", line 663, in _batch_setitems save(v) File "/usr/lib/python2.5/pickle.py", line 306, in save rv = reduce(self.proto) File "/usr/lib/python2.5/copy_reg.py", line 76, in _reduce_ex raise TypeError("a class that defines __slots__ without " TypeError: a class that defines __slots__ without defining __getstate__ cannot be pickled
I didn't explicitly define
__slots__ in my class. Did something I do implicitly define it? How do I work around this? Do I need to define
Update: gnibbler chose a good example. The class of the object I'm trying to pickle wraps a socket. (It occurs to me now that) sockets define
__slots__ and not
__getstate__ for good reason. I assume once a process ends, another process can't unpickle and use the previous process's socket connection. So while I'm accepting Alex Martelli's excellent answer, I'm going to have to pursue a different strategy than pickling to "share" the object reference.
The class defining
__slots__ (and not
__getstate__) can be either an ancestor class of yours, or a class (or ancestor class) of an attribute or item of yours, directly or indirectly: essentially, the class of any object in the directed graph of references with your object as root, since pickling needs to save the entire graph.
A simple solution to your quandary is to use protocol
-1, which means "the best protocol pickle can use"; the default is an ancient ASCII-based protocol which imposes this limitation about
>>> class sic(object): ... __slots__ = 'a', 'b' ... >>> import pickle >>> pickle.dumps(sic(), -1) '\x80\x02c__main__\nsic\nq\x00)\x81q\x01.' >>> pickle.dumps(sic()) Traceback (most recent call last): [snip snip] raise TypeError("a class that defines __slots__ without " TypeError: a class that defines __slots__ without defining __getstate__ cannot be pickled >>>
As you see, protocol
-1 takes the
__slots__ in stride, while the default protocol gives the same exception you saw.
The issues with protocol
-1: it produces a binary string/file, rather than an ASCII one like the default protocol; the resulting pickled file would not be loadable by sufficiently ancient versions of Python. Advantages, besides the key one wrt
__slots__, include more compact results, and better performance.
If you're forced to use the default protocol, then you'll need to identify exactly which class is giving you trouble and exactly why. We can discuss strategies if this is the case (but if you can possibly use the
-1 protocol, that's so much better that it's not worth discussing;-) and simple code inspection looking for the troublesome class/object is proving too complicated (I have in mind some deepcopy-based tricks to get a usable representation of the whole graph, in case you're wondering).
Perhaps an attribute of your instance is using
__slots__ so it can't be pickled
You need to identify which attribute is causing the error and write your own
__setstate__ to ignore that attribute