Skip to content

__globals__ of function in deleted module restored inconsistently #532

@dfremont

Description

@dfremont

My application sometimes pickles functions after the module they were imported from has been removed from sys.modules (for reasons we hopefully don't have to get into). Usually dill handles this just fine, but there seems to be a bug when the pickled function is used in a closure which also maintains a reference to the pickled function's __globals__ dictionary. The following code yields an assertion failure with dill 0.3.5.1 (and also the latest version on master; in each case I was using Python 3.9):

import dill
import sys

import other    # another module; see below

def make():
    func = other.blah
    globs = func.__globals__
    def inner():
        assert globs is func.__globals__
    return inner

f = make()

del sys.modules['other']

f()     # assertion passes here
d = dill.dumps(f)
f2 = dill.loads(d)
f2()    # assertion fails here

where other.py contains the code:

def blah():
    return 42

The assertion fails because in the unpickled version of inner, the free variable func is bound to the unpickled version of blah but the free variable globs is bound to a dict which is not the globals dict of func (nor is it the globals dict of other.blah, in fact).

What seems to be happening is that when blah is unpickled, dill calls _create_function with the argument fglobals set to an empty dictionary, so that the line

func = FunctionType(fcode, fglobals or dict(), fname, fdefaults, fclosure)

creates a new dict to be the globals dict of the new function. Then when the "post-processing" updates the cells of inner, it sets the cell for globs to the old dictionary instead of this newly-created one. To confirm this diagnosis, I removed the or dict() from the line above, and changed the "F1" case of save_function to always set globs = globs_copy (i.e. removing the else branch where it sets globs = {'__name__': obj.__module__}), and indeed the problem went away. I'm not sure how to properly fix the issue, though.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions