Dr Alexis Petrounias

Django application import and missed class_prepared signals

Due to opaque application loading semantics within the Django i18n code, several model classes may be imported before a class_prepared signal listener connects, resulting in missing the signal entirely. Specifically, connecting to this signal from an application which appears first in INSTALLED_APPS, does not guarantee that it will execute when a class is first imported by any of the subsequent applications.

Problem

A class_prepared signal is dispatched when the model class is instantiated for the first time; this happens inside class Model’s metaclass at django.db.models.base: 257:

signals.class_prepared.send(sender=cls)

Therefore, we would expect that connecting to this signal from the __init__.py of an app which appears first in the INSTALLED_APPS list, would ensure that our listener receives the signal for all Models loaded in the remainder apps (following is the beginning of a typical such list):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

INSTALLED_APPS = (
  'class_prepared_listener_app',
  'django.contrib.auth',
  'django.contrib.contenttypes',
  'django.contrib.sessions',
  'django.contrib.sites',
  'django.contrib.messages',
  'django.contrib.admin',
  ...
)

However, this is not (always) the case. Tracing, for example, shell execution, we find that django.core.management.commands.shell: 45:

from django.db.models.loading import get_models
loaded_models = get_models()

requests all models from the app cache which first calls _populate to initialize itself at django.db.models.loading: 167:

self._populate()

which loads all apps into the cache the first time it’s called and does nothing for subsequent calls at django.db.models.loading: 58:

1
2
3
4
5
6
7
8

for app_name in settings.INSTALLED_APPS:
    if app_name in self.handled:
        continue
    self.load_app(app_name, True)
if not self.nesting_level:
    for app_name in self.postponed:
        self.load_app(app_name)
    self.loaded = True

The apps are loaded in _populate in the order they’re listed in INSTALLED_APPS; some apps can be postponed though if they can’t be imported at the time of traversal at django.db.models.loading: 77:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

try:
    models = import_module('.models', app_name)
except ImportError:
    self.nesting_level -= 1
    # If the app doesn't have a models module, we can just ignore the
    # ImportError and return no models for it.
    if not module_has_submodule(app_module, 'models'):
        return None
    # But if the app does have a models module, we need to figure out
    # whether to suppress or propagate the error. If can_postpone is
    # True then it may be that the package is still being imported by
    # Python and the models module isn't available yet. So we add the
    # app to the postponed list and we'll try it again after all the
    # recursion has finished (in populate). If can_postpone is False
    # then it's time to raise the ImportError.
    else:
        if can_postpone:
            self.postponed.append(app_name)
            return None
        else:
            raise

Tracing

A print statement in _populate indicates that no apps are postponed and that all apps are, in fact, loaded in sequence. Furthermore the signal gets connected before any other apps are loaded. The output of manage.py shell with several print statements:

Signal connected.
class_prepared_listener_app
django.contrib.auth
django.contrib.contenttypes
django.contrib.sessions
django.contrib.sites
django.contrib.messages
django.contrib.admin
...

Adding a print statement in the Model’s metaclass indicates that some classes are instantiated before their owner apps are loaded:

Preparing class 'django.contrib.contenttypes.models.ContentType'
Preparing class 'django.contrib.auth.models.Permission'
Preparing class 'django.contrib.auth.models.Group_permissions'
Preparing class 'django.contrib.auth.models.Group'
Preparing class 'django.contrib.auth.models.User_user_permissions'
Preparing class 'django.contrib.auth.models.User_groups'
Preparing class 'django.contrib.auth.models.User'
Preparing class 'django.contrib.auth.models.Message'
Preparing class 'django.contrib.sites.models.Site'
Signal connected.
Preparing class 'django.contrib.sessions.models.Session'
Preparing class 'django.contrib.admin.models.LogEntry'
...

Cause

Further snooping shows that these classes are instantiated by translation-related preparation code, before the shell command is properly handled, at django.core.management.base: 202:

1
2
3
4
5
6
7
8

# Switch to English, because django-admin.py creates database content
# like permissions, and those shouldn't contain any translations.
# But only do this if we can assume we have a working settings file,
# because django.utils.translation requires settings.
if self.can_import_settings:
    try:
        from django.utils import translation
        translation.activate('en-us')

Models are instantiated inside the last call to activate() which eventually calls translation() at django.utils.translation.trans_real: 161:

1
2
3
4
5
6

for appname in reversed(settings.INSTALLED_APPS):
    app = import_module(appname)
    apppath = os.path.join(os.path.dirname(app.__file__), 'locale')

    if os.path.isdir(apppath):
        res = _merge(apppath)

Conclusion

From the above, it is straightforward to see that by removing the admin and auth apps, or by placing our app after the admin and auth apps, our signal connects before any Model is loaded. Neither of these two “solutions” however appear robust or welcome.

It should be noted that, because the call to translation.activate() is being made inside the execute method of the base class from which all management commands derive, this behaviour is to be expected when Django starts up through commands other than shell.

Admittedly, the official Django documentation for this signal states “Django uses this signal internally; it’s not generally used in third-party applications.” however, I strongly feel that Django would benefit from a formal and deterministic load order for INSTALLED_APPS.