Deploying a Django app on the desktop

Posted on May 31st, 2007 in Catalyst, django, Product design, python, Tool by siddharta || 48 Comments

One of the cool things about Silver Catalyst (which is a Django app) is that you can start using it right out of the box. I didn’t want the team working their way around Apache and MSSQL configurations, Python version incompatibilities, database access issues and deployment hassles. The final solution was a simple executable, which when run would start everything required to get going.

In this post, I’ll explain how that was achieved.

A bit of background first. The vision for Silver Catalyst was a tool that could be installed by a team without having to get centralised servers and system administrators involved. I wanted the team to be able to set aside one computer on which they could install Silver Catalyst by themselves for use on their project. That made it pretty important that the setup be absolutely painless.

Silver Catalyst was written in Python using the Django framework. My first thought was to package the application as a zip of compiled python files and then have the team deploy that onto a server. This meant that someone would have to setup a web server, get Django working with it, setup a database, install python, deploy the application and finally configure it to point to the right files. Phew!

Luckily, to the rescue came cx_freeze. cx_freeze is a really nifty utility that will take your python code, check the included libraries and package everything along with the current python version into an executable. Very cool! A bonus was that you would not even need python installed to run the tool since the interpreter is packaged in the executable. This also eliminated issues like python version incompatibilities and so on.

Step two was the decision to use sqlite3 as the database. Since sqlite3 is an embedded database, you don’t need to have a database installed at all. Even better, there are standard modules to access sqlite3. In fact, it has been made a part of the standard distribution from Python 2.5 on.

The only dependency remaining was the web server. If I could use a pure python web server, I could simply hook up Django to it via WSGI and include the whole lot in the executable. That is what I did. I used ToofPy with some modifications so that it is pre-configured to run Django.

When cx_freeze is run on this setup, it bundles together the web server, Django, sqlite3, the application code, and the python interpreter into an executable. The application is pre-configured to use the packaged sqlite3 database code and the web server is pre-configured to load Django.

The result?

Just run the executable and the server starts. Access the server via a browser and you go straight into the Silver Catalyst application. In fact, because there are no external dependencies, it is easy to think that it is a native application. But behind the scenes, it is all python and some cx_freeze magic. If you want to see for yourself what the final result was like, you can download a free version of the tool here.

Doing Distributed Agile?

Share and collaborate with distributed teams with our electronic agile board tools. Get all the benefits of electronic tools without sacrificing the benefits of physical boards. Supports Scrum taskboards, Kanban boards and user story maps. Check it out!

48 Responses to “Deploying a Django app on the desktop”

  1. links for 2007-06-02 « Joost’s weblog Says:

    [...] Deploying a Django app on the desktop (Silver Stripe Blog) With the craze about Google Gears, this is another approach to do offline web applications. Silver Catalyst is a ’scrum tool’ that runs as a server. The server is packaged as an executable. Works great, just tried it: clean interface and interaction. (tags: django scrum webdev offline apollo googlegears) [...]

  2. kai Says:

    One quick question:

    Any reason why you used cz_freeze instead of py2exe (http://www.py2exe.org)? Py2exe seems to be more up-to-date (latest update on 2006-12-30) and it works great with the scripts I used it with so far.

  3. Orestis Markou Says:

    Very interesting!

    Can we get some more details on how to do this ? How to organize a project, possible caveats etc ?

    Thanks!

  4. siddharta Says:

    Yeah py2exe was one choice. cx_freeze uses the techniques in py2exe among others, so I decided to try it first and it worked pretty well for my application.

    The cool part is that cx_freeze automatically checks for imported modules and includes them in the package. The one caveat, and its a big one is that it obviously cannot find modules that are imported at runtime by using the __import__(..) function.

    There are two ways around this.

    One is to put all dynamic modules in the same directory (or a fixed relative directory) as the executable so that python can find them when they are imported. The other option is to statically include these modules somewhere.

    I actually use both techniques. I have some plugin modules that are loaded dynamically. I leave these at a relative path from the executable because I want to add and remove these at runtime.

    Django uses dynamic imports in some places, eg: determining which database code to use. I’ve replaced this with a static import because I know beforehand that I am going to use sqlite3.

    I also have a file called imports.py that has statically imports various modules that cx_freeze was unable to detect. Then in the main script I say “import imports” and when cx_freeze is parsing it finds all these modules and includes them.

  5. Garth Kidd Says:

    I’ve been making my own executable, also, albeit using py2exe and the Django admin server. I don’t suppose you could post your ToofPy integration code at Django Snippets?

  6. siddharta Says:

    Hmm yeah I suppose I could to that..

  7. johan Says:

    Excellent and very inspiring writeup!

  8. Martin Says:

    siddharta,

    Thanks for the write-up – very interesting. Did you ever get around to posting the ToofPy integration code to Django Snippets ?

    Martin

  9. siddharta Says:

    Just did it – http://www.djangosnippets.org/snippets/301/

    It’s 2 lines of code :)

  10. Silver Stripe Blog » Blog Archive » Scoble on Django Says:

    [...] goodies that come with working in Python. Topping the list are the really cool utilities like cx_freeze that allow me to integrate a web server, database (sqlite3), django and my application into one [...]

  11. links for 2007-07-18 « PaxoBlog Says:

    [...] Silver Stripe Blog » Blog Archive » Deploying a Django app on the desktop The vision for Silver Catalyst was a tool that could be installed by a team without having to get centralised servers and system administrators involved. I wanted the team to be able to set aside one computer on which they could install Silver Catalyst by (tags: django tools packs development python) [...]

  12. Jack Says:

    Hi, thanks for the idea. Since Django is distributed with its own lightweight web server, what is the reason you went with ToofPy?. Cheers.

  13. siddharta Says:

    Hi Jack, Django’s server is just for development use. They don’t recommend it for production.

  14. nomadjourney Says:

    I am trying to do something similar. What was the exact command line params that you passed to cx_Freeze to make sure that it picked up everything including the django apps under the main project directory?

  15. nomadjourney Says:

    This is what I did:

    FreezePython –install-dir=../dist manage.py

    And this is what I get when running the app:

    Traceback (most recent call last):
    File “/home/nsayeed/Downloads/cx_Freeze-3.0.3/initscripts/Console.py”, line 27, in ?
    exec code in m.__dict__
    File “manage.py”, line 12, in ?
    File “/var/lib/python-support/python2.4/django/core/management.py”, line 1299, in execute_manager
    project_module = __import__(project_name, ”, ”, [''])
    ImportError: No module named manage

  16. siddharta Says:

    Hi nomadjourney,

    When doing FreezePython, it does not package the modules that are imported dynamically at runtime using __import__ because obviously the software does not know beforehand what is going to be imported. What you need to do is to create a file, say imports.py and import all the dynamically loaded modules there, like so.

    cx_imports.py
    ————-

    import manage
    import xyz

    mainfile.py
    ————-

    import cx_imports

    This way cx_freeze will be able to find the required modules and package them in the exe.

    The other option is to put the files in the same directory as the exe so that they can be found dynamically at runtime. Use this option if you don’t know beforehand which modules might be loaded.

  17. siddharta Says:

    So just to give an example, a part of my cx_imports.py file looks like this -

    import os, sys

    os.environ['DJANGO_SETTINGS_MODULE'] = “your_project.settings”

    import your_project.your_app.models
    import your_project.your_app.views
    import your_project.urls
    import your_project.settings
    import your_project.manage
    import django.template.loaders.filesystem
    import django.template.loaders.app_directories
    import django.middleware.common
    import django.contrib.sessions.middleware
    import django.contrib.auth.middleware
    import django.middleware.doc
    import django.contrib.auth
    import django.contrib.contenttypes
    import django.contrib.sessions
    import django.contrib.sites
    import django.core.cache.backends.simple
    import django.db.backends.sqlite3.base
    import django.db.backends.sqlite3.introspection
    import django.db.backends.sqlite3.creation
    import django.db.backends.sqlite3.client
    import django.template.defaulttags
    import django.template.defaultfilters
    import django.template.loader_tags

    As you can see, I am statically importing the sqlite3 modules because thats what I am using, so although Django supports other databases, I’m not importing them here. The exact combination of files you need to put here depends on the modules your app uses.

  18. Joseph Says:

    Siddharta: I’m trying to package a django application using cx_freeze and I am running into error. I hope you can help me.

    WindowsError: [Error 3] The system cannot find the path specified: ‘C:\Documents and Settings\Joseph\Desktop\SOL\sol.exe\django\db\backends

    I know this is not a support forum for cx_freeze; but as you’ve already packaged django using it, I am just hoping you can help.

    Thank you very much for your time.
    Joseph

  19. siddharta Says:

    There is a part in the django code where it tries to open django/db/backends to search for the list of installed backends. This happens because it’s not able to import the database code for the backend you specified. Check out __init__.py in django/db directory for this bit of code.

    Django uses dynamic imports to import the backend. Since cx_freeze cannot resolve dynamic imports, you’ll can put a static import somewhere in your code to tell cx_freeze to include this module. See Comment 17 above. The example above uses sqlite3 backend. Replace those imports to whichever backend you are using.

  20. Ben Says:

    I got FreezePython (cx_freeze) working fine on a hello_world.py program, but when I follow your instructions on a skeleton django project, I get the following error from running FreezePython on mainfile.py:

    Module cx_imports from file cx_imports.py has invalid syntax (cx_imports.py, line 3).

    That is, the os.environ[] command appears to have invalid syntax. I changed ‘your_project’ to ‘mysite’ everywhere applicable. I’m not sure what else to do!

  21. Ben Says:

    btw, I am just doing everything with the Django development server for now, and there are no databases involved

  22. Ben Says:

    Ok, I found the problem … Beware that if you copy-paste from the instructions in post number 17 that there may be a non-ASCII character hiding in there somewhere

  23. siddharta Says:

    Hi Ben, you’re right. WordPress changes the formatting of some characters like the quotes and dashes to special characters (like curly opening and closing quotes), so when copy-pasting, we need to be a bit careful that the right quotes and dashes are used.

  24. Waldo Says:

    Thanks Siddharta.
    This was a good starting point, although I ended up using CherryPy (instead of ToofPy) and Py2Exe (instead of cx_freeze).
    I ended up doing something more based on Joseph Jude’s work http://www.jjude.com/index.php/archives/70 – itself inspired by your post here.
    Anyway I’ve posted my notes here – http://misunderstandings.wordpress.com/2008/06/26/django-desktop-app/

  25. Django desktop app « Latest Misunderstandings Says:

    [...] me to Siddharta’s site he was doing what I wanted but without the detailed instructions a simpleton (me) [...]

  26. pytaste Says:

    Has anyone seen OpenSway (http://opensway.googlecode.com) It looks promising, but still in early stages. I downloaded it from svn and with some tweeking got it to work. It seems like a cool project though, and it’s based off XULRunner

  27. siddharta Says:

    Just had a quick glance at OpenSway. Looks interesting. Thanks for pointing it out.

  28. webguy Says:

    do you have the source code for the wiki in 15 min.
    I am just starting with py and dj.

    another question, if you were to design a site that become very popular, do you think dj database query is optimized to handle huge traffic?

    thx,

    webguy

  29. siddharta Says:

    I dont have the source with me anymore, but you can follow along in the screencast and get it – http://showmedo.com/videos/series?name=v7kABKL6R

    Note that some parts need to be updated for the latest version of Django

    About the django queries, a lot of high traffic websites use it, so I don’t think it should be a major issue.

  30. Ralph Heinkel Says:

    Hi siddharta,

    again a question using cx_freeze for compiling a django app, maybe you could give me a hint …
    I got pretty far following your suggestion with providing a file with static imports for all dynamically loaded modules.
    However it seems that commands like ‘runserver’ (and all the others) are not available in the compiled django app even though I listed those django modules in the import file. And from the log I can see that they are found by cxfreeze. To me this smells a bit like an initialization problem, what do you think?

    ./manage runserver
    Unknown command: ‘runserver’
    Type ‘manage help’ for usage.

    Thank you very much for your help,

    Ralph

  31. siddharta Says:

    Ralph,

    What happens with the management commands is that Django dynamically loads the commands only at runtime by looking at the command files installed with each app. So you also need to include all the command files in the static imports, otherwise Django doesn’t load the command.

  32. JatStraikarFan Says:

    Various of guys blog about this matter but you wrote down some true words!!

  33. Rosen Sharma Says:

    Could you share the size of the executable created?

    thx
    RS

  34. bentley Says:

    What happens with the management commands is that Django dynamically loads the commands only at runtime by looking at the command files installed with each app. So you also need to include all the command files in the static imports, otherwise Django doesn’t load the command.

  35. google api code Says:

    Hello, I do think your web site may be having browser compatibility problems.
    When I take a look at your website in Safari, it looks fine however, when opening
    in I.E., it’s got some overlapping issues. I just
    wanted to give you a quick heads up! Besides that, fantastic site!

  36. kolikkopeleja netissa Says:

    Good day! Do you know if they make any plugins to protect against
    hackers? I’m kinda paranoid about losing everything I’ve worked
    hard on. Any suggestions?

    Here is my site – kolikkopeleja netissa

  37. gazelle interactive Says:

    I’m no longer sure the place you’re getting your information,
    however great topic. I needs to spend a while studying more or understanding more.
    Thank you for excellent information I was looking for this
    information for my mission.

  38. okinawan kamas Says:

    An interesting discussion is worth comment. I believe
    that you need to write more about this subject matter, it might not be a taboo matter but typically people
    don’t speak about such subjects. To the next!
    Cheers!!

    Feel free to surf to my web site; okinawan kamas

  39. baby tv Says:

    Hi, i think that i noticed you visited my weblog so i got here to go back the
    prefer?.I am attempting to find things to improve my site!I guess its good enough
    to make use of some of your ideas!!

  40. http://Www.Dailymile.com/ Says:

    Thanks for sharing such a pleasant idea, paragraph is fastidious,
    thats why i have read it fully

  41. http://www.imoodle.de/wikis/operatoren/index.php/Benutzer:Lincoln41P Says:

    Pretty nice post. I just stumbled upon your weblog and wanted to say that I have truly enjoyed surfing around your
    blog posts. In any case I’ll be subscribing to your rss feed and I hope you
    write again very soon!

  42. acne remedies guide victoria west pdf Says:

    Attractive section of content. I just stumbled upon your weblog
    and in accessoon capital to assert that I acquire inn fact enjoyed account your blog posts.
    Any way I will bbe subscribing tto your eeds annd even I achievement you access consistently rapidly.

    Also visit myy weblog … acne remedies guide victoria west pdf

  43. Bowflex Walktc Says:

    If you are prone to blisters, then taping your feet in troublesome areas with zinc oxide tape is advisable.
    One has to wonder how Kenya’s fitness DVD has become such a huge hit in Japan. You
    need to find a way to change the way you think
    about food, in order to avoid the bad habit of overeating.

  44. chris brown album x free Says:

    It’s really a nice and useful piece of information. I
    am happy that you just shared this useful information with us.
    Please keep us up to date like this. Thanks for sharing.

  45. vitamin a Says:

    Coconut Oil is Beneficial for Skincare – One of the many benefits of Vitamin E in coconut oil is a fat-soluble vitamin and is composed of compounds well-known for giving skin care great benefits.
    Of particular importance to athletes are Vitamin D, several B vitamins,
    Vitamin C and Vitamin E. Boost Your Energy And Brain Power With Vitamin B12.

  46. dietformula Says:

    Howdy! Do you know if they make any plugins to assist with Search
    Engine Optimization? I’m trying to get my blog to
    rank for some targeted keywords but I’m not seeing very good gains.
    If you know of any please share. Thanks!

    Feel free to surf to my web-site – dietformula

  47. fifa 15 ultimate team point generator Says:

    It’s in fact very difficult in this full of
    activity life to listen news on Television, thus I simply use
    internet for that purpose, and obtain the latest information.

  48. Shoshana Says:

    It’s impressive that you are getting ideas from this
    paragraph as well as from our argument made at this time.

Leave a Reply