Sunday, April 14, 2013

Porting Werkzeug to Python 3 : an experiment, part 1

So last week, I have ported Werkzeug to Python 3. The result is here, grab it if you will, but be aware it comes with important caveats which I'm going to detail below.

Where it's at right now, is that the test suite succeeds in both Python 2.7 and 3.3, with a single code base (it does not use 2to3). The local server runs, so does the debugger, and I have already converted a few of the examples that come with the distribution, and they too, run without problems so far.

Now, the first caveat is that I have done that totally on my own, without asking any permission or advice from the original developer, who so far has not reacted in any way. Quite possibly, he's aware of my port, and would rather say nothing than bad things about it ! Anyway, that means there's absolutely nothing "official" about that work.

The second, important, caveat is that I'm totally new to the Werkzeug code base, and that it was my first try at porting anything from Python 2 to Python 3.

Well I can see your eyes rolling and wondering, why choose such a complicated project as a first porting experience ? Well, here are a couple of reasons:

  • I wanted to learn more about Werkzeug, HTTP, WSGI,  Python 3, and character encodings,
  • Difficult things are challenging, and a challenge is always a source of motivation,
  • It's always fun and instructive to dive into a well-written and proven code base, which also happens to do very useful and interesting things, including powering one of the most famous Python web frameworks, namely, Flask,
  • Up until yesterday, there had been zero signal from Armin Ronacher (Werkzeug's author) that he wanted to port Werkzeug to Python 3, although he's seemingly been supportive of an effort others have started months ago and hasn't shown much progress as of today,
  • I was hoping it would maybe, somehow, get things moving towards Python 3 ports of Werkzeug and Flask, which a lot of people are waiting for. See it as a sociological component of the experiment !
  • Enjoying one of the benefits of free software, which is that you can hack it as much as you want,
  • It won't harm anyone...

So, that's for the "why". In the next post, I'll explain how it was done, and some of the difficulties I encountered along the way. I will also keep you posted about further progress, if any.

8 comments:

  1. This has already been done a long time ago...

    https://github.com/puzzlet/werkzeug

    ReplyDelete
    Replies
    1. I was aware of this port but didn't know more work had been done on it. I think it is different from mine and works on Python 3 only, though.

      Delete
  2. Philippe: Unless im interpreting your comment wrong, are you indicating Béranger shouldn't of bothered?
    If the case, so has living, but your still duplicating the effort of millions before you. Having a go "because you can" is a good enough reason in itself.

    Well done on your voyage of discovery Béranger.

    ReplyDelete
  3. Hey Béranger, based on your experience with this project, do you have any advice for other new-comers to porting Python 2 code to Python 3? Specifically about using the single codebase method. Thanks.

    ReplyDelete
    Replies
    1. Hi Steven, I'm in the process of writing part 2 of this article, in which I'll explain some of the problems I encountered. I don't plan on giving too much general information on this type of port however because much has already been written on the subject.
      Be sure to check those links out, if you haven't already:
      http://pythonhosted.org/six/
      http://python3porting.com/noconv.html#noconv-chapter
      https://docs.djangoproject.com/en/dev/topics/python3/
      http://lukeplant.me.uk/blog/posts/now-its-your-turn/

      And look at what others have done ! Studying how the Django developers did it was especially enlightening, to me at least.

      Delete
  4. Nice! I would love to move my Flask stuff all onto Python 3. Have you done any performance testing? Looking forward to your next article.

    ReplyDelete