replace nevow with twisted.web.template #1963

New Issue

zooko · 2013-05-03T21:41:24Z

zooko commented

2013-05-03 21:41:24 +00:00

Users still occasionally have problems with the nevow dependency:

https://github.com/twisted/nevow/issues/7
(see #2032)

Also, Nevow is unmaintained -- there has been no release since 2009. But it turns out (thanks to Daira for pointing this out to me) that twisted.web.template is the successor to nevow!

http://twistedmatrix.com/documents/current/web/howto/twisted-templates.html

So, we can reduce our dependencies by one, by using twisted.web.template instead of Nevow!

Users still occasionally have problems with the nevow dependency: <https://github.com/twisted/nevow/issues/7> (see #2032) Also, Nevow is unmaintained -- there has been no release since 2009. But it turns out (thanks to Daira for pointing this out to me) that `twisted.web.template` is the successor to nevow! <http://twistedmatrix.com/documents/current/web/howto/twisted-templates.html> So, we can reduce our dependencies by one, by using `twisted.web.template` instead of Nevow!

zooko added the

labels 2013-05-03 21:41:24 +00:00

zooko added this to the undecided milestone 2013-05-03 21:41:24 +00:00

daira commented

2013-05-03 22:26:24 +00:00

[the module name]corrected

tahoe-lafs changed title from ~~replace nevow with twisted.web.templates~~ to replace nevow with twisted.web.template

2013-05-03 22:26:24 +00:00

tahoe-lafs added

code-frontend-web

and removed

unknown

labels 2013-05-03 22:26:59 +00:00

nejucomo commented

2013-07-20 20:36:48 +00:00

I've filed #2032 specifically for the installation issue, which I consider separate from this dependency issue.

daira commented

2014-01-06 04:03:10 +00:00

I made a start on a branch to replace Nevow with twisted.web[.template] at https://github.com/tahoe-lafs/tahoe-lafs/commits/1963-replace-nevow.

I made a start on a branch to replace Nevow with `twisted.web[.template]` at <https://github.com/tahoe-lafs/tahoe-lafs/commits/1963-replace-nevow>.

tahoe-lafs modified the milestone from undecided to eventually

2014-01-06 04:06:15 +00:00

remyroy commented

2014-03-14 18:55:25 +00:00

I'll have a look at this.

daira commented

2014-03-15 15:05:25 +00:00

The idea behind the branch is to first perform a refactoring that means we still use Nevow, but we import its code indirectly via allmydata.web.common.

I also added a dummy @renderer decorator that will be needed for twisted.web (see this doc), but currently does nothing.

Once it's been made to work that way, we can switch the imports in allmydata.web.common to the equivalent twisted.web.template ones in cases where Twisted has a direct equivalent, and add new classes that inherit from or delegate to the Twisted ones to replace any Nevow-specific functionality.

The current state of the branch is that most of the imports in non-test code have been changed, and the @renderer decorator has been added in some of the cases where twisted.web would need it.

The idea behind the branch is to first perform a refactoring that means we still use Nevow, but we import its code indirectly via `allmydata.web.common`. I also added a dummy `@renderer` decorator that will be needed for `twisted.web` (see [this doc](http://twistedmatrix.com/documents/current/api/twisted.web.template.renderer.html)), but currently does nothing. Once it's been made to work that way, we can switch the imports in `allmydata.web.common` to the equivalent `twisted.web.template` ones in cases where Twisted has a direct equivalent, and add new classes that inherit from or delegate to the Twisted ones to replace any Nevow-specific functionality. The current state of the branch is that most of the imports in non-test code have been changed, and the `@renderer` decorator has been added in *some* of the cases where `twisted.web` would need it.

daira commented

2014-03-15 15:07:12 +00:00

BTW, please feel free to approach this differently if you see a better way of doing it, @remyroy.

daira commented

2014-03-15 15:09:42 +00:00

Incidentally, pyflakes is your friend when checking consistency of imports.

Incidentally, `pyflakes` is your friend when checking consistency of imports.

remyroy commented

2014-03-17 21:00:34 +00:00

From my understanding, Nevow is more than just templating. Replacing Nevow with twisted.web.template is not going to work if the goal is to totally remove Nevow. It would not make much sense to replace just the templating part of Nevow and keep the rest of it.

The big question is what part of Nevow are we using that is not templating and can twisted.web fulfill this in an easy and straightforward way?

This is what we are using from Nevow that is not templating related:

nevow.appserver
nevow.appserver.NevowRequest
nevow.appserver.NevowSite
nevow.resource_filename

and whatever is under those including idioms and conventions from Nevow. There seems to be a lot of Nevow code under those from my quick reading of Nevow source code.

I've been reading the twisted.web documentation and it feels quite different from how it's done with Nevow. twisted.web.template might have been inspired by the templating system from Nevow but the rest of twisted.web not so much. I also feel like twisted.web is a third world "web framework" coming from something like Flask.

If we are to replace Nevow with something quite different, I would feel uncomfortable to replace it with twisted.web . I'd prefer to use a battle tested modern minimalist WSGI based "web framework" with twisted.web.wsgi.WSGIResource . I like Flask but I know there are also some nice alternatives. We could still keep using the twisted.web.template to minimize the amount of changes with templates if it makes sense.

I'll keep investigating how twisted.web can be plugged in and check out if there are other big projects who made the switch from Nevow.

Some feedback on this would be appreciated.

Sorry for the long comment.

From my understanding, Nevow is more than just templating. Replacing Nevow with twisted.web.template is not going to work if the goal is to totally remove Nevow. It would not make much sense to replace just the templating part of Nevow and keep the rest of it. The big question is what part of Nevow are we using that is not templating and can twisted.web fulfill this in an easy and straightforward way? This is what we are using from Nevow that is not templating related: * nevow.appserver * nevow.appserver.NevowRequest * nevow.appserver.NevowSite * nevow.resource_filename and whatever is under those including idioms and conventions from Nevow. There seems to be a lot of Nevow code under those from my quick reading of Nevow source code. I've been reading the twisted.web documentation and it feels quite different from how it's done with Nevow. twisted.web.template might have been inspired by the templating system from Nevow but the rest of twisted.web not so much. I also feel like twisted.web is a third world "web framework" coming from something like Flask. If we are to replace Nevow with something quite different, I would feel uncomfortable to replace it with twisted.web . I'd prefer to use a battle tested modern minimalist WSGI based "web framework" with twisted.web.wsgi.WSGIResource . I like Flask but I know there are also some nice alternatives. We could still keep using the twisted.web.template to minimize the amount of changes with templates if it makes sense. I'll keep investigating how twisted.web can be *plugged in* and check out if there are other *big projects* who made the switch from Nevow. Some feedback on this would be appreciated. Sorry for the long comment.

zooko commented

2014-03-17 21:18:41 +00:00

A big desideratum would be to not add new dependencies to Tahoe-LAFS, and in fact to remove dependencies if possible. Twisted is the one major dependency that we know we can't live without, so it is okay to depend on all features of Twisted, but it is a big issue to add a dependency on any other library.

Someone who knows a lot about this topic is dreid! I'll see if I can get him to comment.

A big desideratum would be to not add new dependencies to Tahoe-LAFS, and in fact to remove dependencies if possible. Twisted is the one major dependency that we know we can't live without, so it is okay to depend on all features of Twisted, but it is a big issue to add a dependency on any other library. Someone who knows a lot about this topic is dreid! I'll see if I can get him to comment.

daira commented

2014-03-17 21:24:58 +00:00

Replying to remyroy:

The big question is what part of Nevow are we using that is not templating and can twisted.web fulfill this in an easy and straightforward way?

Yes.

This is what we are using from Nevow that is not templating related:

nevow.appserver

nevow.appserver.NevowRequest

nevow.appserver.NevowSite

nevow.resource_filename

and whatever is under those including idioms and conventions from Nevow. There seems to be a lot of Nevow code under those from my quick reading of Nevow source code.

I've been reading the twisted.web documentation and it feels quite different from how it's done with Nevow.

I think you're probably overestimating the differences. We actually only use a small subset of Nevow, and that subset has almost direct equivalents in twisted.web.

Replying to [remyroy](/tahoe-lafs/trac-2024-07-25/issues/1963#issuecomment-91642): > The big question is what part of Nevow are we using that is not templating and can twisted.web fulfill this in an easy and straightforward way? Yes. > This is what we are using from Nevow that is not templating related: > > * nevow.appserver > * nevow.appserver.NevowRequest > * nevow.appserver.NevowSite > * nevow.resource_filename > > and whatever is under those including idioms and conventions from Nevow. There seems to be a lot of Nevow code under those from my quick reading of Nevow source code. > > I've been reading the twisted.web documentation and it feels quite different from how it's done with Nevow. I think you're probably overestimating the differences. We actually only use a small subset of Nevow, and *that subset* has almost direct equivalents in `twisted.web`.

remyroy commented

2014-03-17 21:27:03 +00:00

Replying to [daira]comment:17:

I think you're probably overestimating the differences. We actually only use a small subset of Nevow, and that subset has almost direct equivalents in twisted.web.

Great, I'll keep digging deeper.

Replying to [daira]comment:17: > I think you're probably overestimating the differences. We actually only use a small subset of Nevow, and *that subset* has almost direct equivalents in `twisted.web`. Great, I'll keep digging deeper.

dreid commented

2014-03-17 22:01:53 +00:00

Last I checked tahoe-lafs made extensive use of nevow:pattern a concept which is not supported by twisted.web.template.

The other differences, like locateChild vs getChild and render_* being able to return Deferreds are relatively minor.

Last I checked tahoe-lafs made extensive use of `nevow:pattern` a concept which is not supported by twisted.web.template. The other differences, like locateChild vs getChild and render_* being able to return Deferreds are relatively minor.

remyroy commented

2014-03-18 20:45:34 +00:00

I'm sorry but this ticket is overwhelming me. I'll pass it back to Daira.

I wish I could be of more help.

I'm sorry but this ticket is overwhelming me. I'll pass it back to Daira. I wish I could be of more help.

daira commented

2014-03-18 23:43:44 +00:00

Replying to remyroy:

I'm sorry but this ticket is overwhelming me. I'll pass it back to Daira.

No problem; it's better to pass it to someone else than to be overwhelmed. (It was quite a complicated ticket to start with.)

Replying to [remyroy](/tahoe-lafs/trac-2024-07-25/issues/1963#issuecomment-91647): > I'm sorry but this ticket is overwhelming me. I'll pass it back to Daira. No problem; it's better to pass it to someone else than to be overwhelmed. (It was quite a complicated ticket to start with.)

daira commented

2014-03-18 23:49:38 +00:00

Replying to dreid:

Last I checked tahoe-lafs made extensive use of nevow:pattern a concept which is not supported by twisted.web.template.

Hmm, can't nevow:pattern be replaced with nevow:render (with some code refactoring obviously)? Or is that also unsupported?

[yes render is supported.]Edit:

Replying to [dreid](/tahoe-lafs/trac-2024-07-25/issues/1963#issuecomment-91646): > Last I checked tahoe-lafs made extensive use of `nevow:pattern` a concept which is not supported by twisted.web.template. Hmm, can't `nevow:pattern` be replaced with `nevow:render` (with some code refactoring obviously)? Or is that also unsupported? [yes `render` is supported.]Edit:

daira commented

2014-03-18 23:50:51 +00:00

Or nevow:slot maybe (which is also supported)?

Or `nevow:slot` maybe (which is also supported)?

daira commented

2014-03-18 23:52:06 +00:00

(http://twistedmatrix.com/trac/ticket/5040) seems relevant.

daira commented

2014-03-19 14:31:47 +00:00

This is the doc for the feature that was added to twisted.web.template in place of nevow:pattern: http://twistedmatrix.com/trac/changeset/32840

This is the doc for the feature that was added to `twisted.web.template` in place of `nevow:pattern`: <http://twistedmatrix.com/trac/changeset/32840>

zooko commented

2014-06-23 21:37:04 +00:00

The two motivations for this ticket in the initial comment were https://github.com/twisted/nevow/issues/7 and that Nevow hasn't had a new release since 2009. As of recently, those two things are no longer true: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2032#comment:18

Should we close this ticket now, or is there another reason to switch from Nevow to twisted.web.template?

The two motivations for this ticket in the initial comment were <https://github.com/twisted/nevow/issues/7> and that `Nevow` hasn't had a new release since 2009. As of recently, those two things are no longer true: <https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2032#comment:18> Should we close this ticket now, or is there another reason to switch from `Nevow` to `twisted.web.template`?

daira commented

2014-07-23 19:21:24 +00:00

Even though Nevow has a new release, it still isn't being actively developed. I would prefer to remove it as a dependency.

daira commented

2014-07-24 00:56:26 +00:00

Also we can't actually use the Nevow 0.11.1 release on Windows due to it requiring Twisted >= 13.0, which triggers #2028.